# Irrespective of input same output iOS TVM model **URL:** \ **Category:** Application\ **Created:** [August 11, 2021, 6:28am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765 "2021-08-11T06:28:40Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![myproject24](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/myproject24/32/2952_2.png) [@myproject24](https://discuss.tvm.apache.org/u/myproject24)\ **Post date:** [August 11, 2021, 6:28am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/1 "2021-08-11T06:28:40Z")
Hi TVM Team, @tqchen @preethireddy @shivmgg @echuraevI am developing an iOS TVM application but Irrespective of input same inference output I am getting in the iOS TVM model. 1. Converted TensorFlow model to TVM model - success 2. Tried to inference converted x86 TVM model in python - success 3. Again converted TF model to TVM model of iOS ARM64 - success 4. Tried to inference converted arm64 model in iOS - Irrespective of input same output not giving expected inference output. Below are the target and target\_host: target = ‘metal’ target\_host = “llvm -mtriple=arm64-apple-darwin” Note: If I set all input values to zero it is giving the same inference output. What will be the issue kindly help me to fix this issue. Thank you. ---
**Author:** ![elvin-n](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/elvin-n/32/2682_2.png) [@elvin-n](https://discuss.tvm.apache.org/u/elvin-n)\ **Post date:** [August 11, 2021, 11:09am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/2 "2021-08-11T11:09:29Z")
Ho do you export model? Do you point iphone sdk like ```auto from tvm.contrib import xcode arch = "arm64" sdk = "iphoneos" libpath = model_folder_path + "/" + model_name + ".dylib" lib.export_library(libpath, xcode.create_dylib, arch=arch, sdk=sdk) ``` > Tried to inference converted arm64 model in iOS - Irrespective of input same output not giving expected inference output. Do you do this through RPC or build in TVM/model to your application? If you do natively and run your own app on iOS, how do you feed data? Could you share this part of code? ---
**Author:** ![myproject24](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/myproject24/32/2952_2.png) [@myproject24](https://discuss.tvm.apache.org/u/myproject24)\ **Post date:** [August 16, 2021, 6:02am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/3 "2021-08-16T06:02:41Z")
Hi @elvin-n i have built a standalone iOS application For iOS CPU. - sdk = “iphoneos” - target = “llvm -mtriple=arm64-apple-darwin” For iOS Metal: - sdk = “iphoneos” - target = “metal” - target\_host = “llvm -mtriple=arm64-apple-darwin” And i used c++ to inference the TVM model // load module tvm::runtime::Module mod\_syslib = tvm::runtime::Module::LoadFromFile(std\_resourcePath); ``` //load graph std::ifstream json_in(std_jsonPath); std::string json_data((std::istreambuf_iterator(json_in)), std::istreambuf_iterator()); json_in.close(); // get global function module for graph runtime auto tvm_graph_runtime_create = tvm::runtime::Registry::Get("tvm.graph_executor.create"); tvm::runtime::Module mod = (*tvm_graph_runtime_create)(json_data, mod_syslib, device_type, device_id); this->m_handle = new tvm::runtime::Module(mod); //parameters needs to be TVMByteArray typr to indecate the binary data std::ifstream params_in(std_paramPath, std::ios::binary); std::string params_data((std::istreambuf_iterator(params_in)), std::istreambuf_iterator()); params_in.close(); TVMByteArray params; params.data = params_data.c_str(); params.size = params_data.length(); mod.GetFunction("load_params")(params); tvm::runtime::PackedFunc set_input = mod->GetFunction("set_input"); set_input("input", m_gpuInput); tvm::runtime::PackedFunc run = mod->GetFunction("run"); run(); tvm::runtime::PackedFunc get_output = mod->GetFunction("get_output"); ``` ---
**Author:** ![elvin-n](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/elvin-n/32/2682_2.png) [@elvin-n](https://discuss.tvm.apache.org/u/elvin-n)\ **Post date:** [August 16, 2021, 6:34am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/4 "2021-08-16T06:34:11Z")
which context do you use for creation of `m_gpuInput` and how do you populate data? BTW, you are loading params and json that means that you are using packed zip format for storing model. Is it more convinient for you? Does it find artefacts during the loading of model? I prefer to use everything been packed to one library created by `lib.export_library` ---
**Author:** ![myproject24](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/myproject24/32/2952_2.png) [@myproject24](https://discuss.tvm.apache.org/u/myproject24)\ **Post date:** [August 16, 2021, 7:40am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/5 "2021-08-16T07:40:19Z")
m\_gpuInput is in metal context **kDLMetal** and I bundled JSON, Params, and dylib separately with the application I feel it is more convenient to understand if any issue. ---
**Author:** ![elvin-n](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/elvin-n/32/2682_2.png) [@elvin-n](https://discuss.tvm.apache.org/u/elvin-n)\ **Post date:** [August 16, 2021, 7:56am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/6 "2021-08-16T07:56:04Z")
> [@myproject24](#): > > m\_gpuInput is in metal context **kDLMetal** How do you populate data to this NDArray? ---
**Author:** ![myproject24](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/myproject24/32/2952_2.png) [@myproject24](https://discuss.tvm.apache.org/u/myproject24)\ **Post date:** [August 16, 2021, 8:19am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/7 "2021-08-16T08:19:22Z")
``` //data is float* TVMArrayCopyFromBytes(m_cpuInput, data, w*h*c); TVMArrayCopyFromTo(m_cpuInput, m_gpuInput, nullptr); set_input("INPUT", m_gpuInput); get_output(0, m_gpuOutput0); TVMArrayCopyFromTo(m_gpuOutput0, m_cpuOutput0, nullptr); ``` ---
**Author:** ![elvin-n](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/elvin-n/32/2682_2.png) [@elvin-n](https://discuss.tvm.apache.org/u/elvin-n)\ **Post date:** [August 16, 2021, 1:08pm UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/8 "2021-08-16T13:08:32Z")
> [@myproject24](#): > > `w*h*c` missing of `*4` - is it a problem of copy-past or you forgot to compensate the size of the float? since TVMArrayCopyFromBytes deal with bytes, not floats. Another question why do you need m\_gpuInput? you can use only NDArray been allocated with CPU context. Copy will happen automatically inside the TVM runtime. ---
**Author:** ![myproject24](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/myproject24/32/2952_2.png) [@myproject24](https://discuss.tvm.apache.org/u/myproject24)\ **Post date:** [August 17, 2021, 2:45am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/9 "2021-08-17T02:45:46Z")
- Yes, it is W_H_C\*4. - And my understanding is we can’t directly access GPU memory so i copied all my input data to CPU memory and then copied to GPU memory please excuse me if my understanding was wrong. ---
**Author:** ![elvin-n](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/elvin-n/32/2682_2.png) [@elvin-n](https://discuss.tvm.apache.org/u/elvin-n)\ **Post date:** [August 17, 2021, 11:58am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/10 "2021-08-17T11:58:52Z")
in [MetalWorkspace::CopyDataFromTo](https://github.com/apache/tvm/blob/main/src/runtime/metal/metal_device_api.mm#L228) three situation is handled: 1. Copy from Metal to Metal 2. Copy from CPU to Metal 3. Copy from Metal to CPU I.e. in your case one more extra copy will be done and it will be much easier currently just to pass NDArray working with CPU context. Could you pass `m_cpuInput` to `set_input("INPUT", m_cpuInput);` and verify the result? Another difference - I looked in my code and figured out that I have not used `TVMArrayCopyFromTo`, but used function of NDArray. Like ```auto tvm::runtime::NDArray output = getOutput_(0); output.CopyTo(y_); ``` where output will be NDArray working with Metal context and y\_ - NDArray allocated for CPU context ---
**Author:** ![elvin-n](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/elvin-n/32/2682_2.png) [@elvin-n](https://discuss.tvm.apache.org/u/elvin-n)\ **Post date:** [August 19, 2021, 6:35am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/11 "2021-08-19T06:35:43Z")
@myproject24 did you have a chance to verify if passing of CPU NDArray to input and using of the other API for copy data helps to solve the problem or not? ---
**Author:** ![myproject24](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/myproject24/32/2952_2.png) [@myproject24](https://discuss.tvm.apache.org/u/myproject24)\ **Post date:** [August 19, 2021, 9:49am UTC](https://discuss.tvm.apache.org/t/irrespective-of-input-same-output-ios-tvm-model/10765/12 "2021-08-19T09:49:24Z")
Sorry i didn’t get time to check i will let you know once i verify it.