Hello everyone!
I want to use TVM for my iOS project. I have successfully built TVM Metal model and TVM Metal runtime and verified that it’s working. So now it’s time for tuning.
Right now I’m at the commit 03c8a6f46 and unfortunately, at this commit tuning of iOS Metal model doesn’t work, because dylib with kernel arrives without export symbols. So far I have fixed an issue by changing the 448 line at measure_methods.py:
func = build(s, args, target=task.target, target_host="llvm -mtriple=arm64-apple-darwin")
This change did fix an issue but right now I’m encountering another one. 80% of kernels variations compile with error. Here’s an output of python tuning algorithm:
Tuning...
Tune tasks...
[Task 1/48] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/1000) | 0.00 s[16:37:29] /Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/metal/metal_device_api.mm:152: Intializing Metal device 0, name=AMD Radeon Pro 5300M
[16:37:29] /Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/metal/metal_device_api.mm:152: Intializing Metal device 1, name=Intel(R) UHD Graphics 630
Error message = Traceback (most recent call last):
[bt] (8) 9 libtvm.dylib 0x0000000145a41116 tvm::runtime::RPCWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const + 1334
[bt] (7) 8 libtvm.dylib 0x0000000145a39d2c tvm::runtime::RPCClientSession::CallFunc(void*, TVMValue const*, int const*, int, std::__1::function<void (tvm::runtime::TVMArgs)> const&) + 124
[bt] (6) 7 libtvm.dylib 0x0000000145a31dad tvm::runtime::RPCEndpoint::CallFunc(void*, TVMValue const*, int const*, int, std::__1::function<void (tvm::runtime::TVMArgs)>) + 333
[bt] (5) 6 libtvm.dylib 0x0000000145a3071e tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::__1::function<void (tvm::runtime::TVMArgs)>) + 622
[bt] (4) 5 libtvm.dylib 0x0000000145a309be tvm::runtime::RPCEndpoint::EventHandler::HandleNextEvent(bool, bool, std::__1::function<void (tvm::runtime::TVMArgs)>) + 494
[bt] (3) 4 libtvm.dylib 0x0000000145a34969 tvm::runtime::RPCEndpoint::EventHandler::HandleProcessPacket(std::__1::function<void (tvm::runtime::TVMArgs)>) + 393
[bt] (2) 3 libtvm.dylib 0x0000000145a36915 tvm::runtime::RPCEndpoint::EventHandler::HandleReturn(tvm::runtime::RPCCode, std::__1::function<void (tvm::runtime::TVMArgs)>) + 213
[bt] (1) 2 libtvm.dylib 0x00000001447f80f9 tvm::runtime::detail::LogFatal::Entry::Finalize() + 89
[bt] (0) 1 libtvm.dylib 0x00000001459dabf8 tvm::runtime::Backtrace() + 24
[bt] (8) 9 tvmrpc 0x0000000104d3bebc std::__1::__function::__func<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13, std::__1::allocator<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13>, void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 88
[bt] (7) 8 tvmrpc 0x0000000104d3d500 std::__1::__function::__alloc_func<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13, std::__1::allocator<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13>, void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 88
[bt] (6) 7 tvmrpc 0x0000000104d3d564 void std::__1::__invoke_void_return_wrapper<void>::__call<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 88
[bt] (5) 6 tvmrpc 0x0000000104d3d5f4 decltype(std::__1::forward<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&>(fp)(std::__1::forward<tvm::runtime::TVMArgs>(fp0), std::__1::forward<tvm::runtime::TVMRetValue*>(fp0))) std::__1::__invoke<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 96
[bt] (4) 5 tvmrpc 0x0000000104d3d7e8 tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const + 468
[bt] (3) 4 tvmrpc 0x0000000104c4656c tvm::runtime::detail::LogFatal::~LogFatal() + 32
[bt] (2) 3 tvmrpc 0x0000000104c47a38 tvm::runtime::detail::LogFatal::~LogFatal() + 80
[bt] (1) 2 tvmrpc 0x0000000104c8b868 tvm::runtime::detail::LogFatalImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 120
[bt] (0) 1 tvmrpc 0x0000000104c5b028 tvm::runtime::Backtrace() + 28
[bt] (8) 9 tvmrpc 0x0000000104c5cf34 tvm::runtime::Module::GetFunction(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) + 68
[bt] (7) 8 tvmrpc 0x0000000104c50754 tvm::runtime::ModuleNode::GetFunction(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) + 144
[bt] (6) 7 tvmrpc 0x0000000104c88e2c tvm::runtime::MetalModuleNode::GetFunction(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&) + 1248
[bt] (5) 6 tvmrpc 0x0000000104c89310 tvm::runtime::MetalWrappedFunc::Init(tvm::runtime::MetalModuleNode*, tvm::runtime::ObjectPtr<tvm::runtime::Object>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long, unsigned long, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) + 248
[bt] (4) 5 tvmrpc 0x0000000104cfcdf8 tvm::runtime::MetalModuleNode::GetPipelineState(unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 2888
[bt] (3) 4 tvmrpc 0x0000000104c4656c tvm::runtime::detail::LogFatal::~LogFatal() + 32
[bt] (2) 3 tvmrpc 0x0000000104c47a38 tvm::runtime::detail::LogFatal::~LogFatal() + 80
[bt] (1) 2 tvmrpc 0x0000000104c8b868 tvm::runtime::detail::LogFatalImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 120
[bt] (0) 1 tvmrpc 0x0000000104c5b028 tvm::runtime::Backtrace() + 28
File "/Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/../../../src/runtime/metal/metal_module.mm", line 148
File "/Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/rpc/rpc_endpoint.cc", line 376
RPCError: Error caught from RPC call:
[16:38:12] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/../../../src/runtime/library_module.cc:78:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: ret == 0 (-1 vs. 0) : TVMError:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: (state != nil) is false: cannot get state: for function default_function_kernel0Compiler encountered an internal error
[Task 1/48] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (1/1000) | 50.20 s[16:38:17] /Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/metal/metal_device_api.mm:152: Intializing Metal device 0, name=AMD Radeon Pro 5300M
[16:38:17] /Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/metal/metal_device_api.mm:152: Intializing Metal device 1, name=Intel(R) UHD Graphics 630
[Task 1/48] Current/Best: 6.16/ 6.16 GFLOPS | Progress: (2/1000) | 61.89 s[16:38:29] /Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/metal/metal_device_api.mm:152: Intializing Metal device 0, name=AMD Radeon Pro 5300M
[16:38:29] /Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/metal/metal_device_api.mm:152: Intializing Metal device 1, name=Intel(R) UHD Graphics 630
Error message = Traceback (most recent call last):
[bt] (8) 9 libtvm.dylib 0x000000013e971116 tvm::runtime::RPCWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const + 1334
[bt] (7) 8 libtvm.dylib 0x000000013e969d2c tvm::runtime::RPCClientSession::CallFunc(void*, TVMValue const*, int const*, int, std::__1::function<void (tvm::runtime::TVMArgs)> const&) + 124
[bt] (6) 7 libtvm.dylib 0x000000013e961dad tvm::runtime::RPCEndpoint::CallFunc(void*, TVMValue const*, int const*, int, std::__1::function<void (tvm::runtime::TVMArgs)>) + 333
[bt] (5) 6 libtvm.dylib 0x000000013e96071e tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::__1::function<void (tvm::runtime::TVMArgs)>) + 622
[bt] (4) 5 libtvm.dylib 0x000000013e9609be tvm::runtime::RPCEndpoint::EventHandler::HandleNextEvent(bool, bool, std::__1::function<void (tvm::runtime::TVMArgs)>) + 494
[bt] (3) 4 libtvm.dylib 0x000000013e964969 tvm::runtime::RPCEndpoint::EventHandler::HandleProcessPacket(std::__1::function<void (tvm::runtime::TVMArgs)>) + 393
[bt] (2) 3 libtvm.dylib 0x000000013e966915 tvm::runtime::RPCEndpoint::EventHandler::HandleReturn(tvm::runtime::RPCCode, std::__1::function<void (tvm::runtime::TVMArgs)>) + 213
[bt] (1) 2 libtvm.dylib 0x000000013d7280f9 tvm::runtime::detail::LogFatal::Entry::Finalize() + 89
[bt] (0) 1 libtvm.dylib 0x000000013e90abf8 tvm::runtime::Backtrace() + 24
[bt] (8) 9 tvmrpc 0x0000000104d3bebc std::__1::__function::__func<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13, std::__1::allocator<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13>, void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 88
[bt] (7) 8 tvmrpc 0x0000000104d3d500 std::__1::__function::__alloc_func<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13, std::__1::allocator<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13>, void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 88
[bt] (6) 7 tvmrpc 0x0000000104d3d564 void std::__1::__invoke_void_return_wrapper<void>::__call<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 88
[bt] (5) 6 tvmrpc 0x0000000104d3d5f4 decltype(std::__1::forward<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&>(fp)(std::__1::forward<tvm::runtime::TVMArgs>(fp0), std::__1::forward<tvm::runtime::TVMRetValue*>(fp0))) std::__1::__invoke<tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) + 96
[bt] (4) 5 tvmrpc 0x0000000104d3d7e8 tvm::runtime::WrapPackedFunc(int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_13::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const + 468
[bt] (3) 4 tvmrpc 0x0000000104c4656c tvm::runtime::detail::LogFatal::~LogFatal() + 32
[bt] (2) 3 tvmrpc 0x0000000104c47a38 tvm::runtime::detail::LogFatal::~LogFatal() + 80
[bt] (1) 2 tvmrpc 0x0000000104c8b868 tvm::runtime::detail::LogFatalImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 120
[bt] (0) 1 tvmrpc 0x0000000104c5b028 tvm::runtime::Backtrace() + 28
[bt] (8) 9 tvmrpc 0x0000000104c5cf34 tvm::runtime::Module::GetFunction(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) + 68
[bt] (7) 8 tvmrpc 0x0000000104c50754 tvm::runtime::ModuleNode::GetFunction(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) + 144
[bt] (6) 7 tvmrpc 0x0000000104c88e2c tvm::runtime::MetalModuleNode::GetFunction(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&) + 1248
[bt] (5) 6 tvmrpc 0x0000000104c89310 tvm::runtime::MetalWrappedFunc::Init(tvm::runtime::MetalModuleNode*, tvm::runtime::ObjectPtr<tvm::runtime::Object>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long, unsigned long, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) + 248
[bt] (4) 5 tvmrpc 0x0000000104cfcdf8 tvm::runtime::MetalModuleNode::GetPipelineState(unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 2888
[bt] (3) 4 tvmrpc 0x0000000104c4656c tvm::runtime::detail::LogFatal::~LogFatal() + 32
[bt] (2) 3 tvmrpc 0x0000000104c47a38 tvm::runtime::detail::LogFatal::~LogFatal() + 80
[bt] (1) 2 tvmrpc 0x0000000104c8b868 tvm::runtime::detail::LogFatalImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 120
[bt] (0) 1 tvmrpc 0x0000000104c5b028 tvm::runtime::Backtrace() + 28
File "/Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/../../../src/runtime/metal/metal_module.mm", line 148
File "/Users/superyevhen/Documents/Work/TVM/tvm/src/runtime/rpc/rpc_endpoint.cc", line 376
RPCError: Error caught from RPC call:
[16:39:25] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/../../../src/runtime/library_module.cc:78:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: ret == 0 (-1 vs. 0) : TVMError:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: (state != nil) is false: cannot get state: for function default_function_kernel0Compiler encountered an internal error
And here’s log from iOS RPC application:
**2021-06-02 16:37:23.528936+0300 tvmrpc[3602:777013] Closing the streams.**
**2021-06-02 16:37:23.529388+0300 tvmrpc[3602:777013] (null)**
**2021-06-02 16:37:23.533090+0300 tvmrpc[3602:777013] Connecting to the proxy server..**
**Data = // Function: default_function_kernel0**
**MTLB**
**Source =**
**2021-06-02 16:37:31.932892+0300 tvmrpc[3602:777013] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/TVMRuntime.mm:172: Load module from /private/var/mobile/Containers/Data/Application/FEC372BE-B9BD-4005-905E-B630F3F85168/tmp/tmp_func_c7368d63ef4a5c67.dylib ...**
**2021-06-02 16:37:32.315861+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:37:32.380866+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:37:32.444970+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:37:32.445110+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**2021-06-02 16:37:42.565492+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:37:52.687714+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:38:02.806710+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:38:02.806859+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**Break!**
**2021-06-02 16:38:13.547388+0300 tvmrpc[3602:777013] Closing the streams.**
**2021-06-02 16:38:13.548283+0300 tvmrpc[3602:777013] (null)**
**2021-06-02 16:38:13.549652+0300 tvmrpc[3602:777013] Connecting to the proxy server..**
**Data = // Function: default_function_kernel0**
**MTLB**
**Source =**
**2021-06-02 16:38:20.097440+0300 tvmrpc[3602:777013] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/TVMRuntime.mm:172: Load module from /private/var/mobile/Containers/Data/Application/FEC372BE-B9BD-4005-905E-B630F3F85168/tmp/tmp_func_b78dbc082e75650f.dylib ...**
**2021-06-02 16:38:25.234332+0300 tvmrpc[3602:777013] Closing the streams.**
**2021-06-02 16:38:25.235164+0300 tvmrpc[3602:777013] (null)**
**2021-06-02 16:38:25.236392+0300 tvmrpc[3602:777013] Connecting to the proxy server..**
**2021-06-02 16:38:31.974683+0300 tvmrpc[3602:777013] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/TVMRuntime.mm:172: Load module from /private/var/mobile/Containers/Data/Application/FEC372BE-B9BD-4005-905E-B630F3F85168/tmp/tmp_func_922e7931c9334a98.dylib ...**
**2021-06-02 16:38:32.773445+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:38:33.323942+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:38:43.915279+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:38:43.915400+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**2021-06-02 16:38:54.513391+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:39:05.107621+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:39:15.701324+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:39:15.701442+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**Break!**
**2021-06-02 16:39:25.970109+0300 tvmrpc[3602:777013] Closing the streams.**
**2021-06-02 16:39:25.970586+0300 tvmrpc[3602:777013] (null)**
**2021-06-02 16:39:25.971306+0300 tvmrpc[3602:777013] Connecting to the proxy server..**
**Data = // Function: default_function_kernel0**
**MTLB**
**Source =**
**2021-06-02 16:39:33.899861+0300 tvmrpc[3602:777013] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/TVMRuntime.mm:172: Load module from /private/var/mobile/Containers/Data/Application/FEC372BE-B9BD-4005-905E-B630F3F85168/tmp/tmp_func_23cfbb8c01c1f5ad.dylib ...**
**2021-06-02 16:39:34.345900+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:39:36.577257+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:39:46.822424+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:39:46.822604+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**2021-06-02 16:39:57.056330+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:40:07.291296+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:40:17.521644+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:40:17.521779+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**Break!**
**2021-06-02 16:40:27.800141+0300 tvmrpc[3602:777013] Closing the streams.**
**2021-06-02 16:40:27.800890+0300 tvmrpc[3602:777013] (null)**
**2021-06-02 16:40:27.802087+0300 tvmrpc[3602:777013] Connecting to the proxy server..**
**Data = // Function: default_function_kernel0**
**MTLB**
**Source =**
**2021-06-02 16:40:34.901491+0300 tvmrpc[3602:777013] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/TVMRuntime.mm:172: Load module from /private/var/mobile/Containers/Data/Application/FEC372BE-B9BD-4005-905E-B630F3F85168/tmp/tmp_func_401a94359d842d2b.dylib ...**
**2021-06-02 16:40:35.233289+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:40:38.336323+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:40:48.438654+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:40:48.438766+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**2021-06-02 16:40:58.536411+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:41:08.640303+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:41:18.742174+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:41:18.742333+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**Break!**
**2021-06-02 16:41:29.092676+0300 tvmrpc[3602:777013] Closing the streams.**
**2021-06-02 16:41:29.093456+0300 tvmrpc[3602:777013] (null)**
**2021-06-02 16:41:29.094701+0300 tvmrpc[3602:777013] Connecting to the proxy server..**
**Data = // Function: default_function_kernel0**
**MTLB**
**Source =**
**2021-06-02 16:41:36.082303+0300 tvmrpc[3602:777013] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/TVMRuntime.mm:172: Load module from /private/var/mobile/Containers/Data/Application/FEC372BE-B9BD-4005-905E-B630F3F85168/tmp/tmp_func_834868fa9d670794.dylib ...**
**2021-06-02 16:41:36.595263+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:41:39.917592+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:41:50.236118+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:41:50.236256+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**2021-06-02 16:42:00.571810+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:42:10.906321+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:42:21.228655+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:42:21.228777+0300 tvmrpc[3602:777013] MTLCompiler: Compilation failed with XPC_ERROR_CONNECTION_INTERRUPTED on 3 try**
**Break!**
**2021-06-02 16:42:31.498133+0300 tvmrpc[3602:777013] Closing the streams.**
**2021-06-02 16:42:31.498951+0300 tvmrpc[3602:777013] (null)**
**2021-06-02 16:42:31.500220+0300 tvmrpc[3602:777013] Connecting to the proxy server..**
**Data = // Function: default_function_kernel0**
**MTLB**
**Source =**
**2021-06-02 16:42:39.472802+0300 tvmrpc[3602:777013] /Users/superyevhen/Documents/Work/TVM/tvm/apps/ios_rpc/tvmrpc/TVMRuntime.mm:172: Load module from /private/var/mobile/Containers/Data/Application/FEC372BE-B9BD-4005-905E-B630F3F85168/tmp/tmp_func_d87cb6929bc6076c.dylib ...**
**2021-06-02 16:42:40.237023+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
**2021-06-02 16:42:41.348591+0300 tvmrpc[3602:777013] Compiler failed with XPC_ERROR_CONNECTION_INTERRUPTED**
As you can see, only one kernel contributed to the tuning process and others fail. So what I did to investigate an issue deeper - I have copied metallib with the kernel that fails and test it independently. And here’s what I have got when I have tried to init id<MTLComputePipelineState>:
**ERROR = Threadgroup memory size (53504) exceeds the maximum threadgroup memory allowed (32768)**
I believe there are some conditions that kernels scheduler doesn’t take into account, so my question is - where do I go from here?
I’d be happy to fix issues myself and contribute to the repo with pull requests. Thank you in advance!