Can not run the model with Vulkan backend on Android

Lunderberg · June 1, 2021, 3:56pm

For this one, this looks like it isn’t finding the vulkan library when compiling. Since it is still included as a dependency in the makefile that cmake outputs, make thinks that it is supposed to create libvulkan.so, but doesn’t know how. I would have expected this to cause an issue earlier when running cmake, since the Vulkan_LIBRARY variable should require finding the correct location of libvulkan.so. After the changes you made to point to a different location, what did the cmake output look like?

The errors on both the Ubuntu and MacOS sides look like it is trying to link together object files targeting different architectures. From the path, the libvulkan.so you’re using is a 64-bit version. What does running uname -a output on your system, and is there any chance you’re using a 32-bit operating system?

Similar question here, what is the architecture of the MacOS system (x86_64 or M1), and what architecture does libvulkan.dylib use? There may be another mismatch there.

@csullivan When you compiled libtvm_runtime.so to run an rpc server on an Android device, were there any special changes you needed to make for it? Could you share the config.cmake file that you used in order to do so?

WildTaras · June 2, 2021, 5:23am

@Lunderberg Thank you very much for your detailed response. I have MacBook with Intel Core I7 on the board There are outputs of the command uname -a

MacBook:

Darwin MacBook-Pro-Mojo.local 19.5.0 Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64

Ubuntu:

Linux whoami-IP 5.8.0-53-generic #60~20.04.1-Ubuntu SMP Thu May 6 09:52:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

I made deep investigation and did the next movements:

Refactored Vulkan.cmake and FindVulkan.cmake. It means that I set all paths to the files for linking vulkan into tvm_runtime lib in a correct way. In my case it looks like:

include_directories(/Users/user/VulkanSDK/1.2.176.1/macOS/include/)
list(APPEND TVM_LINKER_LIBS /Users/user/VulkanSDK/1.2.176.1/macOS/lib/libSPIRV-Tools.a)
list(APPEND TVM_RUNTIME_LINKER_LIBS /Users/user/VulkanSDK/1.2.176.1/macOS/lib/libvulkan.so)

It must work, because all paths are set correctly, but I got the next error:

/Users/user/Library/Android/sdk/ndk/21.1.6352462/toolchains/llvm/prebuilt/darwin-x86_64/lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld: skipping incompatible /Users/user/VulkanSDK/1.2.176.1/macOS/lib/libvulkan.so when searching for -lvulkan

/Users/user/Library/Android/sdk/ndk/21.1.6352462/toolchains/llvm/prebuilt/darwin-x86_64/lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld: cannot find -lvulkan

I saw, that Cmakelists of tvm library try to link path of the libvulkan.so to tvm_runtime, I put the line of the code below:

target_link_libraries(tvm_runtime 
                      PRIVATE 
                      ${TVM_RUNTIME_LINKER_LIBS})

After that, I saw how original cmake implements FindVulkan.cmake. The main idea that it creates the module besides creating the variables with the paths. Sample of code I put below:

if(Vulkan_FOUND AND NOT TARGET Vulkan::Vulkan) add_library(Vulkan::Vulkan UNKNOWN IMPORTED) set_target_properties(Vulkan::Vulkan PROPERTIES IMPORTED_LOCATION “${Vulkan_LIBRARIES}” INTERFACE_INCLUDE_DIRECTORIES “${Vulkan_INCLUDE_DIRS}”)

So I try to create the module and link it to the tvm_runtime: I created the library in CmakeLists to make it global scope:

add_library(Vulkan_LIB SHARED IMPORTED)

After that, I link path to libvulkan.so to the Vukan_LIB library inside the Vulkan.cmake:

  set_target_properties(Vulkan_LIB PROPERTIES IMPORTED_LOCATION "${Vulkan_LIBRARY}"
                                    INTERFACE_INCLUDE_DIRECTORIES "/Users/user/VulkanSDK/1.2.176.1/macOS/include/")

And final step, link created Vulkan_LIB to the tvm_runtime inside CmakeLists:

target_link_libraries(tvm_runtime ${Vulkan_LIB})

But it also gives the error like it doesnt see libvulkan.so, because there are no implementation of different functions:

> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::DeleteHostVisibleBuffer(tvm::runtime::vulkan::VulkanHostVisibleBuffer*)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:76: undefined reference to `vkUnmapMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:76: undefined reference to `vkUnmapMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:79: undefined reference to `vkFreeMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:79: undefined reference to `vkFreeMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:82: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:82: undefined reference to `vkDestroyBuffer'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::FindMemoryType(tvm::runtime::vulkan::VulkanContext const&, VkBufferCreateInfo, unsigned int)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:138: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:138: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:141: undefined reference to `vkGetBufferMemoryRequirements'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:141: undefined reference to `vkGetBufferMemoryRequirements'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:144: undefined reference to `vkGetPhysicalDeviceMemoryProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:144: undefined reference to `vkGetPhysicalDeviceMemoryProperties'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::CreateBuffer(tvm::runtime::vulkan::VulkanContext const&, unsigned long, unsigned int, unsigned int)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:175: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:175: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:208: undefined reference to `vkAllocateMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:208: undefined reference to `vkAllocateMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:224: undefined reference to `vkBindBufferMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:224: undefined reference to `vkBindBufferMemory'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::VulkanDeviceAPI::GetDeviceDescription(VkInstance_T*, VkPhysicalDevice_T*, std::__ndk1::vector<char const*, std::__ndk1::allocator<char const*> > const&, std::__ndk1::vector<char const*, std::__ndk1::allocator<char const*> > const&)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:488: undefined reference to `vkGetPhysicalDeviceProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:488: undefined reference to `vkGetPhysicalDeviceProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:541: undefined reference to `vkGetPhysicalDeviceFeatures'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:541: undefined reference to `vkGetPhysicalDeviceFeatures'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:531: undefined reference to `vkGetInstanceProcAddr'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:531: undefined reference to `vkGetInstanceProcAddr'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:631: undefined reference to `tvm::IntImm::IntImm(tvm::runtime::DataType, long, tvm::Span)'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:643: undefined reference to `tvm::Target::Target(tvm::runtime::Map<tvm::runtime::String, tvm::runtime::ObjectRef, void, void> const&)'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `Bool':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/include/tvm/ir/expr.h:329: undefined reference to `tvm::IntImm::IntImm(tvm::runtime::DataType, long, tvm::Span)'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `Integer':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/include/tvm/ir/expr.h:366: undefined reference to `tvm::IntImm::IntImm(tvm::runtime::DataType, long, tvm::Span)'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::operator<<(std::__ndk1::basic_ostream<char, std::__ndk1::char_traits<char> >&, tvm::runtime::ObjectRef const&)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/include/tvm/node/repr_printer.h:70: undefined reference to `tvm::ReprPrinter::Print(tvm::runtime::ObjectRef const&)'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/include/tvm/node/repr_printer.h:70: undefined reference to `tvm::ReprPrinter::Print(tvm::runtime::ObjectRef const&)'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/include/tvm/node/repr_printer.h:70: undefined reference to `tvm::ReprPrinter::Print(tvm::runtime::ObjectRef const&)'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `operator()':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:723: undefined reference to `vkEnumerateInstanceLayerProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:723: undefined reference to `vkEnumerateInstanceLayerProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:751: undefined reference to `vkEnumerateInstanceExtensionProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:751: undefined reference to `vkEnumerateInstanceExtensionProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:771: undefined reference to `vkGetInstanceProcAddr'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:771: undefined reference to `vkGetInstanceProcAddr'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `VulkanDeviceAPI':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:799: undefined reference to `vkCreateInstance'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:799: undefined reference to `vkCreateInstance'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:803: undefined reference to `vkEnumeratePhysicalDevices'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:803: undefined reference to `vkEnumeratePhysicalDevices'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:825: undefined reference to `vkGetPhysicalDeviceProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:825: undefined reference to `vkGetPhysicalDeviceProperties'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `operator()':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:843: undefined reference to `vkEnumerateDeviceExtensionProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:843: undefined reference to `vkEnumerateDeviceExtensionProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:846: undefined reference to `vkEnumerateDeviceExtensionProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:846: undefined reference to `vkEnumerateDeviceExtensionProperties'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `VulkanDeviceAPI':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:920: undefined reference to `vkCreateDevice'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:920: undefined reference to `vkCreateDevice'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:924: undefined reference to `vkGetDeviceQueue'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:924: undefined reference to `vkGetDeviceQueue'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:941: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:941: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:942: undefined reference to `vkGetBufferMemoryRequirements'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:942: undefined reference to `vkGetBufferMemoryRequirements'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:943: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:943: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:947: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:947: undefined reference to `vkCreateBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:948: undefined reference to `vkGetBufferMemoryRequirements'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:948: undefined reference to `vkGetBufferMemoryRequirements'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:949: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:949: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:955: undefined reference to `vkGetPhysicalDeviceMemoryProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:955: undefined reference to `vkGetPhysicalDeviceMemoryProperties'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::VulkanDeviceAPI::GetComputeQueueFamilies(VkPhysicalDevice_T*)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1017: undefined reference to `vkGetPhysicalDeviceQueueFamilyProperties'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1017: undefined reference to `vkGetPhysicalDeviceQueueFamilyProperties'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::DeleteHostVisibleBuffer(tvm::runtime::vulkan::VulkanHostVisibleBuffer*)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:76: undefined reference to `vkUnmapMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:76: undefined reference to `vkUnmapMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:79: undefined reference to `vkFreeMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:79: undefined reference to `vkFreeMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:82: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:82: undefined reference to `vkDestroyBuffer'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::GetOrAllocate(int, unsigned long, unsigned int, unsigned int, std::__ndk1::unordered_map<unsigned long, std::__ndk1::unique_ptr<tvm::runtime::vulkan::VulkanHostVisibleBuffer, std::__ndk1::default_delete<tvm::runtime::vulkan::VulkanHostVisibleBuffer> >, std::__ndk1::hash<unsigned long>, std::__ndk1::equal_to<unsigned long>, std::__ndk1::allocator<std::__ndk1::pair<unsigned long const, std::__ndk1::unique_ptr<tvm::runtime::vulkan::VulkanHostVisibleBuffer, std::__ndk1::default_delete<tvm::runtime::vulkan::VulkanHostVisibleBuffer> > > > >*, bool)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1379: undefined reference to `vkMapMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1379: undefined reference to `vkMapMemory'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::VulkanStream::Synchronize()':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:140: undefined reference to `vkEndCommandBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:140: undefined reference to `vkEndCommandBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:156: undefined reference to `vkQueueSubmit'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:156: undefined reference to `vkQueueSubmit'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `~lock_guard':
> 
> /Users/user/Library/Android/sdk/ndk/21.1.6352462/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1/__mutex_base:(.text._ZN3tvm7runtime6vulkan12VulkanStream11SynchronizeEv[_ZN3tvm7runtime6vulkan12VulkanStream11SynchronizeEv]+0x39c): undefined reference to `vkWaitForFences'
> 
> /Users/user/Library/Android/sdk/ndk/21.1.6352462/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1/__mutex_base:(.text._ZN3tvm7runtime6vulkan12VulkanStream11SynchronizeEv[_ZN3tvm7runtime6vulkan12VulkanStream11SynchronizeEv]+0x3a0): undefined reference to `vkWaitForFences'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::VulkanStream::Synchronize()':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:164: undefined reference to `vkResetCommandBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:164: undefined reference to `vkResetCommandBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:165: undefined reference to `vkResetFences'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:165: undefined reference to `vkResetFences'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:173: undefined reference to `vkBeginCommandBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:173: undefined reference to `vkBeginCommandBuffer'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `VulkanStream':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:55: undefined reference to `vkCreateCommandPool'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:55: undefined reference to `vkCreateCommandPool'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:63: undefined reference to `vkAllocateCommandBuffers'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:63: undefined reference to `vkAllocateCommandBuffers'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:70: undefined reference to `vkCreateFence'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:70: undefined reference to `vkCreateFence'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:77: undefined reference to `vkBeginCommandBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:77: undefined reference to `vkBeginCommandBuffer'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::VulkanModuleNode::GetPipeline(unsigned long, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, unsigned long)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1143: undefined reference to `vkCreateShaderModule'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1143: undefined reference to `vkCreateShaderModule'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1217: undefined reference to `vkCreateDescriptorSetLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1217: undefined reference to `vkCreateDescriptorSetLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1229: undefined reference to `vkCreateDescriptorPool'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1229: undefined reference to `vkCreateDescriptorPool'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1238: undefined reference to `vkAllocateDescriptorSets'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1238: undefined reference to `vkAllocateDescriptorSets'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1262: undefined reference to `vkCreatePipelineLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1262: undefined reference to `vkCreatePipelineLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1279: undefined reference to `vkCreateComputePipelines'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1279: undefined reference to `vkCreateComputePipelines'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `~VulkanDeviceAPI':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan15VulkanDeviceAPID2Ev[_ZN3tvm7runtime6vulkan15VulkanDeviceAPID2Ev]+0x3c): undefined reference to `vkDestroyDevice'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan15VulkanDeviceAPID2Ev[_ZN3tvm7runtime6vulkan15VulkanDeviceAPID2Ev]+0x40): undefined reference to `vkDestroyDevice'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:239: undefined reference to `vkDestroyInstance'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:239: undefined reference to `vkDestroyInstance'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::VulkanDeviceAPI::FreeDataSpace(DLDevice, void*)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:263: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:263: undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:264: undefined reference to `vkFreeMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:264: undefined reference to `vkFreeMemory'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `tvm::runtime::vulkan::VulkanDeviceAPI::CopyDataFromTo(void const*, unsigned long, void*, unsigned long, unsigned long, DLDevice, DLDevice, DLDataType, void*)':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:351: undefined reference to `vkFlushMappedMemoryRanges'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:351: undefined reference to `vkFlushMappedMemoryRanges'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:333: undefined reference to `vkInvalidateMappedMemoryRanges'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:333: undefined reference to `vkInvalidateMappedMemoryRanges'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `operator()':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:295: undefined reference to `vkCmdCopyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:295: undefined reference to `vkCmdCopyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:305: undefined reference to `vkCmdPipelineBarrier'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:305: undefined reference to `vkCmdPipelineBarrier'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:322: undefined reference to `vkCmdCopyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:322: undefined reference to `vkCmdCopyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:363: undefined reference to `vkCmdPipelineBarrier'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:363: undefined reference to `vkCmdPipelineBarrier'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:371: undefined reference to `vkCmdCopyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:371: undefined reference to `vkCmdCopyBuffer'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `VulkanDescriptorTemplateKHRFunctions':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_common.h:105: undefined reference to `vkGetDeviceProcAddr'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_common.h:105: undefined reference to `vkGetDeviceProcAddr'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `VulkanGetBufferMemoryRequirements2Functions':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_common.h:124: undefined reference to `vkGetDeviceProcAddr'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_common.h:124: undefined reference to `vkGetDeviceProcAddr'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `~VulkanModuleNode':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyPipeline'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyPipelineLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyDescriptorPool'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyPipeline'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyPipelineLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyDescriptorPool'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyDescriptorSetLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1100: undefined reference to `vkDestroyDescriptorSetLayout'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan16VulkanModuleNodeD2Ev[_ZN3tvm7runtime6vulkan16VulkanModuleNodeD2Ev]+0x78): undefined reference to `vkDestroyShaderModule'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan16VulkanModuleNodeD2Ev[_ZN3tvm7runtime6vulkan16VulkanModuleNodeD2Ev]+0x7c): undefined reference to `vkDestroyShaderModule'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `~VulkanThreadEntry':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev[_ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev]+0xc0): undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev[_ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev]+0xc4): undefined reference to `vkFreeMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev[_ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev]+0xc8): undefined reference to `vkUnmapMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev[_ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev]+0xcc): undefined reference to `vkDestroyBuffer'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev[_ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev]+0xd0): undefined reference to `vkFreeMemory'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:(.text._ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev[_ZN3tvm7runtime6vulkan17VulkanThreadEntryD2Ev]+0xd4): undefined reference to `vkUnmapMemory'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `~VulkanStream':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:81: undefined reference to `vkDestroyFence'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:81: undefined reference to `vkDestroyFence'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:82: undefined reference to `vkDestroyCommandPool'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan_stream.h:82: undefined reference to `vkDestroyCommandPool'
> 
> CMakeFiles/tvm_runtime_objs.dir/src/runtime/vulkan/vulkan.cc.o: In function `operator()':
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1455: undefined reference to `vkCmdBindPipeline'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1455: undefined reference to `vkCmdBindPipeline'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1465: undefined reference to `vkCmdPushConstants'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1465: undefined reference to `vkCmdPushConstants'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1470: undefined reference to `vkCmdDispatch'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1470: undefined reference to `vkCmdDispatch'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1477: undefined reference to `vkCmdPipelineBarrier'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1477: undefined reference to `vkCmdPipelineBarrier'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1507: undefined reference to `vkUpdateDescriptorSets'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1507: undefined reference to `vkUpdateDescriptorSets'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1512: undefined reference to `vkCmdBindPipeline'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1512: undefined reference to `vkCmdBindPipeline'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1513: undefined reference to `vkCmdBindDescriptorSets'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1513: undefined reference to `vkCmdBindDescriptorSets'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1521: undefined reference to `vkCmdPushConstants'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1521: undefined reference to `vkCmdPushConstants'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1526: undefined reference to `vkCmdDispatch'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1526: undefined reference to `vkCmdDispatch'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1533: undefined reference to `vkCmdPipelineBarrier'
> 
> /Users/user/Documents/AlgofaceWork/TVM/tvm/src/runtime/vulkan/vulkan.cc:1533: undefined reference to `vkCmdPipelineBarrier'
> 
> clang++: **error:** linker command failed with exit code 1 (use -v to see invocation)

Could you please give me additional comments where I made mistake

WildTaras · June 1, 2021, 8:50pm

@csullivan if you will show config.cmake for compiling libtvm_runtime.so for Android device, could you please elaborate experience with linking Vulkan library. Did you change anything, which env variables did you set etc. I will be grateful for any help.

WildTaras · June 2, 2021, 5:58am

Also, I find out one weird thing, when I use function

find_library(Vulkan_LIBRARY NAMES vulkan vulkan-1 PATHS ${__vulkan_sdk}/lib)

it always return me Vulkan_LIBRARY-NOTFOUND. It seems, that there are no library at this path, but I made

message(STATUS "Vulkan path for checking" ${__vulkan_sdk})

and after that check with ls utility path of __vulkan_sdk + /lib and got next output:

   cmake                       libLLVMPassPrinters.a
    libclang.a                  libLLVMProfileData.a
    libclangAnalysis.a          libLLVMScalarOpts.a
    libclangAST.a               libLLVMSupport.a
    libclangASTMatchers.a       libLLVMTableGen.a
    libclangBasic.a             libLLVMTarget.a
    libclangCodeGen.a           libLLVMTransformUtils.a
    libclangDriver.a            libLLVMVectorize.a
    libclangEdit.a              libMachineIndependent.a
    libclangFormat.a            libOGLCompiler.a
    libclangFrontend.a          libOSDependent.a
    libclangFrontendTool.a      libshaderc.a
    libclangIndex.a             libshaderc_combined.a
    libclangLex.a               libshaderc_shared.so
    libclangParse.a             libshaderc_shared.so.1
    libclangRewrite.a           libshaderc_util.a
    libclangRewriteFrontend.a   libSPIRV.a
    libclangSema.a              libspirv-cross-c.a
    libclangSPIRV.a             libspirv-cross-core.a
    libclangTooling.a           libspirv-cross-cpp.a
    libclangToolingCore.a       libspirv-cross-c-shared.so
    libdxclib.a                 libspirv-cross-c-shared.so.0
    libdxcompiler.so            libspirv-cross-c-shared.so.0.45.0
    libdxcompiler.so.3.7        libspirv-cross-glsl.a
    libGenericCodeGen.a         libspirv-cross-hlsl.a
    libglslang.a                libspirv-cross-msl.a
    libHLSL.a                   libspirv-cross-reflect.a
    libLLVMAnalysis.a           libspirv-cross-util.a
    libLLVMAsmParser.a          libSPIRV-Tools.a
    libLLVMBitReader.a          libSPIRV-Tools-link.a
    libLLVMBitWriter.a          libSPIRV-Tools-opt.a
    libLLVMCore.a               libSPIRV-Tools-reduce.a
    libLLVMDxcSupport.a         libSPIRV-Tools-shared.so
    libLLVMDXIL.a               libSPVRemapper.a
    libLLVMDxilContainer.a      libVkLayer_api_dump.so
    libLLVMDxilPIXPasses.a      libVkLayer_demo_layer.so
    libLLVMDxilRootSignature.a  libVkLayer_device_simulation.so
    libLLVMDxrFallback.a        libVkLayer_gfxreconstruct.so
    libLLVMHLSL.a               libVkLayer_khronos_synchronization2.so
    libLLVMInstCombine.a        libVkLayer_khronos_validation.so
    libLLVMipa.a                libVkLayer_monitor.so
    libLLVMipo.a                libVkLayer_screenshot.so
    libLLVMIRReader.a           libVkLayer_starter_layer.so
    libLLVMLinker.a             libVkLayer_utils.a
    libLLVMLTO.a                libvulkan.so
    libLLVMMiniz.a              libvulkan.so.1
    libLLVMMSSupport.a          libvulkan.so.1.2.170
    libLLVMOption.a             pkgconfig
    libLLVMPasses.a

As you can see, libvulkan.so is existed. The same problem with Vulkan_SPIRV_TOOLS_LIBRARY. It is the reason, why I set these paths manually. Maybe this trouble can give more information about the global problem?

P.S. This problem exist both on macOS and Ubuntu. Above log I got from Ubuntu.

masahi · June 2, 2021, 7:42am

Can you compile and run other normal vulkan applications? I have export VULKAN_SDK=/home/masa/1.2.162.1/x86_64 in my bashrc and this is I all need to get vulkan working with TVM on ubuntu.

WildTaras · June 2, 2021, 8:33am

@masahi You know, you are completely right. Looks please, if I use:

cmake -DUSE_LLVM=llvm-config-8 \
      -DUSE_RPC=ON \
      -DUSE_SORT=ON \
      -DUSE_VULKAN=ON \
      -DUSE_GRAPH_EXECUTOR=ON \
      ..
make -j10

everything are working fine. It finds Vulkan with all necessary includes and libraries. I have compiled tvm_runtime library successfully. But, for Android version of tvm_runtime, I need to use this command line:

cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=armeabi-v7a -DANDROID_NATIVE_API_LEVEL=23 -DANDROID_ARM_NEON=ON ..

What I want to admit, when I use _DCMAKE_TOOLCHAIN_FILE with it’s value, It cause problems with Vulkan. I use this toolchain from android_ndk_r21e, because it was mentioned in this post:

gist.github.com

https://gist.github.com/apivovarov/c3d35e53103b25d625481d2bccf4f606

build-tvm-runtime-for-android-cmake.md

# Build TVM runtime for Android using cmake

#### Install cmake (for example 1.19.6)
https://cmake.org/download/

#### Download and extract NDK (e.g. r21e)
https://developer.android.com/ndk/downloads

#### Set env var NDK (chage the path to correct one)
```

This file has been truncated. show original

Could you please ask someone who has build tvm_runtime for Android, to give me a hint, which toolchain I need to use, or maybe it has another way to build?

masahi · June 2, 2021, 8:48am

I’m not familiar with android toolchain, but this README tvm/README.md at main · apache/tvm · GitHub suggests android-24 is the right one?

WildTaras · June 2, 2021, 9:06am

android-24 is version of SDK platform, but I need to use .cmake from ndk

WildTaras · June 2, 2021, 9:19am

Sorry, I will investigate that topic and post correct result of my work

WildTaras · June 2, 2021, 9:36pm

After my small investigation I decide to use the next command:

make -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-24 -DANDROID_NATIVE_API_LEVEL=23 -DANDROID_ARM_NEON=ON ..

but with small refactoring. It found Vulkan_LIBRARY by the path

Vulkan_LIBRARY=/home/whoami/android-ndk-r21e/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/24/libvulkan.so

so it means, that I need to set others paths in findVulkan.cmake manually to the downloaded VulkanSDK. It seems like:

Vulkan_SPIRV_TOOLS_LIBRARY=/whoami/vulkan/1.2.162.1/x86_64/lib/libSPIRV-Tools.a

and

Vulkan_INCLUDE_DIRS=/home/whoami/vulkan/1.2.162.1/x86_64/include/home/whoami/vulkan/1.2.162.1/x86_64/include/spirv-tools/home/whoami/vulkan/1.2.162.1/x86_64/include/spirv/unified1/home/whoami/vulkan/1.2.162.1/x86_64/include/spirv/unified1

Conclusion of all investigation is - tvm_runtime compilation for android should use libvulkan.so from android_ndk folder.

WildTaras · June 2, 2021, 11:29am

Dear @masahi, could you please me one more time, I successfully compile tvm_runtime with Vulkan and my model with Vulkan for Android deployment. But when I launch the program in runtime on Android device it crashes with the next log:

E/libc++abi: terminating with uncaught exception of type tvm::runtime::InternalError: [14:09:13] /home/whoami/tvm/src/runtime/vulkan/vulkan_stream.cc:137: 
    ---------------------------------------------------------------
    An error occurred during the execution of TVM.
    For more information, please see: https://tvm.apache.org/docs/errors.html
    ---------------------------------------------------------------
      Check failed: (__e == VK_SUCCESS) is false: Vulkan Error, code=-4: VK_ERROR_DEVICE_LOST
    Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

It occurs during the execution of next line of the code:

m_output.CopyToBytes(*data, size);

All my code on the next sample;

bool TVMhandler::initModel(AAssetManager * assetManager, const char* modelName) {

    Device dev {kDLVulkan, device_id};
    LOGE("before module");   
    m_mod = tvm::runtime::Module::LoadFromFile(modelName);
    LOGE("After module");
    Module def = m_mod.GetFunction("default")(dev);
    m_inputFunc = def.GetFunction("set_input");
    m_runFunc = def.GetFunction("run");
    m_outputFunc = def.GetFunction("get_output");

    m_input = NDArray::Empty({1, 3, 256, 256}, DLDataType{kDLFloat, 32, 1}, dev);
    m_output = NDArray::Empty({1, 209, 64, 64}, DLDataType{kDLFloat, 32, 1}, dev);

    LOGE("End of the init");
    return true;
}

bool TVMhandler::track(cv::Mat &input, float **data, int &size, int **shape) {

    cv::Mat blob = cv::dnn::blobFromImage(input, 1.0 / 255.0);

    m_input.CopyFromBytes(blob.data, blob.total()*blob.elemSize());

    m_inputFunc("input", m_input);
    LOGE("Before run");
    m_runFunc();
    LOGE("AFTER run");
    m_outputFunc(1, m_output);
    LOGE("After output");
    *shape = new int[4];

    (*shape)[0] = m_output->shape[0];
    (*shape)[1] =  m_output->shape[1];
    (*shape)[2] = m_output->shape[2];
    (*shape)[3] = m_output->shape[3];

    size = m_output->shape[0] * m_output->shape[1] * m_output->shape[2] * m_output->shape[3] * sizeof(float);
    *data = new float[size];
    LOGE("copy data");
    m_output.CopyToBytes(*data, size);
    LOGE("end of the queu");
    return true;
}

I would like to notice that above code works successfully with tvm_runtime and model were built with CPU, (of course instead of kDLVulkan - I use kDLCPU)

Could you give me please your suggestion, why it doesnt work. I will be very grateful.

masahi · June 2, 2021, 12:46pm

I think it should be m_output.CopyToBytes(*data, sizeof(float) * size);

WildTaras · June 2, 2021, 1:02pm

Look please into the code, I already made multiplication with a sizeof(float)

 size = m_output->shape[0] * m_output->shape[1] * m_output->shape[2] * m_output->shape[3] * sizeof(float);
    *data = new float[size];
    LOGE("copy data");
    m_output.CopyToBytes(*data, size);

WildTaras · June 2, 2021, 8:23pm

Additional information. I have looked into vulkan_stream.cc, place, where error was occured is

VULKAN_CALL(vkQueueSubmit(vctx_->queue, 1, &cb_submit, state_->fence_));

Maybe I need to set additional parameters, which for CPU version is not necessary?

WildTaras · June 3, 2021, 12:44pm

Hello @masahi, could you tell me please, how to set TVM_VULKAN_ENABLE_VALIDATION_LAYERS to enable validation layer in Vulkan for debugging the problem above?

csullivan · June 3, 2021, 9:53pm

Hi @WildTaras, a bit late, and it looks like you may have a working configuration, however replying nonetheless, here are the relevant lines from my cmake.config for cross compiling to android libtvm_runtime.so with OpenCL support.

set(USE_OPENCL /home/csullivan/octoml/android_libs)
set(USE_CPP_RPC ON)
set(CMAKE_TOOLCHAIN_FILE $ENV{ANDROID_NDK}/build/cmake/android.toolchain.cmake)
set(ANDROID_ABI "arm64-v8a")
set(ANDROID_PLATFORM android-28)
message(STATUS "Using toolchain file: ${CMAKE_TOOLCHAIN_FILE}.")

You mentioned above some confusion on which libvulkan.so to use when linking the cross compiled binary. I suspect you’ll want to use adb to extract the libvulkan.so from your device as described here. Not sure, but it is possible that the VK_ERROR_DEVICE_LOST error you see is the result of differences in the vulkan lib you linked against at compile time and that which is on the device.

masahi · June 3, 2021, 10:03pm

I think you can use export TVM_VULKAN_ENABLE_VALIDATION_LAYERS=1.

WildTaras · June 4, 2021, 10:09am

@csullivan Thank you, for your response. But when I took the libvulkan.so and compiled tvm_runtime - all the same, it gives me the error VK_ERROR_DEVICE_LOST. I would like to admit, that I have run tvm_runtime with openCL optimization for checking my way of using tvm library. It works! Tvm_runtime with opencl library works well. Do you have any additional suggestions?

perrryosa · February 7, 2023, 6:10am

Hi @WildTaras , can you please share the details about building the TVM with vulkan on android. I am getting the SPIRV related errors. Can you also share the tvm config file for building

perrryosa · February 20, 2023, 6:37am

Can you please point me the process of building tvm with vulkan for android?

I am facing multiple redefinition errors as well.