ROCM target fails with LLVM ERROR

Troubleshooting
#1

While compiling mxnet resnet18 model with rocm target seeing below error

LLVM ERROR: Unknown specifier in datalayout string

The same issue is reported here. I am using following configuration

  1. set(USE_LLVM llvm-config-9)
  2. set(USE_ROCM on)
  3. ROCM Version 4.3.0
  4. MI100 GPU

I have verified rocm app form here. It works as expected.

(vitis-ai-tensorflow) Vitis-AI /workspace/HIP/samples/0_Intro/square > ./square.out 
info: running on device Device 738c
info: allocate host mem (  7.63 MB)
info: allocate device mem (  7.63 MB)
info: copy Host2Device
info: launch 'vector_square' kernel
info: copy Device2Host
info: check result
PASSED!

@masahi Do you have any suggestions for resolving this issue?

#2

Yes, I met this issue recently with rocm 4.3. I tried various LLVM versions up to 12, but what worked at the end was to use LLVM shipped as part of rocm install. But their LLVM fork is using some out-dated API which was changed recently in the upstream, so I had to modify codegen_amdgpu.cc and codegen_nvptx.cc to build TVM.

Probably using the recently released LLVM 13 would also work.

#3

Thanks @masahi for the info.

I have tried building tvm with LLVM 13 version but seeing below error.

/opt/tvm-vai/tvm/src/target/llvm/codegen_llvm.cc: In member function ‘virtual llvm::Value* tvm::codegen::CodeGenLLVM::CreateIntrinsic(const tvm::tir::CallNode*)’:
/opt/tvm-vai/tvm/src/target/llvm/codegen_llvm.cc:875:35: error: ‘getBaseName’ is not a member of ‘llvm::Intrinsic’; did you mean ‘getName’?
  875 |               << llvm::Intrinsic::getBaseName(id).str();
      |                                   ^~~~~~~~~~~
      |                                   getName
#4

This is the patch to I made to workaround the compile error: https://github.com/apache/tvm/compare/main...masahi:build-fix-rocm-llvm?expand=1

#5

@masahi Tried with your branch but still seeing errors

/workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_nvptx.cc: In member function ‘virtual llvm::Value* tvm::codegen::CodeGenNVPTX::CreateIntrinsic(const tvm::tir::CallNode*)’:
/workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_nvptx.cc:242:71: error: no matching function for call to ‘llvm::IRBuilder<>::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*&, llvm::Value*&, llvm::AtomicOrdering)’
  242 |                                        llvm::AtomicOrdering::Monotonic);
      |                                                                       ^
In file included from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/llvm_common.h:52,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_llvm.h:49,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_nvptx.cc:30:
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note: candidate: ‘llvm::AtomicRMWInst* llvm::IRBuilderBase::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*, llvm::Value*, llvm::MaybeAlign, llvm::AtomicOrdering, llvm::SyncScope::ID)’
 1757 |   AtomicRMWInst *CreateAtomicRMW(AtomicRMWInst::BinOp Op, Value *Ptr,
      |                  ^~~~~~~~~~~~~~~
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note:   candidate expects 6 arguments, 4 provided
/workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_nvptx.cc:253:69: error: no matching function for call to ‘llvm::IRBuilder<>::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*&, llvm::Value*&, llvm::AtomicOrdering)’
  253 |                                      llvm::AtomicOrdering::Monotonic);
      |                                                                     ^
In file included from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/llvm_common.h:52,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_llvm.h:49,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_nvptx.cc:30:
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note: candidate: ‘llvm::AtomicRMWInst* llvm::IRBuilderBase::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*, llvm::Value*, llvm::MaybeAlign, llvm::AtomicOrdering, llvm::SyncScope::ID)’
 1757 |   AtomicRMWInst *CreateAtomicRMW(AtomicRMWInst::BinOp Op, Value *Ptr,
      |                  ^~~~~~~~~~~~~~~
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note:   candidate expects 6 arguments, 4 provided
/workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_amdgpu.cc: In member function ‘virtual llvm::Value* tvm::codegen::CodeGenAMDGPU::CreateIntrinsic(const tvm::tir::CallNode*)’:
/workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_amdgpu.cc:193:74: error: no matching function for call to ‘llvm::IRBuilder<>::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*&, llvm::Value*&, llvm::AtomicOrdering)’
  193 |                                           llvm::AtomicOrdering::Monotonic);
      |                                                                          ^
In file included from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/llvm_common.h:52,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_llvm.h:49,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_amdgpu.cc:32:
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note: candidate: ‘llvm::AtomicRMWInst* llvm::IRBuilderBase::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*, llvm::Value*, llvm::MaybeAlign, llvm::AtomicOrdering, llvm::SyncScope::ID)’
 1757 |   AtomicRMWInst *CreateAtomicRMW(AtomicRMWInst::BinOp Op, Value *Ptr,
      |                  ^~~~~~~~~~~~~~~
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note:   candidate expects 6 arguments, 4 provided
/workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_amdgpu.cc:204:71: error: no matching function for call to ‘llvm::IRBuilder<>::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*&, llvm::Value*&, llvm::AtomicOrdering)’
  204 |                                        llvm::AtomicOrdering::Monotonic);
      |                                                                       ^
In file included from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/llvm_common.h:52,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_llvm.h:49,
                 from /workspace/external/tvm/examples/tvm-mashi/src/target/llvm/codegen_amdgpu.cc:32:
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note: candidate: ‘llvm::AtomicRMWInst* llvm::IRBuilderBase::CreateAtomicRMW(llvm::AtomicRMWInst::BinOp, llvm::Value*, llvm::Value*, llvm::MaybeAlign, llvm::AtomicOrdering, llvm::SyncScope::ID)’
 1757 |   AtomicRMWInst *CreateAtomicRMW(AtomicRMWInst::BinOp Op, Value *Ptr,
      |                  ^~~~~~~~~~~~~~~
/usr/include/llvm-13/llvm/IR/IRBuilder.h:1757:18: note:   candidate expects 6 arguments, 4 provided
CMakeFiles/tvm_objs.dir/build.make:5605: recipe for target 'CMakeFiles/tvm_objs.dir/src/target/llvm/codegen_nvptx.cc.o' failed
make[2]: *** [CMakeFiles/tvm_objs.dir/src/target/llvm/codegen_nvptx.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/tvm_objs.dir/build.make:5521: recipe for target 'CMakeFiles/tvm_objs.dir/src/target/llvm/codegen_amdgpu.cc.o' failed
make[2]: *** [CMakeFiles/tvm_objs.dir/src/target/llvm/codegen_amdgpu.cc.o] Error 1
CMakeFiles/Makefile2:174: recipe for target 'CMakeFiles/tvm_objs.dir/all' failed
make[1]: *** [CMakeFiles/tvm_objs.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2
#6

You are using the upstream llvm-13. My patch is meant to be used with rocm’s fork (which is based on llvm 13 but using outdated API, -DLLVM_DIR=/opt/rocm/llvm/lib/cmake/llvm). If you are using the upstream one, main branch should compile. Have you tried that?

#7

If this is using the upstream llvm-13 and this is the only error with main, just replace getBAseName with getName as suggested by the compiler message. Or copy paste the change from my branch:

#8

By using main branch with upstream llvm-13 and changing getBaseName to getName it is working as expected. Thank you @masahi for your inputs.