[TFLite] [LLVM]: LLVM Error when compiling TFLite model

Troubleshooting
#1

Hi, I’m trying to compile a mobilenet v1_0.5_128 quantified TFLite model with TVM 0.12.0 and TVM 0.14.dev0 for raspb4 target and pynq target (Xilinx), but I keep getting this LLVM error: LLVM ERROR: Cannot select: intrinsic %llvm.aarch64.neon.uaddlp

Aborted

I am using LLVM 16, but I also tried building TVM with LLVM 15, LLVM 14 and LLVM 10 but keep getting the same error.

Note: I used to work with TVM 0.10.0 and did not face this error before, it was working well.

Here is my code, it is inspired by TVM tutorials:

import tvm
import tvm.relay as relay
from tvm.contrib.download import download_testdata
from PIL import Image
import numpy as np
from scipy.special import softmax

#define the target and the compiler
target= tvm.target.arm_cpu("rasp4b")

tflite_model_buf = open("mobilenet_v1_0.5_128_quant.tflite", "rb").read()

# Get TFLite model from buffer
try:
    import tflite

    tflite_model = tflite.Model.GetRootAsModel(tflite_model_buf, 0)
except AttributeError:
    import tflite.Model

    tflite_model = tflite.Model.Model.GetRootAsModel(tflite_model_buf, 0)


#get the test image
img_url = "https://s3.amazonaws.com/model-server/inputs/kitten.jpg"
img_path = download_testdata(img_url, "imagenet_cat.png", module="data")

# Resize it to 128x128
resized_image = Image.open(img_path).resize((128, 128))
img_data = np.asarray(resized_image).astype(np.uint8)

# Normalize according to the ImageNet input specification
img_data_final = np.expand_dims(img_data, axis=0)

#create the portable function func
shape_dict = {"input": img_data_final.shape}
dtype_dict = {"input": img_data_final.dtype.name}
mod, params = relay.frontend.from_tflite(tflite_model, shape_dict,dtype_dict)

#compile
with tvm.transform.PassContext(opt_level=3):
	lib = relay.build(mod, target, params=params)

# Save the library at local directory
lib.export_library("tflmodel.tar")
Error: Cannot select intrinsic
#2

It looks like TVM might currently use the 64-bit exclusive ARM intrinsic mentioned in your error message for some quantized convolutions if I understand the source code correctly. If this is the case, you probably need to use the rasp4b64 target and make sure that your system is set to 64-bit mode accordingly. Does your Pi4 run in 64-bit or 32-bit mode?

#3

Thank you for taking time to look at this issue @OoJJBoO . My Pi4 is on 32-bit mode. I wonder why I wasn’t getting the error when I was working with TVM v0.10 Do you have any idea how can I fix this?

#4

Hi @Hiba, it seems as though there was a subtle change in the schedule selection logic for arm_cpu between the versions of tvm.

Firstly, rasp4b translates to the following target:

"rasp4b": [
    "-model=bcm2711",
    "-mtriple=armv8l-linux-gnueabihf",
     "-mattr=+neon",
     "-mcpu=cortex-a72",
],

which does not declare support for aarch64, but does have support for +neon.

For v0.10.0, see this line that decides whether to use the "conv2d_NHWC_quantized_interleaved.arm_cpu" schedule:

if is_aarch64 and data.dtype in ["int8", "uint8"]:

and then this line which decides whether to use "conv2d_nhwc_spatial_pack.arm_cpu":

if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):

Since you are compiling without support for aarch64, the schedule that gets selected is "conv2d_nhwc_spatial_pack.arm_cpu".

Now for the current version of TVM, these conditions have changed to:

if has_asimd and data.dtype in ["int8", "uint8"]:

if (not has_asimd) or (data.dtype not in ["int8", "uint8"]):

see this line and this line. Where has_asimd is checking for the existence of the +neon feature. Therefore, "conv2d_NHWC_quantized_interleaved.arm_cpu" gets selected which contains calls the aarch64 intrinsics which your compile target does not support, hence the error message you are seeing.

As a temporary fix I’d suggest changing both the has_asimd conditions to use target.features.is_aarch64, which I believe should result in the same behaviour as in v0.10.0.

The suggestion above is by no means elegant, we should implement a proper fix for this. I just didn’t get to the bottom of the reason for these changes today.

Hope this helps!

1 Like
#6

Hi @lhutton1, thank you for taking time to look at this issue. It was indeed that “if” change that caused the problem. I tried to do as you suggested, the problem was fixed for the addition (uaddlp), but another problem arised with multiplication (smull). I then tried to also fix it by changing another if clause. Now I manage to compile my TFLite model, but with a bunch of warnings: /local/home/meznihib/tvm14/src/target/llvm/codegen_cpu.cc:1545: Warning: Unknown pragma pragma_import_c

I don’t know if this will affect my work later, for the model use or for compiling other non-TFLite models.

#7

Hi @Hiba, apologies, looks like there were a couple more issues here. I just posted https://github.com/apache/tvm/pull/15468 which should fix the issues you’re facing