ValueError: Direct host side access to device memory is detected in fused_vision_get_valid_counts

Application
#1

ValueError Traceback (most recent call last)
in
10
11 with relay.build_config(opt_level=3):
—> 12 graph, lib, params = relay.build(func, target, params=params, target_host=target_host)
13
14 # After relay.build, get three return values: graph,library and the new parameter,

~/Env/tvm/python/tvm/relay/build_module.py in build(func, target, target_host, params)
303 graph_json, lowered_funcs, params = graph_gen.codegen(func)
304 mod = _tvm_build_module(
–> 305 lowered_funcs, target=target, target_host=target_host)
306 return graph_json, mod, params
307

~/Env/tvm/python/tvm/build_module.py in build(inputs, args, target, target_host, name, binds)
615 device_modules = []
616 for tar, flist in target_flist.items():
–> 617 fhost, mdev = _build_for_device(flist, tar, target_host)
618 # Save the current lowered functions of the host and the device module.
619 fhost_all += fhost

~/Env/tvm/python/tvm/build_module.py in _build_for_device(flist, target, target_host)
444 raise ValueError(
445 "Direct host side access to device memory is detected in %s. "
–> 446 “Did you forget to bind?” % func.name)
447 if func.func_type == container.LoweredFunc.MixedFunc:
448 if current_build_config().detect_global_barrier:

ValueError: Direct host side access to device memory is detected in fused_vision_get_valid_counts. Did you forget to bind?

I have run this command in my device. what does it mean forget to bind？

python -m tvm.exec.rpc_server --host 0.0.0.0 --port=9090

and it successfully returns:

INFO:RPCServer:bind to 0.0.0.0:9090
#2

What model are you trying to run? Could you provide your script?

#3

@Laurawly

I’m having the same error using metal on ios. Code is here: https://github.com/dmlc/tvm/issues/3093, metal is set to True

ValueError                                Traceback (most recent call last)
<ipython-input-7-d0af4c19f06c> in <module>()
      2 
      3 # only the cpu version
----> 4 graph, lib, params = compile(TARGET)

<ipython-input-1-8622b22ab0c8> in compile(target)
     48     with relay.build_config(opt_level=3):
     49         if args.metal:
---> 50             graph, lib, params = relay.build(net, target='metal', target_host=target, params=params)
     51         else:
     52             graph, lib, params = relay.build(net, target, params=params)

~/github/tvm/python/tvm/relay/build_module.py in build(func, target, target_host, params)
    303         graph_json, lowered_funcs, params = graph_gen.codegen(func)
    304         mod = _tvm_build_module(
--> 305             lowered_funcs, target=target, target_host=target_host)
    306     return graph_json, mod, params
    307 

~/github/tvm/python/tvm/build_module.py in build(inputs, args, target, target_host, name, binds)
    615     device_modules = []
    616     for tar, flist in target_flist.items():
--> 617         fhost, mdev = _build_for_device(flist, tar, target_host)
    618         # Save the current lowered functions of the host and the device module.
    619         fhost_all += fhost

~/github/tvm/python/tvm/build_module.py in _build_for_device(flist, target, target_host)
    444             raise ValueError(
    445                 "Direct host side access to device memory is detected in %s. "
--> 446                 "Did you forget to bind?" % func.name)
    447         if func.func_type == container.LoweredFunc.MixedFunc:
    448             if current_build_config().detect_global_barrier:

ValueError: Direct host side access to device memory is detected in fused_vision_get_valid_counts. Did you forget to bind?
#4

@KayneWest As far as I know, metal on ios is not fully tested yet. @tqchen please correct me if I’m wrong.

#5

This is my full script，please check for me

%matplotlib inline

import tvm
from matplotlib import pyplot as plt
from tvm.relay.testing.config import ctx_list
from tvm import relay
from tvm.contrib import util, graph_runtime as runtime
from tvm.contrib.download import download_testdata
from gluoncv import model_zoo, data, utils
import numpy as np
from tvm import rpc

get the model

block = model_zoo.get_model(‘ssd_512_resnet50_v1_voc’, pretrained=True)

model_name = “ssd_512_resnet50_v1_voc”
dshape = (1, 3, 512, 512)
dtype = “float32”

target = tvm.target.mali()
target_host = ‘llvm -target=aarch64-linux-gnu’
device_key = ‘rk3399’
use_android = False

im_fname = download_testdata(‘https://github.com/dmlc/web-data/blob/master/’ +
‘gluoncv/detection/street_small.jpg?raw=true’,
‘street_small.jpg’, module=‘data’)
x, img = data.transforms.presets.ssd.load_test(im_fname, short=512)

shape_dict = {‘data’: dshape}
net, params = relay.frontend.from_mxnet(block, shape_dict)

with relay.build_config(opt_level=3):
graph, lib, params = relay.build(net, target, params=params, target_host=target_host)

tmp = util.tempdir()
lib_fname = tmp.relpath(‘net.tar’)
lib.export_library(lib_fname)

local_demo = False
if local_demo:
remote = rpc.LocalSession()
else:
# The following is my environment, change this to the IP address of your target device
host = ‘192.168.100.177’
port = 9090
remote = rpc.connect(host, port)

upload the library to remote device and load it

remote.upload(lib_fname)
rlib = remote.load_module(‘net.tar’)

create the remote runtime module

ctx = remote.cl(0) if not local_demo else remote.cpu(0)
module = runtime.create(graph, rlib, ctx)

set parameter (upload params to the remote device. This may take a while)

module.set_input(**params)

set input data

tvm_input = tvm.nd.array(x.asnumpy(), ctx=ctx)
module.set_input(‘data’, tvm_input)

run

module.run()

get outputs

class_IDs, scores, bounding_boxs = module.get_output(0), module.get_output(1), module.get_output(2)

ax = utils.viz.plot_bbox(img, bounding_boxs.asnumpy()[0], scores.asnumpy()[0],
class_IDs.asnumpy()[0], class_names=block.classes)
plt.show()

#6

@petit-ami Your settings are same as mine. Did you use commit on or after a706ad16f83d810f4a6268cf604cece79c2c4791?

#7

what do you mean by this?

#8

Did you have this merged PR (https://github.com/dmlc/tvm/pull/2784) when you run experiments?

#9

I think I have solve this problem, It’s because the way how I build the model

#10

Hi, @Laurawly @petit-ami @KayneWest what is the Mac machine requirements to install tvm?

I have Mac mini with 500GB storage, 4GB RAM, and Intel Core i5 processor. is it possible to install tvmon my Mac mini?

OS: MacOS Mojave 10.14.2
Processor: Intel Core i5 2.5 GHz
Graphics: Intel HD Graphics 4000 1536 MB
RAM: 4GB DDR3
Storage: 500GB SATA

#11

@myproject24There shouldn’t be any problem. You can just follow the installation doc: https://docs.tvm.ai/install/from_source.html#build-the-shared-library if you want to build from source. I installed on my mac pro notebook and there’s no problem.

#12

@Laurawly thank you for your response. I will try to install.

#13

Hi @Laurawly, I am trying to run tvm iOS-rpc sample application from tvm/app/ios-rpc for metal CPU I ran python -m tvm.exec.rpc_proxy and it successfully returns INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
INFO:root:RPCProxy: Websock port bind to 8888

  • My doubt is what is key for Mac CPU in sample app it is key=“iPhone” for CPU?
  • TVM_IOS_RPC_PROXY_HOST is by default it is 0.0.0.0, whether I need to change is to my system IP?
  • What is TVM_IOS_CODESIGN? How to create it?

Note: I am not using iOS device I am trying it for Mac CPU with metal.

I am facing this error when I tried to run the sample application from tvm.

File “/Users/sfacecita/Mallappa/tvm/src/runtime/rpc/rpc_socket_impl.cc”, line 80
TVMError: URL 0.0.0.0:9090 cannot find server that matches key=client:iphone