OpenCl code extraction

Questions
1

I am running this simple example in order to better understand how the tvm works . I am extracting the opencl code as you can see but in the output file some parts are missing. The part that is missing is the host code that is required for the data transaction and the device setup. Is there a way that i can extract the hole code? What I want is to export a file that i could use to run without the tvm dependencies. Hope it makes sense
PS.: I think that this task is performed somehow with the tvm.graph_runtime.create() but I am not sure.

out_channels = 16
data = sym.Variable(name="data")
simple_net = sym.conv2d(data=data, kernel_size=(3,3), channels=out_channels, padding = (1, 1), use_bias=True)
simple_net = sym.batch_norm(data=simple_net)
simple_net = sym.relu(data=simple_net)

batch_size = 1
data_shape = (batch_size, 3, 224, 224)
net, params = utils.create_workload(simple_net, batch_size, data_shape[1:])
logging.basicConfig(filename='simple_net.log',level=logging.DEBUG)

target = "opencl"

with nnvm.compiler.build_config(opt_level=0):
    graph, lib, params = nnvm.compiler.build(
    net,target_host='llvm',
        target=target, shape={"data": data_shape}, params=params)


try:
    f = open('simple_net_graph_ir.txt', 'w')
except IOError:
    cprint("Could not create file to dump Graph IR: " + f, 'red')
f.write(graph.ir())
f.close()

try:
    f = open('simple_net_opencl_code.txt', 'w')
except IOError:
    cprint("Could not create file to dump code: " + f, 'red')
if target == "llvm":
    f.write(lib.get_source())
else:
    f.write(lib.imported_modules[0].get_source())
f.close()
2

Host code is directly embeded in LLVM IR. That is why you need ‘llvm’ for target_host.

If you do lib.imported_modules[0].get_source(), you get opencl kernel. if you do lib.get_source(), you get llvm host side code. OpenCL kernel maybe used without TVM, but host side code is strongly tied to TVM so it is not useful without TVM.

1 Like
3

I tried this script and got an error:

ValueError: Direct host side access to device memory is detected in fuse_conv2d. Did you forget to bind?

Did you see this?

Thanks

4

@lixiaoquan I havent seen it, it runs on my machine.Here is the hole code maybe something is missing on the block that I posted.

import numpy as np import sys import tvm import nnvm.compiler from tvm.contrib import graph_runtime as runtime import logging import nnvm.symbol as sym from nnvm.testing import utils

def main():

out_channels = 16
data = sym.Variable(name="data")
simple_net = sym.conv2d(data=data, kernel_size=(3,3), channels=out_channels, padding = (1, 1), use_bias=True)
simple_net = sym.batch_norm(data=simple_net)
simple_net = sym.relu(data=simple_net)

batch_size = 1
data_shape = (batch_size, 3, 224, 224)
net, params = utils.create_workload(simple_net, batch_size, data_shape[1:])
logging.basicConfig(filename='simple_net.log',level=logging.DEBUG)

target = "opencl"

with nnvm.compiler.build_config(opt_level=0):
    graph, lib, params = nnvm.compiler.build(
    net,target_host='llvm',
        target=target, shape={"data": data_shape}, params=params)


try:
    f = open('simple_net_graph_ir.txt', 'w')
except IOError:
    cprint("Could not create file to dump Graph IR: " + f, 'red')
f.write(graph.ir())
f.close()

try:
    f = open('simple_net_llvm_code.txt', 'w')
except IOError:
    cprint("Could not create file to dump code: " + f, 'red')
if target == "llvm":
    f.write(lib.get_source())
else:
    f.write(lib.get_source())
    f.write(lib.imported_modules[0].get_source())
f.close()


'''
ctx = tvm.context(target, 0)
data = np.random.uniform(-1, 1, size=data_shape).astype("float32")
module = runtime.create(graph, lib, ctx)
module.set_input(**params)
module.set_input("data", data)
module.run()
out_shape = (batch_size, out_channels, 224, 224)
out = module.get_output(0, tvm.nd.empty(out_shape))
out_cuda = out.asnumpy()

'''

return

if name == ‘main’: main()

@masahi thank you for your reply. The code that is generated is from the codegen functions that are here codegen?Also since we are talking about just raw data shouldnt it be detached from tvm? or am I missing something?

5

CPU side codegen is done in this file. I don’t understand your second question. What do you mean by “raw data”?

6

I mean that the host side of the code does the data transmission from the cpu to the gpu and visa versa + the memory and hardware setting (or at least this is what is needed in order to run an openCL kernel in general). This is why I cant understand the tvm bondage

7

are you aware of this file? maybe this is what you are looking for.

8

Yes i think it is but, could you guide me also where to look in order to understand why the llvm host code is bonded with the tvm?

9

are you asking why we use llvm for host side code, instead of spitting out standalone opencl host code? I think the answer is, by using llvm for host code, we can use the same host code generator for all backend. We don’t want to implement different host code generator for CUDA or OpenCL.

10

I see,you made things a bit easier to understand! Thank you very much :slight_smile:

11

Could you please recommend ways to debug the execution of a DNN model using OpenCL in TVM ? For example, the top-1 result is correct using CUDA, but totally wrong using OpenCL.

Thank you very much!