What is the meaning of tvm.cpu(0) and tvm.cpu(1)?

popojames · January 25, 2022, 5:15am

Hello TVM developers and community,

I have seen tvm.cpu() in tvm.runtime document.

I wondering what is the difference when calling tvm.cpu(0) and tvm.cpu(1)

For example, I am using Hikey970 which contains 4 ARM-A53 LITTLE cores and 4 ARM-A73 big cores. Does tvm.cpu(0) represent 4 LITTLE cores and tvm.cpu(1) represent 4 big cores?

Also, is it possible to check all available CPU device IDs? I noticed this comment:

github.com

apache/tvm/blob/6720d3593d4dac6015418d4b7e9ad875bbf0b0a2/tests/python/relay/test_pipeline_executor.py#L203


        # Checking the pipeline executor runtime errors.
        with pytest.raises(RuntimeError):
            pipeline_module.set_params("param_0", None)


        with pytest.raises(RuntimeError):
            pipeline_module.set_params("param_1", customized_parameters)




def test_pipeline():
    if pipeline_executor.pipeline_executor_enabled():
        target_list = tvm.testing.enabled_targets()
        for target in target_list:
            # Get the three pipeline modules here.
            (mod1, mod2, mod3), dshape = get_mannual_mod()


            # Prepare batch data for pipeline computation.
            datas = []
            for i in range(5):
                datas.append(np.full(dshape, 3 + i).astype("float32"))


            pipe_config = pipeline_executor.PipelineConfig()

but when I tried it, it shows an error regarding CL_OUT_OF_HOST_MEMORY:

Thanks.

AndrewZhaoLuo · January 26, 2022, 6:15am

I do not know the answer, but @tkonolige might

umangyadav · January 26, 2022, 2:30pm

If you have dual socket CPU in your machine then with tvm.cpu(0) or tvm.cpu(1) you can target either first or second CPU.

I don’t think id in tvm.cpu(id) corresponds to the core count.

popojames · January 26, 2022, 3:36pm

Hello @umangyadav, Thanks for answering.

So for Hikey970 which contains 4 ARM-A53 LITTLE cores and 4 ARM-A73 big cores.

Does tvm.cpu(0) represent 4 LITTLE cores and tvm.cpu(1) represent 4 big cores?

umangyadav · January 26, 2022, 7:37pm

I don’t think so, I am not too much familiar with HiKey970. But it seems it has single SoC. So for both little and big cores, it should be tvm.cpu(0). I think @hogepodge may know better about what tvm.cpu(id) means.

hogepodge · January 27, 2022, 2:10am

Someone should correct me on this if I’m wrong, because I’m definitely not an authority, but it really depends on the device. As far as I can tell setting a device ID on a CPU device does nothing. When you compare the CPU runtime to the CUDA runtime, it looks like the device ID is unused.

popojames · January 27, 2022, 2:24am

Thanks for all the replies. Now It seems that changing them doesn’t affect my result. But I will try again and report here.

junrushao · January 27, 2022, 5:41am

@hogepodge is largely correct, in terms of checking how device_id is used in the corresponding device-api.

In our particular case, in CPUDeviceAPI, it doesnt affect anything other than the index of the memory pool. Usually sticking around with device_id=0 is good enough