BYOC: Runtime access Tensor values

max1996 · September 2, 2020, 12:28pm

Hi, I am working on a BYOC backend and finally reached the runtime level. But I’ve looked at the DNNL JSON example (I am using JSON as well) and have not found a way to access the real tensor values to serialize them for my target. From the JSON serialization I can access the shapes, etc, but not the data.

Where do I have to look for that?

zhiics · September 2, 2020, 4:10pm

The inputs are passed through NDArray in the DNNL example. You can use nd_array->data to get the data pointer.

comaniac · September 2, 2020, 4:43pm

Here is an example of Zhi’s response. The expression data_entry_[eid]->data accesses the values of the tesnor.

github.com

apache/incubator-tvm/blob/master/src/runtime/contrib/dnnl/dnnl_json_runtime.cc#L71


  SetupConstants(consts);
}

void Run() override {
  // Fill in the input buffers.
  for (size_t i = 0; i < input_nodes_.size(); ++i) {
    auto eid = EntryID(input_nodes_[i], 0);
    // TODO(@comaniac): Support other data lengths.
    size_t offset_in_bytes = entry_out_mem_[eid].second * 4;
    size_t buffer_size = GetDataSize(*data_entry_[eid]);
    write_to_dnnl_memory(data_entry_[eid]->data, entry_out_mem_[eid].first, buffer_size,
                         offset_in_bytes);
  }

  // Invoke the engine through intepreting the stream.
  for (size_t i = 0; i < net_.size(); ++i) {
    net_.at(i).execute(stream_, net_args_.at(i));
  }
  stream_.wait();

  // Read output buffers.

max1996 · September 7, 2020, 9:43am

Ok, thank you, that works, but for some reason only for constant tensors.

The input tensors still have a nullpointer instead of the data field.

comaniac · September 7, 2020, 9:48pm

It should not happen because the example I pointed out was for data tensors.

Did you set inputs before running the graph runtime?

max1996 · September 8, 2020, 6:17am

Ah, I guess, I found my issue.

I looked into my code and did not a line, that initializes the input tensors. But the ACL and DNNL Json implementations do not contain anything like this as well.