[RFC][Runtime] Bring `PackedFunc` into TVM Object System

[junrushao]

To summarize our offline discussion with @areusch @tqchen.

Clarification:

1. This RFC doesn’t change any of the existing functionality, including C ABI or PackedFunc’s C++ API. Any modification to the C ABI is out of scope of this RFC.
2. Calling a PackedFunc inside TVM codebase directly uses the C++ API PackedFunc::operator() or CallPacked with C++ ABI, where there is no C ABI involved, which is a shortcut. For function calls across FFI boundary, our system uses C ABI instead, which this RFC doesn’t aim to change

There are a few levels that goes into this consideration:

• L0: Minimally we have the c_runtime_api.h that is the common ground of everything, and different kind of runtime impls
• L1: At C++ runtime there is a need of object system, which is now still specific to the C++, they are needed for flexibility reasons in the compiler side and sometimes complicated runtime(like VM).

It is not desirable to bring L1’s design considerations into L0 (specifically standards that L0 should follow), since L0 should really be kept stable for cases like embedded settings. What the proposal was talking about, we believe, is to streamline some of the L1 data structures, just like what we did for String. They would bring benefit to the implementations (use packed func as object). There is also “ABI” at L1 level but never made official.

To answer @areusch’s questions (with my understanding):

@areusch

it seems like this might be one of them. Given we can already pass PackedFunc as arguments to other PackedFunc and return them, it seems like one of the big changes here is making the TVM data structures more expressive. I can certainly think of some good use cases for storing function pointers in a map, but I wonder about passing such a data structure at an API boundary.

By making PackedFunc a TVM object, we are able to put them into TVM containers (Array, Map), which as Andrew said, makes TVM data structures more expressive. It does allow us to pass PackedFunc across the FFI boundary, because all the the existing mechanism is unchanged and still works. More particularly, calling a PackedFunc on the python side still uses the good old TVMFunctionHandle and the ABIs defined in c_runtime_api.

@areusch

Could you say what you mean/imply by “ABI stable” here? Are you separately compiling a library which needs to invoke PackedFunc?

As mentioned in the section “Unresolved questions”, here we only consider C++ ABI, and C ABI is left to future work.

@areusch

Could you elaborate? Would there be operator PackedFuncObj?

@cyx

> 
> ```

PackedFuncObj is not going to be user-facing, so users won’t have to deal with it any time (IIUC)

@areusch

Sort of the inverse of my last question–suppose I have e.g. a Map whose values are PackedFuncObj. How can I invoke the PackedFunc or reconstruct? The function pointer seems to be private in this definition.

For example, say the signature of the Map is: Map<String, PackedFunc> my_map. Users could invoke it via PackedFunc::operator(), i.e. this functionality is unchanged from the existing PackedFunc:

my_map["name"](arg1, arg2, arg3);