# \[C++ RPC + AOT\] Error "Can not pass in local device" when deploying AOT-compiled model via C++ RPC client

**URL:** <https://discuss.tvm.apache.org/t/c-rpc-aot-error-can-not-pass-in-local-device-when-deploying-aot-compiled-model-via-c-rpc-client/18845>\
**Category:** Questions\
**Created:** [December 15, 2025, 9:33am UTC](https://discuss.tvm.apache.org/t/c-rpc-aot-error-can-not-pass-in-local-device-when-deploying-aot-compiled-model-via-c-rpc-client/18845 "2025-12-15T09:33:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![markwester](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/markwester/32/6096_2.png) [@markwester](https://discuss.tvm.apache.org/u/markwester)\
**Post date:** [December 15, 2025, 9:33am UTC](https://discuss.tvm.apache.org/t/c-rpc-aot-error-can-not-pass-in-local-device-when-deploying-aot-compiled-model-via-c-rpc-client/18845/1 "2025-12-15T09:33:38Z")

</div>

Hi TVM community,

I’m trying to deploy an **AOT-compiled graph executor model** using the **TVM C++ RPC API** , but I keep encountering a runtime error related to device handling. Unfortunately, documentation and examples for the **C++ RPC client usage** are very limited, so I’m seeking guidance.

#### Environment

- **Platform** : Linux (aarch64)

```bash
Linux lima-ubuntu 6.8.0-87-generic #88-Ubuntu SMP PREEMPT_DYNAMIC Sat Oct 11 09:16:38 UTC 2025 aarch64 aarch64 aarch64 GNU/Linux

```

- **TVM version** : `v0.12.0` (built from source)
- **RPC server** : Running via `./tvm_rpc server --host=0.0.0.0 --port=9000`

#### Model Compilation (Python)

I compiled a simple `add(x, y)` model in **AOT mode** with the C++ runtime:

```python
import tvm
from tvm import relay

x = relay.var("x", shape=(1, 3, 224, 224), dtype="float32")
y = relay.var("y", shape=(1, 3, 224, 224), dtype="float32")
z = relay.add(x, y)
func = relay.Function([x, y], z)

mod = tvm.IRModule.from_expr(func)
target = "llvm"

with tvm.transform.PassContext(opt_level=3):
    lib = relay.build(
        mod,
        target=tvm.target.Target(target),
        params={},
        runtime=relay.backend.Runtime("cpp"),
        executor=relay.backend.Executor("aot"),
    )

lib.export_library("aot_mod.so")

```

#### Error on C++ Client

When running my C++ client, I get the following error:

```auto
Error: [16:34:29] /path/to/tvm/src/runtime/rpc/rpc_module.cc:153:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
  Check failed: (IsRPCSessionDevice(dev)) is false: Can not pass in local device
Stack trace:
  0: tvm::runtime::RPCWrappedFunc::RemoveSessMask(DLDevice) const
        at /path/to/tvm/src/runtime/rpc/rpc_module.cc:153
  ...
  5: main
        at /path/to/rpc_deploy.cpp:148

```

#### C++ Client Code (simplified)

```cpp
// Connect to RPC server
auto connect_func = tvm::runtime::Registry::Get("rpc.Connect");
auto remote_session = (*connect_func)(server_ip, server_port, "", false);

// Upload and load module on remote
f_upload("/tmp/mymod.so", mod_data);
mod = f_load_module("/tmp/mymod.so");

// This line triggers the error:
DLDevice dev = {kDLCPU, 0};
tvm::runtime::Module gmod = mod.GetFunction("default")(dev); // ← ERROR HERE

```

#### Questions

1. When using **AOT + RPC** , should the `DLDevice` passed to `mod["default"]` be a **remote device**?  
But `DLDevice` is just a struct — how do I specify it belongs to the RPC session?

2. Is AOT + RPC fully supported in the C++ runtime? Or should I switch to **Graph Executor (non-AOT)** for RPC deployments?

Any working C++ RPC + AOT example would be extremely helpful!

Thanks in advance for your support!

---

<div class="post-metadata">

**Author:** ![markwester](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/markwester/32/6096_2.png) [@markwester](https://discuss.tvm.apache.org/u/markwester)\
**Post date:** [December 22, 2025, 2:40pm UTC](https://discuss.tvm.apache.org/t/c-rpc-aot-error-can-not-pass-in-local-device-when-deploying-aot-compiled-model-via-c-rpc-client/18845/2 "2025-12-22T14:40:58Z")

</div>

It works that using “rpc.SessTableIndex” and AddRPCSessionMask, but I don’t know whether it’s a correct way.

```c++
auto f_get_idx = tvm::runtime::Registry::Get("rpc.SessTableIndex");
int my_idx = (*f_get_idx)(mod);
DLDevice local_dev = {kDLCPU, 0};
DLDevice remote_dev = tvm::runtime::AddRPCSessionMask(local_dev, my_idx);
tvm::runtime::Module gmod = mod.GetFunction("default")(remote_dev);

```