[Relax] How to initialize tvm.relax.testing.nn.Parameter with a real tensor

[crispyberry]

Hi guys,

I find in PyTorch we can use torch.nn.parameter.Parameter to initialize by a real tensor, but in relax it seems that we can only initial with shape. If I have a tensor(numpy or list), how can I set it as tvm.relax.testing.nn.Parameter, just like PyTorch can do?

Cheers.

[anders]

@crispyberry

Hi guys,

I find in PyTorch we can use torch.nn.parameter.Parameter to initialize by a real tensor, but in relax it seems that we can only initial with shape. If I have a tensor(numpy or list), how can I set it as tvm.relax.testing.nn.Parameter, just like PyTorch can do?

Cheers.

you can optimize it by donating a pull-request, come on, man!

[crispyberry]

Thanks for your reply! May I ask how to operate on parameters? For example, weight = nn.Parameter((self.out_channels, self.in_channels)) weight[0] << 2 Now I cannot do this operation:TypeError: unsupported operand type(s) for >>: ‘TupleGetItem’ and ‘int’ I cannot get the value of weight[0], because now it is a TupleGetItem type?

[junrushao]

This is a feature non-existent today but on the 2nd top priority

[crispyberry]

Thanks for your information!

[yzh119]

More user friendly relax nn.modules (behaves more like torch.nn.module) is one of the important directions we are working on, currently we have several proposals and we will make them available in MLC-LLM soon.