Hi @Necrotos,
It it seems like the constants in your graph are currently represented using relay.Var’s which will mean constant folding won’t be able to recognise the constants and therefore won’t perform any optimization. There is a related discussion here that might help: Constant params should be constants - #3 by lhutton1
I think if you used the bind_params_by_name functionality (mentioned in the link above) then run the relay.transform.FoldConstant pass on the module, it should solve your issue.