[TSIM] TSIM stuck in 'Compute' module when try restnet18 sample

Questions
1

Hi @vegaluis @thierry,

I am trying to run the resnet18 sample on tsim, and experience a stuck issue, seems like compute module stuck in “isGemm” state due to inflight keep bounce from 1 to 2 and back to 1, log as following.

Reproduce is easy, just change vta_confg.json from sim into tsim, then add tsim init logic in resnet18 sample file and run, would saw this issue.

Regards
Hua

dec.io.isGemm is 1
inflight is 1
dec.io.isGemm is 1
inflight is 1
[TensorGemm] [uop] acc:03cc inp:0293 wgt:0013
dec.io.isGemm is 1
inflight is 1
[TensorGemm] [inp] i:0 val:0e0000000000180000271d1108030001
[TensorGemm] [wgt] i:0 val:0bfffdfa000005000008f5f4fafe0001
[TensorGemm] [wgt] i:1 val:ffffff03000011000008f0e0fcfb0004
[TensorGemm] [wgt] i:2 val:ff02fffc0000000000fafefe07f5000e
[TensorGemm] [wgt] i:3 val:fbedfb0300000200000800fdf30500f3
[TensorGemm] [wgt] i:4 val:f10a01020000050000fd04010bfe0004
[TensorGemm] [wgt] i:5 val:fbf302fd0000020000fcfe0206050005
[TensorGemm] [wgt] i:6 val:ff04f304000003000005080503f30002
[TensorGemm] [wgt] i:7 val:0202fcff00000100000207020cfd00fc
[TensorGemm] [wgt] i:8 val:fffd02fe0000020000fafaf9fa0000fd
[TensorGemm] [wgt] i:9 val:fcfe0005000000000007010201f90000
[TensorGemm] [wgt] i:a val:fbfc02f60000fa0000060103030900fd
[TensorGemm] [wgt] i:b val:0800010600000100000102fc05fe0000
[TensorGemm] [wgt] i:c val:fe00080e0000f4000007fff1fefd0008
[TensorGemm] [wgt] i:d val:fd0105fe0000010000ff0200ef0800fd
[TensorGemm] [wgt] i:e val:010103000000fd00000305fc02000000
[TensorGemm] [wgt] i:f val:f604070800000200000202fa010300f7
[TensorGemm] [acc_i] i:0 val:fffffe26
[TensorGemm] [acc_i] i:1 val:fffffe03
[TensorGemm] [acc_i] i:2 val:00000118
[TensorGemm] [acc_i] i:3 val:000000a2
[TensorGemm] [acc_i] i:4 val:00000279
[TensorGemm] [acc_i] i:5 val:ffffffb5
[TensorGemm] [acc_i] i:6 val:fffffebf
[TensorGemm] [acc_i] i:7 val:ffffff2e
[TensorGemm] [acc_i] i:8 val:0000061b
[TensorGemm] [acc_i] i:9 val:fffffdd3
[TensorGemm] [acc_i] i:a val:00000263
[TensorGemm] [acc_i] i:b val:0000005e
[TensorGemm] [acc_i] i:c val:fffff9d7
[TensorGemm] [acc_i] i:d val:00000173
[TensorGemm] [acc_i] i:e val:fffff9e6
[TensorGemm] [acc_i] i:f val:fffffc4c
dec.io.isGemm is 1
inflight is 2
[TensorGemm] [uop] idx:0042
[TensorGemm] [acc_o] i:0 val:fffffbb8
[TensorGemm] [acc_o] i:1 val:fffffe94
[TensorGemm] [acc_o] i:2 val:0000003b
[TensorGemm] [acc_o] i:3 val:fffffe99
[TensorGemm] [acc_o] i:4 val:000000ea
[TensorGemm] [acc_o] i:5 val:00000040
[TensorGemm] [acc_o] i:6 val:fffffd37
[TensorGemm] [acc_o] i:7 val:fffffec6
[TensorGemm] [acc_o] i:8 val:0000056f
[TensorGemm] [acc_o] i:9 val:fffffd54
[TensorGemm] [acc_o] i:a val:000004a0
[TensorGemm] [acc_o] i:b val:ffffffa8
[TensorGemm] [acc_o] i:c val:fffffadf
[TensorGemm] [acc_o] i:d val:000000dc
[TensorGemm] [acc_o] i:e val:fffffb62
[TensorGemm] [acc_o] i:f val:fffffddb
[TensorGemm] [out] i:0 val:b8
[TensorGemm] [out] i:1 val:94
[TensorGemm] [out] i:2 val:3b
[TensorGemm] [out] i:3 val:99
[TensorGemm] [out] i:4 val:ea
[TensorGemm] [out] i:5 val:40
[TensorGemm] [out] i:6 val:37
[TensorGemm] [out] i:7 val:c6
[TensorGemm] [out] i:8 val:6f
[TensorGemm] [out] i:9 val:54
[TensorGemm] [out] i:a val:a0
[TensorGemm] [out] i:b val:a8
[TensorGemm] [out] i:c val:df
[TensorGemm] [out] i:d val:dc
[TensorGemm] [out] i:e val:62
[TensorGemm] [out] i:f val:db
dec.io.isGemm is 1
inflight is 1
dec.io.isGemm is 1
inflight is 1
[TensorGemm] [uop] acc:0404 inp:02cd wgt:0013
dec.io.isGemm is 1
inflight is 1
[TensorGemm] [inp] i:0 val:0e0000000000180000271d1e0b030001
[TensorGemm] [wgt] i:0 val:0bfffdfa000005000008f5f4fafe0001
[TensorGemm] [wgt] i:1 val:ffffff03000011000008f0e0fcfb0004
[TensorGemm] [wgt] i:2 val:ff02fffc0000000000fafefe07f5000e
[TensorGemm] [wgt] i:3 val:fbedfb0300000200000800fdf30500f3
[TensorGemm] [wgt] i:4 val:f10a01020000050000fd04010bfe0004
[TensorGemm] [wgt] i:5 val:fbf302fd0000020000fcfe0206050005
[TensorGemm] [wgt] i:6 val:ff04f304000003000005080503f30002
[TensorGemm] [wgt] i:7 val:0202fcff00000100000207020cfd00fc
[TensorGemm] [wgt] i:8 val:fffd02fe0000020000fafaf9fa0000fd
[TensorGemm] [wgt] i:9 val:fcfe0005000000000007010201f90000
[TensorGemm] [wgt] i:a val:fbfc02f60000fa0000060103030900fd
[TensorGemm] [wgt] i:b val:0800010600000100000102fc05fe0000
[TensorGemm] [wgt] i:c val:fe00080e0000f4000007fff1fefd0008
[TensorGemm] [wgt] i:d val:fd0105fe0000010000ff0200ef0800fd
[TensorGemm] [wgt] i:e val:010103000000fd00000305fc02000000
[TensorGemm] [wgt] i:f val:f604070800000200000202fa010300f7
[TensorGemm] [acc_i] i:0 val:fffffff8
[TensorGemm] [acc_i] i:1 val:fffffeec
[TensorGemm] [acc_i] i:2 val:00000140
[TensorGemm] [acc_i] i:3 val:0000006e
[TensorGemm] [acc_i] i:4 val:000002a9
[TensorGemm] [acc_i] i:5 val:00000062
[TensorGemm] [acc_i] i:6 val:fffffecd
[TensorGemm] [acc_i] i:7 val:ffffff8e
[TensorGemm] [acc_i] i:8 val:0000053f
[TensorGemm] [acc_i] i:9 val:fffffd77
[TensorGemm] [acc_i] i:a val:000002ad
[TensorGemm] [acc_i] i:b val:fffffff6
[TensorGemm] [acc_i] i:c val:fffff93c
[TensorGemm] [acc_i] i:d val:000001e6
[TensorGemm] [acc_i] i:e val:fffff98a
[TensorGemm] [acc_i] i:f val:fffffc08
dec.io.isGemm is 1
inflight is 2
[TensorGemm] [uop] idx:0043
[TensorGemm] [acc_o] i:0 val:fffffe30
[TensorGemm] [acc_o] i:1 val:fffffcaa
[TensorGemm] [acc_o] i:2 val:ffffffe9
[TensorGemm] [acc_o] i:3 val:0000012b
[TensorGemm] [acc_o] i:4 val:00000285
[TensorGemm] [acc_o] i:5 val:ffffff2f
[TensorGemm] [acc_o] i:6 val:000000ec
[TensorGemm] [acc_o] i:7 val:000000f0
[TensorGemm] [acc_o] i:8 val:000003fb
[TensorGemm] [acc_o] i:9 val:fffffede
[TensorGemm] [acc_o] i:a val:000002f7
[TensorGemm] [acc_o] i:b val:00000125
[TensorGemm] [acc_o] i:c val:fffff87f
[TensorGemm] [acc_o] i:d val:00000101
[TensorGemm] [acc_o] i:e val:fffffa7e
[TensorGemm] [acc_o] i:f val:fffffc1a
[TensorGemm] [out] i:0 val:30
[TensorGemm] [out] i:1 val:aa
[TensorGemm] [out] i:2 val:e9
[TensorGemm] [out] i:3 val:2b
[TensorGemm] [out] i:4 val:85
[TensorGemm] [out] i:5 val:2f
[TensorGemm] [out] i:6 val:ec
[TensorGemm] [out] i:7 val:f0
[TensorGemm] [out] i:8 val:fb
[TensorGemm] [out] i:9 val:de
[TensorGemm] [out] i:a val:f7
[TensorGemm] [out] i:b val:25
[TensorGemm] [out] i:c val:7f
[TensorGemm] [out] i:d val:01
[TensorGemm] [out] i:e val:7e
[TensorGemm] [out] i:f val:1a
dec.io.isGemm is 1
inflight is 1
dec.io.isGemm is 1
inflight is 1
[TensorGemm] [uop] acc:043c inp:0307 wgt:0013
dec.io.isGemm is 1
inflight is 1
[TensorGemm] [inp] i:0 val:0b01000000001400002739310b0e0004
[TensorGemm] [wgt] i:0 val:0bfffdfa000005000008f5f4fafe0001
[TensorGemm] [wgt] i:1 val:ffffff03000011000008f0e0fcfb0004
[TensorGemm] [wgt] i:2 val:ff02fffc0000000000fafefe07f5000e
[TensorGemm] [wgt] i:3 val:fbedfb0300000200000800fdf30500f3
[TensorGemm] [wgt] i:4 val:f10a01020000050000fd04010bfe0004
[TensorGemm] [wgt] i:5 val:fbf302fd0000020000fcfe0206050005
[TensorGemm] [wgt] i:6 val:ff04f304000003000005080503f30002
[TensorGemm] [wgt] i:7 val:0202fcff00000100000207020cfd00fc
[TensorGemm] [wgt] i:8 val:fffd02fe0000020000fafaf9fa0000fd
[TensorGemm] [wgt] i:9 val:fcfe0005000000000007010201f90000
[TensorGemm] [wgt] i:a val:fbfc02f60000fa0000060103030900fd
[TensorGemm] [wgt] i:b val:0800010600000100000102fc05fe0000
[TensorGemm] [wgt] i:c val:fe00080e0000f4000007fff1fefd0008
[TensorGemm] [wgt] i:d val:fd0105fe0000010000ff0200ef0800fd
[TensorGemm] [wgt] i:e val:010103000000fd00000305fc02000000
[TensorGemm] [wgt] i:f val:f604070800000200000202fa010300f7
[TensorGemm] [acc_i] i:0 val:000005ab
[TensorGemm] [acc_i] i:1 val:000001ab
[TensorGemm] [acc_i] i:2 val:00000184
[TensorGemm] [acc_i] i:3 val:00000044
[TensorGemm] [acc_i] i:4 val:0000038b
[TensorGemm] [acc_i] i:5 val:0000018c
[TensorGemm] [acc_i] i:6 val:fffffe53
[TensorGemm] [acc_i] i:7 val:00000000
[TensorGemm] [acc_i] i:8 val:000002f2
[TensorGemm] [acc_i] i:9 val:fffffcfa
[TensorGemm] [acc_i] i:a val:0000023e
[TensorGemm] [acc_i] i:b val:0000001c
[TensorGemm] [acc_i] i:c val:fffff878
[TensorGemm] [acc_i] i:d val:0000019c
[TensorGemm] [acc_i] i:e val:fffff85c
[TensorGemm] [acc_i] i:f val:fffffbda

2

Hi Hua,

ResNet18 won’t work right now because the python-resnet-file is not set up to load-the-hardware-symbol (shared library) before every invocation of f(). Currently, we require to use that for TSIM. So, basically what you are doing right now is running one-layer or one VTA invocation, and it is hanging after that because the simulation-thread is not flushed.

See how we load symbol,

3

@vegaluis,get it, thanks for the prompt reply .
One quick question is why here need reload the TSIM library before every invocation f()? is that chisel’s limitation or in future solution we can do something to reset the TSIM state after get FINISH opcode then no need reload the library every time?

Regards
Hua

4

This is a limitation that we should be able to overcome in the TSIM technically

5

@tqchen, cool, thanks for the follow up, that sounds good.

Regards
Hua

6

Yeah, this is not about the hardware language. It is more about how we are setting up hardware simulation. We are working on that.