[Question] why bert base gpu float32 latency are same in batch 1, 16, 64?

[comaniac]

You should also include the time of getting the output to make sure the device is synced, as we don’t have a device barrier API on the Python side.