I am using the Mobilenet provided with Relay. When I pass in random image data, the output is coming out as all equal, [0.1 ... 0.1] for example, in the 10-class case. I am checking the intermediate values in the network, and after a few layers, most values are going to 0 (which is presumably why the softmax is returning all 0.1s). Is this expected when the network is initialized with random parameters and passed random data? Is there any way to easily generate parameters that won’t cause intermediate values to go to 0 (without getting pretrained parameters)?
What’s the range of your random numbers? Consider an image with 3-channel RGB, then its value ranging from 0 to 255. If you generated random numbers within 0 and 1, it’s like a black image.
We are running mobilenet like so:
from tvm.relay.testing.mobilenet import get_workload as get_mobilenet
module, params = get_mobilenet(image_shape=(3, 32, 32), num_classes=10)
ex = tvm.relay.create_executor("graph", mod=module)
input = np.random.uniform(0, 1, size=(1, 3, 32, 32)).astype('float32')
result = ex.evaluate()(input, **params)
print(result) # result: [[0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1]]
@comaniac generating numbers between 0 and 255 had the same result
I see. Then it should be just the initialized weights issue as @gussmith23 pointed out. I used the MobileNet from Gluon CV model zoo and got reasonable results.
yep thanks @comaniac! the problem was two-fold: randomly initialized parameters + randomly initialized input data caused our output to be [0.1 … 0.1]