[Vulkan][Windows] test_gemm.py failed

I tried vulkan backend on Windows, but cannot pass some basic tests.

I need help!

tvm: e8045b1f

RTX 2080

GeForce Game Ready 466.27

Vulkan: 1.2.176.1

PS F:\tvm-clean\tests\python\integration> python .\test_gemm.py
[19:49:41] F:\tvm-clean\src\runtime\vulkan\vulkan.cc:760: Initialize Vulkan with 1 devices..
[19:49:41] F:\tvm-clean\src\runtime\vulkan\vulkan.cc:762: vulkan(0)='NVIDIA GeForce RTX 2080' phy_dev_id=0000025AFEDA8060 use_immediate=1
vulkan(0): exec=0.0016541 sec/op
Traceback (most recent call last):
  File ".\test_gemm.py", line 113, in <module>
    test_gemm()
  File ".\test_gemm.py", line 104, in test_gemm
    check_device("vulkan")
  File ".\test_gemm.py", line 102, in check_device
    tvm.testing.assert_allclose(c.asnumpy(), np.dot(a_np, b_np.T), rtol=1e-5)
  File "f:\tvm-clean\python\tvm\testing.py", line 82, in assert_allclose
    np.testing.assert_allclose(actual, desired, rtol=rtol, atol=atol, verbose=True)
  File "C:\Program Files\Python36\lib\site-packages\numpy\testing\_private\utils.py", line 1528, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "C:\Program Files\Python36\lib\site-packages\numpy\testing\_private\utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=1e-05, atol=1e-07

Mismatched elements: 1048259 / 1048576 (100%)
Max absolute difference: 114.56476
Max relative difference: 0.47000498
 x: array([[330.80978, 254.14168, 252.06648, ..., 256.36505, 263.76697,
        260.7766 ],
       [254.14168, 338.71793, 253.24164, ..., 259.01147, 258.9355 ,...
 y: array([[257.81796, 261.3609 , 258.75165, ..., 262.34116, 266.445  ,
        266.63632],
       [248.1334 , 257.94534, 258.8119 , ..., 262.41382, 264.9256 ,...

Yes, nvidia vulkan drivers are known to have problems with TVM-generated code unfortunately.

Thanks @masahi .

So do we have any plan to adapt our TVM-generated code to nvidia vulkan, or we just wait for nvidia to fix the related issues?

There is not much we can do. We cannot change our code to work with nvidia vulkan.