Deprecate OpenGL/WebGL in favor of Vulkan/WebGPU

Development pre-RFC
#1

TVM stack has a primilinary OpenGL backend that translates some of the compute code into opengl shaders. However, due to the limitation of GLSL in terms of its compute shader capability, we do not have the flexibility of the other programming models such as OpenCL/CUDA when targeting OpenGL. These limitations causes the inability to get the state of art performance due to these restrictions.

As the Graphics standard moves forward, most of the desktop/mobile platforms now shift to Vulakan(and Metal in the Appleā€™s case), which has better supports for compute shaders to match the CUDA/OpenCL prorgamming model. There is also an emerging WebGPU standard(based on similar APIs as vulkan), that is not yet shipped in stable releases of browsers, but is part of nightly build of Chrome and other browswer.

Given that the opengl backend is not being actively used or maintained currently, this RFC proposes to drop the support and instead focus on supporting the current and future graphics API via vulkan and webGPU.

2 Likes
Pure WebAssembly support
How to get an example to use resnet50 with pure wasm?
#2

Iā€™m interested in this topic. Having worked on frontend stuff for some time, now Iā€™m looking to revive my backend-fu :slight_smile: Also I wanted to learn about graphics API, this seems a good opportunity.

I also hope that

WebGPU + WASM = win

Although GPU support in WASM doesnā€™t seem official yet.

I think WebGPU implementations (dawn, wgpu) can be used in a standard environment outside of browser. So development workflow should be similar to using Vulkan API (but higher level than vulkan abstraction)

1 Like
#3

I am told some times about the web acceleration in Mobile (The scenario is often the H5 App). For the compatibility of most mobile phones, they even could only use WebGL1 (not WebGL2). How do we consider this situation?

#4

Due to the restriction of the OpenGL shader language ā€” you cannot use shared memory and have to write to a single predefined axis. we cannot get as performant kernels that unleashes power as cuda or OpenCL.

In this case, probably the optimized web assembly will outperform the webGL(although the wasm only runs on CPU).

#5

Hi

I am completely new to tvm but I was actually looking into this project for compiling my models targeting WebGL. So I was a bit sad to see that OpenGL support is being left aside :confused:

From my understanding, wasm can have an advantage over WebGL for small models, but maybe not for bigger ones?

Since we might have to wait for a while before WebGPU is widely available, wouldnā€™t it be good to keep some basic support for WebGL? Looking at some other messages you posted, I understand that there might be a significant workload involved to keep this support alive. How far are we to have something working from your point of view?

Thanks

#6

@pvanhaes Thank you for your inputs, we certainly value communities input that is why this thread is posed as an RFC.

We certainly see values in GPU if we can make good use of the resources.

The main limitation of WebGL is the limited GPGPU programming model. Because WebGL does not support compute shaders, we have to convert all stages into OpenGL fragment programs. While the fragment shaders works well for graphics, we need richer compute shader semantics(e.g. shared memory, read/write to multiple buffers) to best utilize the GPU resources as in other backend like CUDA and OpenCL.

In the case of WASM, it is possible to take full benefit of all of its features and offer the STOA perf because there is no programming model limitation.

So the original rationale is that a best supported WASM and that might be better than a (not fully utilized) GPU solution via WebGL. This is likely going to be the case on mobile devices. I am not too certain how much of that would be true for desktops.

Because WebGPU has a richer programming model(on par with Vulkan), we can likely make full utilization of the GPU resources and get the expected speedup we want.

See the following link for the WebGPU availability status https://github.com/gpuweb/gpuweb/wiki/Implementation-Status, given that WebGPU already starts to ship in browser nightlies, the hope is by the time this feature get matured, the browsers will be ready as well.

#7

Well, that might very well be true. When Iā€™ll have time Iā€™ll do some experimentation to see if thatā€™s the case for me.

It would be a great news also given that unlike WebGL (OffscreenCanvas), more browsers support web assembly in web workers.

In any case Iā€™m looking forward to the WebGPU backend!

#8