In this post I would like to cover some of the difficulties that a developer, working on GPU computing, might encounter.
Here is some background fist.
There exists a nonprofit organisation The Khronos Group that creates open standards for cross-platform technologies which are then used by all the major players, like Apple, Nvidia, ATI, Intel, ARM, etc. Khronos itself doesn’t develop any software, it’s just a medium for discussions of standards for different technologies and is a kind of mediator.
After the members of Khronos agree on the API, each of the hardware manufacturers starts implementing this API for its platform. When all is ready and Khronos certifies it, a manufacturer can indicate that they support this technology on their product. This indication is a sign to developers, who are interested in it, that they can start using this API for their programs or games.
Khronos’s biggest projects are OpenGL, OpenCL, WebGL, WebCL and many others (Vulkan®, COLLADA™, glTF™, EGL™, OpenSL ES™, OpenMAX™, SPIR™, SYCL™, NNEF™, OpenVX™, Safety Critical, OpenKCam™, OpenVG™, Data Format).
Since the beginning of Khronos it had leaders in all the main spheres:
- OpenGL (a cross-language, cross-platform application programming interface for Windows, Linux, OS X), being a massively developed graphic system, became a key technology for cross-platform games, .
- OpenCL (a framework for writing programs that execute across heterogeneous platforms consisting of central processing units (CPUs), graphics processing units (GPUs), digital signal processors) became a key technology for GPU computing.
- There was also DirectX for MS.
With the flow of time and the development of new technologies, the approaches used as the basis for these APIs revealed some flaws (How GPUs and graphics drivers work):
- The transfer of objects from RAM to VRAM takes time, and if you perform many calculations of small objects, the time wasted for transferring them to VRAM can nullify the performance improvement gained with the help of GPU computing.
- Being a graphic system, OpenGL has many different validations and a necessity to store graphic information in the buffer, which slows down the execution of commands.
Everything changes with the appearance of Metal & Vulkan presented at the Game Developers Conference in 2015. The main idea behind them is to make a developer and GPU closer to each other and provide the possibility to access RAM objects directly from GPU.
How is the situation now in the domain of GPU computing?
- OpenCL continues to develop, but the problem is that Apple stopped supporting its canonical API since version 1.2. Only OpenCL.framework with a strange version 2.8. is still supported. Apple’s further plans regarding OpenCL are not announced.
- Vulkan also evolves, but it is available only for Linux and Windows.
- There also exists the realisation of Vulkan API via Metal: MoltanVK, but it is too early to call it a full-scale solution.
- Metal only supports MacOS, iOS, tvOS.
- CUDA is only available for Nvidia family.
As you can see, the possibilities for cross-platform GPU computing are very limited, and the plans for cross-platform GPU calculations via Swift can hardly be reached now.
That is why it is necessary to implement more complicated mechanisms of intermediate abstractions which do not depend on a driver. After that it will be possible to develop a module for calculations which will use a drive available for GPU platform and API.
Author: Volodymyr Pavliukevych
Senior Software Engineer, Data Scientist.