Introducing Eyot - A programming language where the GPU is just another thread

Eyot is a new language I’m building to make offloading work to the GPU as seamless as spawning a background thread.

Eyot source code is transparently compiled for both CPU and GPU, with communication between the two handled by the runtime. Traditional GPU programming expects you to handle many tasks, such as memory allocation, compiling the kernel, scheduling work, etc. These have long been handled by a language runtime when writing code for the CPU, and Eyot extends that convenience to code destined for the GPU as well.

The intended users are those in areas where the GPU or other accelerators are used heavily, e.g. game development, numerical analysis and AI.

It is early days for Eyot. It is not ready for real work, but you can experiment with it, and if you do, I’d love to hear your thoughts. To take a simple example (available in the playground)

fn square(value i64) i64 {
   print_ln("square(", value, ")")
   return value * value
}

cpu fn main() {
    // 1. call it directly
    print_ln("square(2) on cpu = ", square(2))

    // 2. call it as a worker running on cpu
    let cpu_worker = cpu square
    send(cpu_worker, [i64]{ 3 })
    print_ln(receive(cpu_worker))

    // 3. call it as a worker running on the gpu
    let gpu_worker = gpu square
    send(gpu_worker, [i64]{ 4, 5, 6 })
    print_ln(receive(gpu_worker))
}

First, this declares the square function which takes and returns a 64 bit integer. main then calls this in 3 different ways that illustrate Eyot’s distinguishing feature

1. The square function is called as you’d expect, directly, and on the CPU

2. A CPU worker is created from the square function (let cpu_worker = cpu square). This worker processes values sent to it with the send function on a background CPU thread. After squaring the number, the worker returns it through the call to receive

3. This time a GPU worker is created rather than a CPU worker (let gpu_worker = gpu square). This causes the square function to be compiled as a kernel, and run on the GPU, otherwise it acts identically. As you can see Eyot’s print_ln works GPU-side

Motivation

I’ve worked on many projects where shifting computation to the GPU presented an obvious path to better performance that got ignored due to the difficulty of doing so. These projects were not just in obvious areas like computer vision or game development, but also in unlikely matches for GPU programming, like desktop application development.

For example, back when I worked on Texifier, a macOS LaTeX editor, I adjusted the venerable TeX typesetting system to output polygons directly into GPU memory, rather than writing a PDF. This reduced latency far enough that we could update the output in real time. The feature was popular, but the difficulty of making it work left me questioning if the project was worth it.

With Eyot I want to build a language where working on the GPU is ingrained so deeply in the language’s design that it becomes trivial. For a long time we have thought about the CPU/OS combination as something that runs our code, rather than a device to be manipulated. Eyot simply extends this to the GPU. Options like CUDA exist already, but with Eyot the intention is to build the entire language around that model of GPU concurrency.

Current status

Progress is slow as I work on this in my spare time (sponsors appreciated!), and I’ve recently had a break with the arrival of a new baby, but my major roadmap items are:

• Rendering Eyot facilitates access to the GPU for computational purposes only at this stage. Game development is a big target for this project, so rendering support is high on my wishlist. I’m hoping to do this using Vulkan, and simultaneously replace OpenCL in favour of Vulkan compute

• Syntax I’ve deferred development of Eyot’s syntax so I can experiment with the CPU/GPU interaction without adding language features that would not be viable in both cases. Major missing syntax features for me are Algebraic Data Types, Lambdas and some form of interface/trait style polymorphism

• GPU Memory management There is a lot of work here. Vectors and strings can only be allocated CPU-side, this is something that should work on the GPU, I’d also like the memory manager to be able to transparently shift allocations to shared buffers when appropriate

• Performance Following the principle of Make it work, make it right, make it fast mantra, I will probably leave this for the immediate future, but it would be nice to throw real workloads at Eyot soon and get them up to speed

• Standard library This doesn’t need improving, I have only a handful of functions, it needs starting…

It is also useful to state some things I will not be working on

• Automatic parallelisation Eyot does not, and will not, automatically parallelise work across CPU/GPU cores. The intention is to be a convenient option for distributing work across processors, not reduce control.

• Theoretically optimal performance Eyot is not intended as a total replacement for current GPGPU libraries any more than C and C++ are intended as a total replacements for Assembly. I would consider significant performance deviations between Eyot code and equivalent C/Vulkan code to be a bug, but for me ease of use is an acceptable price to pay for some performance penalties

• Being the next great general purpose language There will be as few syntax differences between GPU and CPU as possible, so the language design will be bound by the GPU capabilities, which may restrict what I can add to Eyot’s syntax

Thanks for reading. You can learn more about Eyot from its documentation and source code. If you want to try it, there is the playground, or you can install Eyot on your machine.