Ray Tracing
Introduction
In 3D graphics programming, Ray tracing is the future. A lighting technique that brings an extra level of realism to games. It emulates the way light reflects and refracts in the real world, providing a more believable environment than what’s typically seen using rasterization in most games today.
In this series of blogs, I would write a simple ray tracer to demonstrate the concepts of Ray tracing.
We will start by writing a small application to output images in Portable Pixel Map (ppm) format. This format is a lowest common denominator color image file format, you can read more about it here .
Lets Start!

Create a new CLI project, I will call it Raytracing.
import Foundation let filename = getDocumentsDirectory().appendingPathComponent("output.ppm") let row = 400 let column = 400 var out = "" out.append("P3\n\(row) \(column)\n255\n") for i in 0..<row{ for j in 0..<column { let color = float3(Float(i)/Float(row), Float(j)/Float(column), 0.5) let r = Int(256*color.x) let g = Int(256*color.y) let b = Int(256*color.z) out.append("\(r) \(g) \(b) \n") } } do { try out.write(to: filename, atomically: true, encoding: String.Encoding.utf8) } catch { // failed to write file – bad permissions, bad filename, // missing permissions, or more likely it can't be converted //to the encoding print("Something went wrong with the file") }
Create a new folder(Group) and called it Common, add a new Swift file, Files.swift, and add the following:
import Foundation import simd typealias float2 = SIMD2<Float> typealias float3 = SIMD3<Float> typealias float4 = SIMD4<Float> let π = Float.pi func getDocumentsDirectory() -> URL { let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask) return paths[0] }
Make sure your file is added to the target build phase

Run RayTracing and you should see output.ppm created in your documents folder

You can play with the values and test it for yourself. You can also use textedit application, or if you prefer the command line use ‘vi output.ppm’ and view the file in text.
Rays
After the above setup and introduction we will start talking about rays. A Ray is a narrow beam of light traveling in a straight line from its place of origin. An Origin in our case will be the eye, or camera. As the ray travels from the camera to the scene we would like to know what colors it sees along the path. In essence the ray is a function of the real number parameter t from the camera into the scene, p(t)=\overrightarrow{O}+t \times \overrightarrow{L} where \overrightarrow{L}is the direction. At zero the color is completely dark, a negative number would be behind the camera, and a positive number would be in front of the camera.

Create a new target, I will call it RayTracing3 and add Ray.swift
struct Ray { private var _origin = float3(0,0,0) private var _direction = float3(0,0,0) var origin: float3 { get { return _origin } set { _origin = newValue } } var direction : float3 { get { return _direction } set { _direction = newValue } } func pointAtParameter(parameter t: Float) -> float3 { return origin + t * direction } init(origin: float3 = float3(0,0,0), direction: float3 = float3(0,0,0)) { self.origin = origin self.direction = direction } }
Starting with iOS 14 and Big Sur Apple’s ray struct
will be available on A13 processors. In our case will target as many Apple devices as possible.
Ray Tracers
All Ray tracers center around the concept of sending rays from the camera to the scene and compute the colors of the pixels it intersects with. We will start simple, to get the code up and running, we will render a simple background from the ground green to the sky blue.

This time we will stick to a rectangle 800×600 image, set the camera at (0,0,0) and use the left-hand notation of Metal as the image above.
To color the intersection of the rays we will use Hemispheric lighting, where half of the screen is colored by one color while the other is colored by another.
Here is the Lighting code:
func HemisphericLighting(ray r : Ray) -> float3 { //1 create Unit Vector to compute the intensity let unitDirection : float3 = normalize(r.direction) //2 shifting and scaling t in [0.0, 1.0] let t = 0.5*(unitDirection.y + 1.0) //3 applying lerp return (1.0 - t) * float3(0.34,0.9,1.0) + t * float3(0.29,0.58,0.2) }
Going through the code:
- Get the unit vector of the direction of the ray, we will use the y direction to move vertically. The range will be between -1.0 and 1.0.
- The intensity needs to shifted and scaled to the ranged [0.0, 1.0].
- The linear blend is defined as
blendedValue = (1-t) \times startValue + t \times endValue, \textrm {where } t \in [0.0, 1.0].
The rest of the main file:
//1 setting the file and size of the picture 800x600 let filename = getDocumentsDirectory().appendingPathComponent("output3.ppm") let row = 800 let column = 600 var out = "" out.append("P3\n\(row) \(column)\n255\n") //2 setting the values according to the image above let lowerLeftCorner = float3(-4.0,-1.0, 1.0) let horizontal = float3(8.0,0.0,0.0) let vertical = float3(0.0,4.0,0.0) let origin = float3(0.0,0.0,0.0) //3 we will iterate in a 2 dimention and light the image for j in 0..<column { for i in 0..<row { //4 moving the cursor over the 2D image let u = Float(i) / Float(row) let v = Float(j) / Float(column) //5 shooting the ray in the direction of the cursor let ray = Ray(direction: lowerLeftCorner + u*horizontal + v*vertical) let color = HemisphericLighting(ray: ray) //6 preparing the writing to the ppm file let r = Int(256*color.x) let g = Int(256*color.y) let b = Int(256*color.z) out.append("\(r) \(g) \(b) \n") } } //7 writing the file do { try out.write(to: filename, atomically: true, encoding: String.Encoding.utf8) } catch { // failed to write file – bad permissions, bad filename, //missing permissions, or more likely it can't be converted //to the encoding print("Something went wrong with the file") }
Going through the code:
- Setting the filename by invoking a function from the common code, and setting the dimension of the image.
- Setting the initial values for iterations as the illustration above.
- Iteration to light the image from the ground up going line by line from left to right.
- The (u,v) coordinate which is a ratio between [0.0 \rightarrow x, 0.0 \rightarrow y] of the unit direction vector.
- Shooting the ray in the direction of the (u,v) from the camera.
- Calculating the RGB color values to write to the ppm file
- Flushing the output to the file.
The final output:

If you have realized all the work has been done on the CPU side, which is time consuming and inefficient. The total running time without saving the file is
Time to execute, without saving the file: 6.756080374 seconds`
GPU
We need to abolish the sequential operation, the loops O(r \times c). There are two main ways on Apple platforms. Using concurrency on the CPU side by utilizing the GCD as we demonstrated in my previous blog and/or utilizing Apple powerful GPU’s, as demonstrated in my blog here.
With the excitement in WWDC20 of Apple silicon we should always utilize the GPU.
The fun part, coding. Start by creating a new target:

I will call it RayTracingGPU and add the Files in Common

Here is the main file:
import Foundation import MetalKit import Accelerate let start = DispatchTime.now() // <<<<<<<<<< Start time //1 setting up the GPU var device = MTLCreateSystemDefaultDevice()! var commandQueue = device.makeCommandQueue()! var library = device.makeDefaultLibrary() let commandBuffer = commandQueue.makeCommandBuffer() let computeEncoder = commandBuffer?.makeComputeCommandEncoder() var computeFunction = library?.makeFunction(name: "kernal_ray")! var computePipelineState = try! device.makeComputePipelineState(function: computeFunction!) //setting the 2 dimention image for the GPU to write var outputTexture : MTLTexture let row = 800 let column = 600 //3 creating an output texturedescriptor from the input let textureDescriptor = MTLTextureDescriptor() textureDescriptor.textureType = .type2D textureDescriptor.pixelFormat = .bgra8Unorm textureDescriptor.width = row textureDescriptor.height = column textureDescriptor.usage = [.shaderWrite] outputTexture = device.makeTexture(descriptor: textureDescriptor)! //4 Encoding the command to the GPU computeEncoder?.pushDebugGroup("State") computeEncoder?.setComputePipelineState(computePipelineState) computeEncoder?.setTexture(outputTexture, index: 0) //5 creating the Threads in a 2 dimension var width = computePipelineState.threadExecutionWidth var height = computePipelineState.maxTotalThreadsPerThreadgroup / width let threadPerThreadgroup = MTLSizeMake(width, height, 1) let threadsPerGrid = MTLSizeMake(row, column, 1) computeEncoder?.dispatchThreads(threadsPerGrid, threadsPerThreadgroup: threadPerThreadgroup) //6 Off to the GPU computeEncoder?.endEncoding() computeEncoder?.popDebugGroup() commandBuffer?.commit() commandBuffer?.waitUntilCompleted() let end = DispatchTime.now() // <<<<<<<<<< end time let nanoTime = end.uptimeNanoseconds - start.uptimeNanoseconds // <<<<< Difference in nano seconds (UInt64) let timeInterval = Double(nanoTime) / 1_000_000_000 // Technically could overflow for long running tests print("Time to execute, without saving the file: \(timeInterval) seconds") //7 copying the image to cpu memory and saving it to the file. //Thanks to all who shared their knowledge on the Internet //https://computergraphics.stackexchange.com/questions/7428/mtltexture-getbytes-returning-blank-image //https://developer.apple.com/forums/thread/30488 let commandBuffer2 = commandQueue.makeCommandBuffer() let blitEncoder = commandBuffer2?.makeBlitCommandEncoder() blitEncoder?.synchronize(texture: outputTexture, slice: 0, level: 0) blitEncoder?.endEncoding() commandBuffer2?.commit() commandBuffer2?.waitUntilCompleted() let outImage = makeImage(from: outputTexture) saveImage(outImage!, atUrl: getDocumentsDirectory().appendingPathComponent("outputGPU.png")) print("finished")
Going through the code:
- From the device we setup the GPU utilizing Metalkit which we imported. For more please check the previous blog here.
- We set the dimension of the image.
- We need to setup the memory for the GPU to write in, unlike our previous example, this time we need a two dimension grid. To create a texture in Metal we set a texture descriptor similar to our pipeline descriptor. We invoke the device to create a texture to the specifics of the descriptor.
- Encoding the command to the GPU by setting the compute pipeline and send the output texture to the GPU at buffer index zero. We could send other uniform data.
- Creating the two dimension thread group, you can refer to the Metal manual here.
- Off to the GPU, sending it and waiting for the completion.
- The remaining is to save the image to file. Most of the code was copied from the Internet, the references are in the code.
On the GPU side:
kernel void kernal_ray(texture2d<half, access::write> outTexture [[texture(0)]], uint2 pid [[thread_position_in_grid]]){ //1 Check if the pixel is within the bounds of the output texture if((pid.x >= outTexture.get_width()) || (pid.y >= outTexture.get_height())) { // Return early if the pixel is out of bounds return; } //2 passed from the CPU int row = outTexture.get_width(); int column = outTexture.get_height(); //3 These uniform values can be sent from the CPU, not worth the investigation the data is small half4 bottom(0.34,0.9,1.0,1); half4 top(0.29,0.58,0.2,1); float3 horizontal(8.0,0.0,0.0); float3 vertical(0.0,4.0,0.0); float3 lowerLeftCorner(-4.0,-1.0, 1.0); //4 using the values to caluculate the direction vector of the ray float u = float(pid.x)/float(row); float v = float(pid.y)/float(column); float3 direction = lowerLeftCorner + u * horizontal + v * vertical; float intensity = normalize(direction).y * 0.5 + 0.5; //5 the mix function is the lerp blending function half4 answer = mix(bottom, top, intensity); //6 writing the answer to the texture outTexture.write(answer, pid); }
Going through the code:
- Check the boundary of the thread grid with the image size grid, to avoid no work threads.
- To compute the ratio we get the denominators from the CPU.
- The same uniforms values that we used in RayTracing3 on the CPU side (above), we initialize them on the GPU side.
- We calculate the (u,v) components for the ray direction the same way we did on the CPU side in the previous example.
- The mix function computes the leap blend
- Finally each thread writes the answer on the same texture in parallel.
The total running time without saving the file:
Time to execute, without saving the file: 0.037714654 seconds
This is 179.2 times faster! A dramatic increase. Imagine how much energy you saved!
There should be no doubt in your mind when you see big loops in your code, is to think about compute kernels on the GPU.
This will be the first part of this Ray tracing series. Next will be adding objects to our scene.
The entire project can be pulled from my Github account here. Please follow me here and on Medium.
Til next time stay tuned the fun just started.