Ray Tracing

Posted on by mjeragh

Introduction

In 3D graphics programming, Ray tracing is the future. A lighting technique that brings an extra level of realism to games. It emulates the way light reflects and refracts in the real world, providing a more believable environment than what’s typically seen using rasterization in most games today.
In this series of blogs, I would write a simple ray tracer to demonstrate the concepts of Ray tracing.
We will start by writing a small application to output images in Portable Pixel Map (ppm) format. This format is a lowest common denominator color image file format, you can read more about it here .

Lets Start!

Create a new CLI project, I will call it Raytracing.

import Foundation


let filename = getDocumentsDirectory().appendingPathComponent("output.ppm")

let row = 400
let column = 400
var out = ""

out.append("P3\n\(row) \(column)\n255\n")
for i in 0..<row{
    for j in 0..<column {
        let color = float3(Float(i)/Float(row), Float(j)/Float(column), 0.5)
        let r = Int(256*color.x)
        let g = Int(256*color.y)
        let b = Int(256*color.z)
        out.append("\(r) \(g) \(b) \n")
    }
}

do {
    try out.write(to: filename, atomically: true, encoding: String.Encoding.utf8)
} catch {
    // failed to write file – bad permissions, bad filename,
  	// missing permissions, or more likely it can't be converted 
 	 //to the encoding
    print("Something went wrong with the file")
}

Create a new folder(Group) and called it Common, add a new Swift file, Files.swift, and add the following:

import Foundation
import simd

typealias float2 = SIMD2<Float>
typealias float3 = SIMD3<Float>
typealias float4 = SIMD4<Float>

let π = Float.pi

func getDocumentsDirectory() -> URL {
    let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
    return paths[0]
}

Make sure your file is added to the target build phase

Run RayTracing and you should see output.ppm created in your documents folder

You can play with the values and test it for yourself. You can also use textedit application, or if you prefer the command line use ‘vi output.ppm’ and view the file in text.

Rays

After the above setup and introduction we will start talking about rays. A Ray is a narrow beam of light traveling in a straight line from its place of origin. An Origin in our case will be the eye, or camera. As the ray travels from the camera to the scene we would like to know what colors it sees along the path. In essence the ray is a function of the real number parameter t from the camera into the scene, p(t)=\overrightarrow{O}+t \times \overrightarrow{L} where \overrightarrow{L}is the direction. At zero the color is completely dark, a negative number would be behind the camera, and a positive number would be in front of the camera.

Create a new target, I will call it RayTracing3 and add Ray.swift

struct Ray {
    private var _origin = float3(0,0,0)
    private var _direction = float3(0,0,0)
    
    
    var origin: float3 {
        get {
            return _origin
        }
        set {
            _origin = newValue
        }
    }
    var direction : float3 {
        get {
            return _direction
        }
        set {
            _direction = newValue
        }
    }
    func pointAtParameter(parameter t: Float) -> float3 {
        return origin + t * direction
    }
    
    init(origin: float3 = float3(0,0,0), 
             direction: float3 = float3(0,0,0)) {
            
            self.origin = origin
            self.direction = direction
        }
   
}

Starting with iOS 14 and Big Sur Apple’s ray struct will be available on A13 processors. In our case will target as many Apple devices as possible.

Ray Tracers

All Ray tracers center around the concept of sending rays from the camera to the scene and compute the colors of the pixels it intersects with. We will start simple, to get the code up and running, we will render a simple background from the ground green to the sky blue.

This time we will stick to a rectangle 800×600 image, set the camera at (0,0,0) and use the left-hand notation of Metal as the image above.

To color the intersection of the rays we will use Hemispheric lighting, where half of the screen is colored by one color while the other is colored by another.
Here is the Lighting code:

func HemisphericLighting(ray r : Ray) -> float3 {
    //1 create Unit Vector to compute the intensity
    let unitDirection : float3 = normalize(r.direction)
    //2 shifting and scaling t in  [0.0, 1.0]
    let t = 0.5*(unitDirection.y + 1.0)
    //3 applying lerp
    return (1.0 - t) * float3(0.34,0.9,1.0) + t * float3(0.29,0.58,0.2)
}

Going through the code:

  1. Get the unit vector of the direction of the ray, we will use the y direction to move vertically. The range will be between -1.0 and 1.0.
  2. The intensity needs to shifted and scaled to the ranged [0.0, 1.0].
  3. The linear blend is defined as
    blendedValue = (1-t) \times startValue + t \times endValue, \textrm {where } t \in [0.0, 1.0].

The rest of the main file:

//1 setting the file and size of the picture 800x600
let filename = getDocumentsDirectory().appendingPathComponent("output3.ppm")
let row = 800
let column = 600
var out = ""
out.append("P3\n\(row) \(column)\n255\n")

//2 setting the values according to the image above

let lowerLeftCorner = float3(-4.0,-1.0, 1.0)
let horizontal = float3(8.0,0.0,0.0)
let vertical = float3(0.0,4.0,0.0)
let origin = float3(0.0,0.0,0.0)

//3 we will iterate in a 2 dimention and light the image

for j in 0..<column {
    for i in 0..<row {
        //4 moving the cursor over the 2D image
        let u = Float(i) / Float(row)
        let v = Float(j) / Float(column)
        //5 shooting the ray in the direction of the cursor
        let ray = Ray(direction: lowerLeftCorner + u*horizontal + v*vertical)
        let color = HemisphericLighting(ray: ray)
        //6 preparing the writing to the ppm file
        let r = Int(256*color.x)
        let g = Int(256*color.y)
        let b = Int(256*color.z)
        out.append("\(r) \(g) \(b) \n")
    }
}

//7 writing the file

do {
    try out.write(to: filename, atomically: true, encoding: String.Encoding.utf8)
} catch {
    // failed to write file – bad permissions, bad filename, 
  //missing permissions, or more likely it can't be converted 
  //to the encoding
    print("Something went wrong with the file")
}

Going through the code:

  1. Setting the filename by invoking a function from the common code, and setting the dimension of the image.
  2. Setting the initial values for iterations as the illustration above.
  3. Iteration to light the image from the ground up going line by line from left to right.
  4. The (u,v) coordinate which is a ratio between [0.0 \rightarrow x, 0.0 \rightarrow y] of the unit direction vector.
  5. Shooting the ray in the direction of the (u,v) from the camera.
  6. Calculating the RGB color values to write to the ppm file
  7. Flushing the output to the file.

The final output:

If you have realized all the work has been done on the CPU side, which is time consuming and inefficient. The total running time without saving the file is

Time to execute, without saving the file: 6.756080374 seconds`

GPU

We need to abolish the sequential operation, the loops O(r \times c). There are two main ways on Apple platforms. Using concurrency on the CPU side by utilizing the GCD as we demonstrated in my previous blog and/or utilizing Apple powerful GPU’s, as demonstrated in my blog here.
With the excitement in WWDC20 of Apple silicon we should always utilize the GPU.

The fun part, coding. Start by creating a new target:

I will call it RayTracingGPU and add the Files in Common

Here is the main file:

import Foundation
import MetalKit
import Accelerate


let start = DispatchTime.now() // <<<<<<<<<< Start time

//1 setting up the GPU
var device = MTLCreateSystemDefaultDevice()!
var commandQueue = device.makeCommandQueue()!
var library = device.makeDefaultLibrary()
let commandBuffer = commandQueue.makeCommandBuffer()
let computeEncoder = commandBuffer?.makeComputeCommandEncoder()

var computeFunction = library?.makeFunction(name: "kernal_ray")!
var computePipelineState = try! device.makeComputePipelineState(function: computeFunction!)

//setting the 2 dimention image for the GPU to write
var outputTexture : MTLTexture
let row = 800
let column = 600
//3 creating an output texturedescriptor from the input
let textureDescriptor = MTLTextureDescriptor()
textureDescriptor.textureType = .type2D
textureDescriptor.pixelFormat = .bgra8Unorm
textureDescriptor.width = row
textureDescriptor.height = column
textureDescriptor.usage = [.shaderWrite]
outputTexture = device.makeTexture(descriptor: textureDescriptor)!

//4 Encoding the command to the GPU
computeEncoder?.pushDebugGroup("State")
computeEncoder?.setComputePipelineState(computePipelineState)
computeEncoder?.setTexture(outputTexture, index: 0)
//5 creating the Threads in a 2 dimension
var width = computePipelineState.threadExecutionWidth
var height = computePipelineState.maxTotalThreadsPerThreadgroup / width
let threadPerThreadgroup = MTLSizeMake(width, height, 1)
let threadsPerGrid = MTLSizeMake(row, column, 1)
computeEncoder?.dispatchThreads(threadsPerGrid, threadsPerThreadgroup: threadPerThreadgroup)
//6 Off to the GPU
computeEncoder?.endEncoding()
computeEncoder?.popDebugGroup()
commandBuffer?.commit()
commandBuffer?.waitUntilCompleted()

let end = DispatchTime.now()   // <<<<<<<<<<   end time

let nanoTime = end.uptimeNanoseconds - start.uptimeNanoseconds // <<<<< Difference in nano seconds (UInt64)
let timeInterval = Double(nanoTime) / 1_000_000_000 // Technically could overflow for long running tests

print("Time to execute, without saving the file: \(timeInterval) seconds")

//7 copying the image to cpu memory and saving it to the file.
//Thanks to all who shared their knowledge on the Internet

//https://computergraphics.stackexchange.com/questions/7428/mtltexture-getbytes-returning-blank-image
//https://developer.apple.com/forums/thread/30488
let commandBuffer2 = commandQueue.makeCommandBuffer()
let blitEncoder = commandBuffer2?.makeBlitCommandEncoder()
blitEncoder?.synchronize(texture: outputTexture, slice: 0, level: 0)
blitEncoder?.endEncoding()
commandBuffer2?.commit()
commandBuffer2?.waitUntilCompleted()




let outImage = makeImage(from: outputTexture)
saveImage(outImage!, atUrl: getDocumentsDirectory().appendingPathComponent("outputGPU.png"))
print("finished")

Going through the code:

  1. From the device we setup the GPU utilizing Metalkit which we imported. For more please check the previous blog here.
  2. We set the dimension of the image.
  3. We need to setup the memory for the GPU to write in, unlike our previous example, this time we need a two dimension grid. To create a texture in Metal we set a texture descriptor similar to our pipeline descriptor. We invoke the device to create a texture to the specifics of the descriptor.
  4. Encoding the command to the GPU by setting the compute pipeline and send the output texture to the GPU at buffer index zero. We could send other uniform data.
  5. Creating the two dimension thread group, you can refer to the Metal manual here.
  6. Off to the GPU, sending it and waiting for the completion.
  7. The remaining is to save the image to file. Most of the code was copied from the Internet, the references are in the code.

On the GPU side:

kernel void kernal_ray(texture2d<half, access::write> outTexture [[texture(0)]],
                       uint2 pid [[thread_position_in_grid]]){
       //1 Check if the pixel is within the bounds of the output texture
       if((pid.x >= outTexture.get_width()) || (pid.y >= outTexture.get_height()))
       {
           // Return early if the pixel is out of bounds
           return;
       }
    //2 passed from the CPU
    int row = outTexture.get_width();
    int column = outTexture.get_height();
    //3 These uniform values can be sent from the CPU, not worth the investigation the data is small
    half4 bottom(0.34,0.9,1.0,1);
    half4 top(0.29,0.58,0.2,1);
    float3 horizontal(8.0,0.0,0.0);
    float3 vertical(0.0,4.0,0.0);
    float3 lowerLeftCorner(-4.0,-1.0, 1.0);
    //4 using the values to caluculate the direction vector of the ray
    float u = float(pid.x)/float(row);
    float v = float(pid.y)/float(column);
    float3 direction = lowerLeftCorner + u * horizontal + v * vertical;
    float intensity = normalize(direction).y * 0.5 + 0.5;
    //5 the mix function is the lerp blending function
    half4 answer = mix(bottom, top, intensity);
    //6 writing the answer to the texture
    outTexture.write(answer, pid);
   }

Going through the code:

  1. Check the boundary of the thread grid with the image size grid, to avoid no work threads.
  2. To compute the ratio we get the denominators from the CPU.
  3. The same uniforms values that we used in RayTracing3 on the CPU side (above), we initialize them on the GPU side.
  4. We calculate the (u,v) components for the ray direction the same way we did on the CPU side in the previous example.
  5. The mix function computes the leap blend
  6. Finally each thread writes the answer on the same texture in parallel.

The total running time without saving the file:

Time to execute, without saving the file: 0.037714654 seconds

This is 179.2 times faster! A dramatic increase. Imagine how much energy you saved!
There should be no doubt in your mind when you see big loops in your code, is to think about compute kernels on the GPU.

This will be the first part of this Ray tracing series. Next will be adding objects to our scene.

The entire project can be pulled from my Github account here. Please follow me here and on Medium.

Til next time stay tuned the fun just started.

1656
%d bloggers like this: