Computing on the Go: Part 2

Taras Kushnir
Software engineer at ELEKS

In the previous article, we discussed networking in Advanced Load Balancer. Today, we’ll discuss how to apply Go bindings to C++ libraries, solve interesting issues with Go and finally compare building a Load Balancer with GO and with C++.

Chapter 6. Working with C++ Libraries

One of our objectives was to have the ability to launch computations which are distributed as compiled shared libraries written in C/C++. First versions of Go could link only with C libraries and used gcc inside. Starting from Go 1.2, we can link to C++ libraries.

In general, Go has a Cgo package that allows importing pseudo-package “C” in your code and refer to types and variables like this: C.your_type, C.printf etc.

The process of using C++ libraries usually consists of just a few steps:

  1. Getting headers and binaries of your actual C++ library.
  2. Writing correct compiler and linker flags as #cgo comments in your Go code.
  3. [optional] Writing Go wrappers over calls to C functions (just to make other Go code look like Go code and store all C code in one place).

You can always get more info from the official documentation.

Chapter 7. Solving Interesting Issues

Now it’s time to solve an interesting problem using network Load Balancer. Because our objective is to test the abilities of the adaptation network Balancer to real-world problems, we’ll skip advanced topics like building Kd-tree or alike. We chose ray tracing – a technique for generating images using simulation of moving light beams through a predefined scene of objects. This problem fits our restrains perfectly: each point on an image is calculated independently and can become our task for the Worker.,

7.1. Raytracing Library in C++

After writing a Raytracing library in C++, lets create an interface for Go. In the header file session_inteface.h let’s put three methods createComputator(), traceRays(), destroyComputator() in the extern “C” section. It’s an interface for our computations:

    #ifdef __cplusplus
    extern "C" {
    #endif
     void *createComputator(int heigth, int width, float angle, int objectsNumber, float *objects);
     void traceRays(void *computator, long long startIndex, long long endIndex, float **results);
     void destroyComputator(void *computator);
    #ifdef __cplusplus
    }
    #endif

createComputator() allocates memory for a scene and saves it for future computations. Scene of objects is data needed for calculations of any pixel, so this method will return the void* pointer to the instance of some Computator. We’ll save it in the Go part and pass it to traceRays() later.

traceRays() does actual raytracing. It takes two indices and pointers to the results storage. Indices in parameters are pixel indices. If we have a pixel with coordinates (x, y), it will have the index (y*width + x), so if we know index and width (stored in the Computator), we can restore our coordinates. This is an optimisation for memory: no need to pass any other data to the Computator.

destroyComputator() will obviously free the memory allocated by createComputator().

Of course, I omit implementations, but you can find them at the end of the article.

7.2. Go Wrappers for Worker

On the Go side, I’ve implemented wrappers for calls of C methods and a wrapper for the void* type. Because it’s not obvious how to deal with pointers in Go-C bindings, you can find some answers in the full code below .

    // #cgo CFLAGS: -I../../../go-ray/src/cpp
    // #cgo LDFLAGS: -L../../../go-ray/bin -lraytracer -lstdc++ -lm
    // #include "session_interface.h"
    import "C"
    import (
     "../common"
     "unsafe"
     "bytes"
     "errors"
    )
    
    type CComputationManager struct {
     p unsafe.Pointer
    }
    
    func cCreateComputator(height, width int32, angle float32, objects []float32, objectsNumber int32) CComputationManager {
     cObjects := (*C.float)(unsafe.Pointer(&objects[0]))
     return C.createComputator(C.int(height), C.int(width), C.float(angle), C.int(objectsNumber), cObjects)
    }
    
    func cTraceRays(computator CComputationManager, startIndex, endIndex int64) [][]float32 {
     indicesSize := endIndex - startIndex + 1
     results := make([][]float32, indicesSize)
     refResults := make([]*float32, indicesSize)
     for i := range(results) {
      data := make([]float32, 3)
      results[i] = data
      refResults[i] = (*float32)(unsafe.Pointer(&data[0]))
     }
    
     resultsPtr := (**C.float)(unsafe.Pointer((&refResults[0])))
     C.traceRays(computator.p, C.longlong(startIndex), C.longlong(endIndex), resultsPtr)
    
     return results
    }
    
    func cDestroyComputator(computator CComputationManager) {
     C.destroyComputator(computator.p)
    }

7.3. Requester Side

In the case of the Raytracing problem, the requester creates a scene, sends it to the Balancer and asks the latter to start computations. When the Balancer reports to the Requester that 100% of tasks are computed, the Requester asks the Balancer to collect the results and starts a loop of retrieving results from the Balancer. After unpacking results, the Requester creates an image and saves the computed colours to each pixel. Here is a simplified version of the main functions:

One result is saved to an image data in the memory:

    func saveTaskResult(rh *ResultsHolder, cr *common.ComputationResult) {
     rcr := new(common.RaysComputationResult)
     r := bytes.NewReader(cr.Data)
     rcr.Load(r)
    
     for i:=rcr.StartIndex; i <= rcr.EndIndex; i++ {
      p := rcr.Colors[i - rcr.StartIndex]
      r, g, b := p[0], p[1], p[2]
      x, y := i % width, i / width
      rh.image.SetRGBA(x, y, color.RGBA{
       math.Min(1.0, r) * 255,
       math.Min(1.0, g) * 255,
       math.Min(1.0, b) * 255,
       255})
     }
    }

And then all results are saved to a file on the hard drive:

    func saveResults(rh *ResultsHolder) {
     toimg, _ := os.Create("scene.png")
     defer toimg.Close()
     png.Encode(toimg, rh.image)
    }

Computing_Go_ELEKSlabs

Chapter 8. Results, Conclusions and Useful Links

The speed of Go leaves much to be desired. So, if you are planning to create an application where execution speed is crucial, Go is not an option. But when compared to other languages in terms of time required to develop system software and its running speed is not that important, Go is definitely among the winners.

It took me a bit more than two weeks to develop a custom Load Balancer, a Worker and a Requester and a few more days to adapt it to a specific task (raytracing). In contrast, a similar Load Balancer in C++ took us almost two month to develop plus a few more weeks for the Worker and additional debugging.

When speaking about computation time, I’ve implemented the raytracing core in Go and compared it with the C++ one. It ran almost 10 times slower on my hardware and on my configuration, but that’s why we implemented raytracing in C++, not in Go!

Development of a networked Load Balancer in Go was overall an enjoyable experience. Go seems to be a fairly good language for developing system applications and its ability to link with existing libraries gives you real freedom of choosing what you want to develop easily and what you want to run quickly.

Networked Load Balancer is an example of a fairly complex application that was developed in Go with minimal developer efforts.

Useful links:
Simple raytracer core with bindings to Go
The network Load Balancer itself
The best introduction to Go
Network programming with Go
“Concurrency is not Parallelism” presentation by Rob Pike

What are your thoughts on the matter? Share with us in the comments below.

tags

Comments