7. Concurrency

Do not communicate by sharing memory; instead, share memory by communicating. — Effective Go

If you observe, you could see many things happening around you at any given time. This is how the world function - the train is gently moving, passengers talking each other, farmers working in the field and many other things are happening simultaneously. We can say, the world we live in function concurrently.

Go has built-in concurrency features with syntax support. The Go concurrency is inspired by a paper published in 1978 by Tony Hoare. The paper title is Communicating sequential processes [1].

Go has some new terminologies and keywords related to concurrent programming. The two important words are goroutine and channel. This chapter will go through these concepts and walk through some examples to further explain concurrency in Go.

The Go runtime is part of the executable binary created when compiling any Go code. The Go runtime contains a garbage collector and a scheduler to manage lightweight threads called Goroutines. Goroutine is a fundamental abstraction to support concurrency. Goroutine is an independently executing part of the program. You can invoke any number of goroutines and all of them could run concurrently.

Goroutines can communicate to each other via typed conduits called channels. Channels can be used to send and receive data.

7.1. Goroutine

Goroutine is like a process running in the background. A function with go keyword as prefix starts the goroutine. Any function including anonymous function can be invoked with go keyword. In fact, the main function is a special goroutine invoked during the starup of any program by the Go runtime.

To understand the Goroutine better let’s look at a simple program:

package main

import (
    "fmt"
    "time"
)

var msg string

func setMessage() {
    msg = "Hello, World!"
}

func main() {
    go setMessage()
    time.Sleep(1 * time.Millisecond)
    fmt.Println(msg)
}

In the above program, setMessage function is invoked as a goroutine in line no 15 using the go keyword. If you run this program, you will get the hello world message printed. If you change the sleep time to Zero, the message will not be printed. This is because, the program exits when main function completes execution. And in this case, since setMessage is called as a goroutine, it goes to background and main goroutine execution continues. In the earlier case when the time sleep was 1 second, the goroutine gets some time to execute before main completed. That’s why the msg value is set and printed.

7.2. Channels

Multiple goroutines can communicate using channels. Channels can be used to send and receive any type of values. You can send and receive values with this channel operator: <-

This is how to declare a channel of int values:

ch := make(chan int)

To send a value to ch channel:

ch <- 4

To receive a value from ch channel and assign to a variable:

v := <-ch

You can also receive value without really assigning:

<-ch

Sending and receiving values from channels becomes a blocking operation. So, if you try to receive value from a channel, there should be some other part of the code which sends a value this channel. Until a value sends to the channel, the receiving part of the code will block the execution.

Here is an example:

package main

import (
    "fmt"
)

var c = make(chan int)
var msg string

func setMessage() {
    msg = "Hello, World!"
    c <- 0
}

func main() {
    go setMessage()
    <-c
    fmt.Println(msg)
}

In the above example, an int channel is assigned to a global variable named c. In line number 17, immediately after calling goroutines, channel is trying to receive a value. This becomes a blocking operation in the main goroutine. In line number 12, inside the setMessage function, after setting a value for msg, a value is send to the c channel. This will make the operation to continue in the main goroutine.

7.3. Waitgroups

Go standard library has a sync package which provides few synchronization primitives. One of the mechanism is Waitgroups which can be used to wait for multiple goroutines to complete. The Add function add the number of goroutines to wait for. At the end of these goroutines call Done function to indicate the task has completed. The Wait function call, block further operations until all goroutines are completed.

Here is a modified version of the previous example using Waitgroups.

package main

import (
    "fmt"
    "sync"
)

var msg string
var wg sync.WaitGroup

func setMessage() {
    msg = "Hello, World!"
    wg.Done()
}

func main() {
    wg.Add(1)
    go setMessage()
    wg.Wait()
    fmt.Println(msg)
}

In the above example, the Add method at line number 17 make one item to wait for. The next line invoke the goroutine. The line number 19, the Wait method call blocks any further operations until goroutines are completed. The previous line made goroutine and inside the goroutine, at the end of that goroutine, there is a Done call at line number 13.

Here is another example:

package main

import (
    "fmt"
    "sync"
    "time"
)

func someWork(i int) {
    time.Sleep(time.Millisecond * 10)
    fmt.Println(i)
}

func main() {
    var wg sync.WaitGroup
    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(j int) {
            defer wg.Done()
            someWork(j)
        }(i)
    }
    wg.Wait()
}

7.4. Select

The select is a statement with some similarity to switch, but used with channels and goroutines. The select statement lets a goroutine wait on multiple communication operations through channels.

Under a select statement, you can add multiple cases. A select statement blocks until one of its case is available for run – that is the channel has some value. If multiple channels used in cases has value readily avaibale, select chooses one at random.

Here is an example:

package main

import "time"
import "fmt"

func main() {

    c1 := make(chan string)
    c2 := make(chan string)

    go func() {
        time.Sleep(time.Second * 1)
        c1 <- "one"
    }()
    go func() {
        time.Sleep(time.Second * 2)
        c2 <- "two"
    }()

    for i := 0; i < 2; i++ {
        select {
        case msg1 := <-c1:
            fmt.Println("received", msg1)
        case msg2 := <-c2:
            fmt.Println("received", msg2)
        }
    }
}

7.5. Buffered Channels

Buffered channels are channels with a given capacity. The capacity is the size of channel in terms of number of elements. If the capacity is zero or absent, the channel is unbuffered. For a buffered channel communication succeeds only when both a sender and receiver are ready. Whereas for a buffered channel, communication succeeds without blocking if the buffer is not full (sends) or not empty (receives).

The capacity can be given as the third argument to make function:

make(chan int, 100)

Consider the below example:

package main

import "fmt"

func main() {
    ch := make(chan string, 2)
    ch <- "Hello"
    ch <- "World"
    fmt.Println(<-ch)
    fmt.Println(<-ch)
}

The ch channel is a buffered channel, this makes it possible to send value without any receiver present.

7.6. Channel Direction

When declaring a function with channels as input parameters, you can also specify the direction of the channel. The direction of channel declares whether it can only receive or only send values. The channel direction helps to increases the type-safety of the program.

Here is an example:

package main

import "fmt"

func sendOnly(name chan<- string) {
    name <- "Hi"
}

func receiveOnly(name <-chan string) {
    fmt.Println(<-name)
}

func main() {
    n := make(chan string)

    go func() {
        fmt.Println(<-n)
    }()

    sendOnly(n)

    go func() {
        n <- "Hello"
    }()

    receiveOnly(n)
}

In the above example, the sendOnly function define a channel variable which can be only used for sending data. If you tried to read from that channel within that function, it’s going to be compile time error. Similary the receiveOnly function define a channel variable which can be only user for receive data. You cannot send any value to that channel from that function.

7.7. Lazy Initialization Using sync.Once

The sync package provide another struct called Once which is useful for lazy initialization.

Here is an example:

import (
    "sync"
)

type DB struct{}

var db *DB
var once sync.Once

func GetDB() *DB {
    once.Do(func() {
        db = &DB{}
    })
    return db
}

If the above GetDB function is called multiple times, only once the DB object will get constructed.

7.8. Exercises

Exercise 1: Write a program to download a list of web pages concurrently using Goroutines.

Hint: Use this tool for serving junk content for testing: https://github.com/baijum/lipsum

Solution:

package main

import (
        "io/ioutil"
        "log"
        "net/http"
        "net/url"
        "sync"
)

func main() {
        urls := []string{
                "http://localhost:9999/1.txt",
                "http://localhost:9999/2.txt",
                "http://localhost:9999/3.txt",
                "http://localhost:9999/4.txt",
        }
        var wg sync.WaitGroup
        for _, u := range urls {
                wg.Add(1)
                go func(u string) {
                        defer wg.Done()
                        ul, err := url.Parse(u)
                        fn := ul.Path[1:len(ul.Path)]
                        res, err := http.Get(u)
                        if err != nil {
                                log.Println(err, u)
                        }
                        content, _ := ioutil.ReadAll(res.Body)
                        ioutil.WriteFile(fn, content, 0644)
                        res.Body.Close()
                }(u)
        }
        wg.Wait()
}

7.8.1. Additional Exercises

Answers to these additional exercises are given in the Appendix A.

Problem 1: Write a program to watch log files and detect any entry with a particular word.

7.9. Summary

This chapter explained how to use Go’s concurrency features. You can choose channels or other synchronization techniques, depending on your problem. This chapter covered goroutines and how to use channels. It also covered Waitgroups and Select statements. Additionally, it covered buffered channels and channel direction. Finally, the chapter briefly discussed the sync.Once function.

[1]

http://usingcsp.com