Inside the Go playground

Francesc Campoy Flores

Developer Advocate, Gopher

Agenda

The Go playground

play.golang.org

De facto pastebin of the Go community

play.golang.org/p/bJYnajZ6Kp

The Go tour

tour.golang.org

Executable examples on documentation

golang.org/pkg/strings

Executable code on blog posts

blog.golang.org/slices

Executable slides

package main

import "fmt"

func main() {
    fmt.Println("Hello, gophers!")
}

These slides are driven by the present Go tool

go get code.google.com/go.tools/cmd/present

Naive implementation

Architecture

Backend

Let's start with something simple

What could go wrong?

Resource exhaustion

Exhausting memory on the stack

stack overflow

package main

func foo(a [1000]byte) {
    foo(a)
}

func main() {
    foo([1000]byte{})
}

The runtime catches the error and panics.

Too much memory on the heap

out of memory

package main

type list struct {
    buf  [100000]byte
    next *list
}

func main() {
    var l *list
    for {
        l = &list{next: l}
    }
}

Again the runtime catches the error and panics.

Too much CPU time

package main

func main() {
    for {
    }
}

Stealing resources by sleeping

package main

import (
    "fmt"
    "time"
)

func main() {
    fmt.Println("Good night")
    time.Sleep(8 * time.Hour)
    fmt.Println("Good morning")
}

A sleeping program still consumes resources.

Easy way of having a Denial of Service attack.

Accessing things you shouldn't

File system access

User code shouldn't be able to modify the backend's file system.

// +build OMIT

package main

import (
	"log"
	"os"
)

func main() {
    err := os.RemoveAll("/foo")
    if err != nil {
        log.Fatal(err)
    }
}

Network access

// +build OMIT

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"net/http"
)

func main() {
    res, err := http.Get("http://api.openweathermap.org/data/2.5/weather?q=Portland")
    if err != nil {
        log.Fatal(err)
    }
    defer res.Body.Close()

    var w struct {
        Weather []struct {
            Desc string `json:"description"`
        } `json:"weather"`
    }
    if err := json.NewDecoder(res.Body).Decode(&w); err != nil {
        log.Fatal(err)
    }

    fmt.Printf("No need to rush outside, we have %v.", w.Weather[0].Desc)
}

Use your imagination

Countermeasures

Restricting resource usage with ulimit

Default limits are not safe enough.

ulimit could solve this.

-d    maximum size of data segment or heap (in kbytes)

-s    maximum size of stack segment (in kbytes)

-t    maximum CPU time (in seconds)

-v    maximum size of virtual memory (in kbytes)

Native Client

Originally designed to execute native code in Chrome safely.

NaCl defines restrictions on the binaries being executed.

The code runs in a sandbox isolated from the underlying OS.

Isolating process execution with NaCl

We use NaCl to:

Process can only write to stdout/stderr.

Limiting user time

"No sleeping in the playground."

Custom runtime with a fake time package.

func Sleep(d time.Duration) {
    panic("No sleeping in the playground")
}

Restoring functionality

Faking the file system

The syscall package is the only link between user code and the OS kernel.

The playground runtime has a custom syscall package.

File system operations operate on a fake in-memory file system.

// +build OMIT

package main

import (
	"fmt"
	"io/ioutil"
	"log"
)

func main() {
    const filename = "/tmp/file.txt"

    err := ioutil.WriteFile(filename, []byte("Hello, file system\n"), 0644)
    if err != nil {
        log.Fatal(err)
    }

    b, err := ioutil.ReadFile(filename)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("%s", b)
}

Faking the network

All network operations also use the syscall package.

The network stack is also faked in-memory.

// +build OMIT

package main

import (
	"io"
	"log"
	"net"
	"os"
)

func main() {
    l, err := net.Listen("tcp", "127.0.0.1:4000")
    if err != nil {
        log.Fatal(err)
    }
    defer l.Close()

    go dial()

    c, err := l.Accept()
    if err != nil {
        log.Fatal(err)
    }
    defer c.Close()

    io.Copy(os.Stdout, c)
}

func dial() {
	c, err := net.Dial("tcp", "127.0.0.1:4000")
	if err != nil {
		log.Fatal(err)
	}
	defer c.Close()
	c.Write([]byte("Hello, network\n"))
}

Faking the network (continued)

// +build OMIT

package main

import (
	"io"
	"log"
	"net"
	"os"
)

func main() {
	l, err := net.Listen("tcp", "127.0.0.1:4000")
	if err != nil {
		log.Fatal(err)
	}
	defer l.Close()

	go dial()

	c, err := l.Accept()
	if err != nil {
		log.Fatal(err)
	}
	defer c.Close()

	io.Copy(os.Stdout, c)
}

func dial() {
    c, err := net.Dial("tcp", "127.0.0.1:4000")
    if err != nil {
        log.Fatal(err)
    }
    defer c.Close()
    c.Write([]byte("Hello, network\n"))
}

Sleeping in the playground

Go is about concurrency.

We need to demonstrate concurrency in blog posts and talks.

And demonstrating concurrency without time is hard.

What to do if an open source project lacks a feature?

File a bug!

bug 4280

Normal behavior

There's a special goroutine managing timers T.

A goroutine G calls time.Sleep:

1. G adds a timer to the timer heap.

2. G puts itself to sleep.

3. T tells the OS to wake it when the next timer expires and puts itself to sleep.

4. When T is woken up it looks at the timer on the top of the heap, and wakes the corresponding goroutine.

Intermission: deadlocks

package main

func main() {
    c := make(chan int)

    <-c
}

Many flavors of deadlocks.

One common property: all goroutines are asleep.

New behavior

A goroutine G calls time.Sleep:

1. G adds a timer to the timer heap.

2. G puts itself to sleep.

3. The scheduler detects a deadlock, checks the timer heap for pending timers.

4. The internal clock is advanced to the next timer expiration.

5. The corresponding goroutines are woken up.

Sleeping fast

Faking time allows precise sleep durations.

package main

import (
    "fmt"
    "time"
)

func main() {
    start := time.Now()
    fmt.Println(start)

    for i := 0; i < 10; i++ {
        time.Sleep(time.Nanosecond)
        fmt.Println(time.Since(start))
    }
}

So there's no actual sleep?

The playground's write syscall inserts a timestamp before each write.

The front end translates that into a series of "events" that the browser can play back.

// +build OMIT

package main

import (
	"fmt"
	"time"
)

func main() {
    fmt.Println("Good night")
    time.Sleep(8 * time.Hour)
    fmt.Println("Good morning")
}

Returns directly

{
    "Errors":"",
    "Events":[
        {"Message":"Good night\n","Delay":0},
        {"Message":"Good morning\n","Delay":28800000000000}
    ]
}

So the bug was fixed

play.golang.org/p/3fv0L3-z0s

And people were happy

play.golang.org/p/rX_3WcpUOZ

Very happy

play.golang.org/p/P-Dk0NH_vf
play.golang.org/p/NOycgN2i6b

References

These slides: talks.golang.org/2014/playground.slide

More about the Go tour:

More about Go on NaCl:

Thank you

Francesc Campoy Flores

Developer Advocate, Gopher

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)