Toward Go 1.3

(and beyond)

Andrew Gerrand

Gopher

Go 1.3

Code freeze is March 1, 2014.

Release is June 1, 2014.

(Six months after Go 1.2, released December 1, 2013.)

A to-do list

After Go 1.2 the Go contributors compiled a to-do list:

The list is aspirational; not all of it will get done.

This talk is based on that list.

100% precise GC

Finally!

Copying stacks (1/2)

Go 1.2's stack split mechanism has a "hot split" problem.

Copying (or "contiguous") stacks are grown by reallocation and copying.
Resolves the "hot split" problem.
Makes smaller initial stacks practical - more goroutines in the same space.

Copying stacks (2/2)

Dmitry's bag of performance tricks

Runtime changes:

And many more to come...

Channel rewrite

Goals:

Non-goals:

sync.Pool (1/2)

Many Go libraries include custom thread-safe free lists, like this:

var objPool = make(chan *Object, 10)

func obj() *Object {
    select {
    case p := <-objPool:
        return p
    default:
    }
    return NewObject()
}

func objPut(p *Object) {
    select {
    case objPool <- p:
    default:
    }
}
p := obj()
// use p
objPut(p)

sync.Pool (2/2)

The sync.Pool type provides a general thread-safe global free list.

It allows the runtime to reclaim entries when appropriate
(for example, during garbage collection).

var objPool = sync.Pool{
    New: func() interface{} {
        return NewObject()
    },
}
p := objPool.Get().(*Object)
// use p
objPool.Put(p)

This is an experimental type and might not be released.

Native Client port

Native Client (NaCl) is a restricted execution environment for x86 binaries.

Notably used to run compiled binaries inside Google Chrome.
NaCl also provides a tool for executing command-line binaries

Go 1.3 targets that command-line tool for 32-bit and 64-bit x86 architectures.
(NaCl supports 32-bit ARM, but we have no plans to support it.)

The Go Playground uses the NaCl tool chain to safely execute untrusted programs.

The NaCl tool chain includes the fake time, network, and file system capabilities of the playground.

OS ports

Solaris: work in progress, on track for Go 1.3.

DragonflyBSD: work is done, looking for a maintainer.

Plan 9: still not finished.

darwin/arm, android/arm: a contributor is working on these, some way to go.

The go command and fsnotify

In Go 1.2, go build stats every dependent source file to see whether they have changed.

This is a big chunk of total build time.

The proposed "go background" command starts a daemon that watches source files for changes.

When building, the go commands can ask the daemon which files have changed.

A new os/fsnotify package will be added to the standard library to support the go command.

A proposed interface is discussed here:

Support for linking against Objective C code

The Go 1.2 tool chain can link against C++ code using cgo (but you need to write a small C bridge into the C++ code).

The same can be done for Objective C code, with some modifications to the go tool.

This will make it easier to write native OS X applications.

Address binary bloat

Go binaries are getting pretty big. Rob ran an experiment:

As an experiment, I built "hello, world" at the release points for go 1.0. 1.1, and 1.2.
Here are the binary's sizes:

% ls -l x.1.?
-rwxr-xr-x  1 r  staff  1191952 Nov 30 10:25 x.1.0
-rwxr-xr-x  1 r  staff  1525936 Nov 30 10:20 x.1.1
-rwxr-xr-x  1 r  staff  2188576 Nov 30 10:18 x.1.2

Go binaries contain several sets of debugging symbols (for gdb, profilers, reflection, etc).

We intend to rationalize these as part of some work on the linker.
Speaking of which...

Linker overhaul (1/3)

The gc tool chain is a bit unconventional.

The compilers don't emit machine code but an intermediate assembly language.

The linker translates it into machine code.

The packages can be compiled in parallel by independent runs of the compiler,
but the linking must be done by a single linker process after compilation is complete.

The gc linker has become a bottleneck in building programs
because it does more work than a typical linker.

Linker overhaul (2/3)

The Go 1.2 linker's job can be split into two parts:

Linker overhaul (3/3)

In Go 1.3, much of the old linker is moved to a liblink library that is then used by assemblers and compilers (6a, 6c, 6g, etc). This allows more work to be done in parallel.

And because the linker is much simpler now, we can rewrite it in Go.

Compiler overhaul

The "gc" tool chain is based on the Plan 9 C compilers.

The assemblers, C compilers, and linkers were lifted wholesale.

The Go compilers are new C programs that fit into that tool chain.

Wouldn't it be nice to have a Go compiler written in Go?

Compiler overhaul: why C then?

Many benefits to writing the compiler in C:

Today, Go does exist and is stable as of Go 1.
These benefits not as relevant now.

Compiler overhaul: why Go now?

The benefits of a Go-based compiler:

Compiler overhaul: the plan

Not a rewrite.

Translate the C compilers to Go.

Write and use an automatic translator to do this.

Start the process with Go 1.3 and continue in future releases.

Compiler overhaul: five phases

Compiler overhaul: bootstrapping

Must have a way to build the compiler from scratch.

Our plan is that the Go 1.3 compiler must compile using Go 1.2, and Go 1.4 must compile with Go 1.3, and so on.

Write a shell script to do this automatically. Bootstrap once per machine.

This scales poorly over time, so we might write a back end for the compiler that generates C code, and keep the C version of the compiler sources checked in.

Compiler overhaul: alternatives

Write new compilers from scratch?

Translate the compiler manually?

Translate just the back ends and connect to go/parser and go/types immediately?

Discard the current compilers and use gccgo (or go/parser and go/types and LLVM)?

Lots of small things

As with previous releases, we'll see a long tail of small fixes and changes.

Thank you

Andrew Gerrand

Gopher