"Go is efficient, scalable, and productive. Some programmers find it fun to work in; others find it unimaginative, even boring. In this article we will explain why those are not contradictory positions. Go was designed to address the problems faced in software development at Google, which led to a language that is not a breakthrough research language but is nonetheless an excellent tool for engineering large software projects."

— Rob Pike, 2012

Introduction

Joyent has a storied history with regards to Go. Without providing modern treatment to past issues, it is undeniable that many, if not most (as of 2017) distributed systems are being written in Go. To that extent, Joyent is embracing of Go’s contribution to distributed, enterprise, production-grade computing and celebrates the software engineering ethos held by many in the Go community because their beliefs are aligned with our principles.

In this document we will define and articulate Joyent’s evolving set of best-practices with regards to how to Go.

Each item has a recommendation and a rationale. Over time these recommendations and rationale are apt to evolve and be refined to further reflect our experiences. Omissions or gaps in the list of items, the recommendations, or their rationale is almost certainly unintentional. At times several of the recommendations or rationale become pithy due to time constraints or the perceived lack of a need to provide a more robust explanation, and similarly, this is unintentional. If an issue warrants additional discussion, please open an issue.

To provide feedback, bug fixes, request clarification, or initiate a discussion, please open an RFD issue and reference RFD 106 along with the necessary specifics. For issues where there is a debate or discussion, this document will include a link back to the individual issues where a particular item was addressed or debated.

This RFD makes use of terminology defined in BCP 14 (RFC 2119 and RFC 8174, i.e. "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL").

Core Beliefs

At Joyent we have the following beliefs regarding software:

  • Build robust, trusted systems

    • Build secure systems

    • Data integrity

  • Operability

  • Maintainability

  • Availability

  • Debuggablity

It’s not possible to prescribe a fool-proof set of best-practices but it is possible to influence the outcome. To wit, in order to satisfy these beliefs, we take principled stances on the following (in no particular order, but roughly ordered from philosophical to practical):

  • Version of Go

  • Project Structure

  • Workflow

  • Style

  • $EDITOR Integration

  • vendor/ Management

  • Naming Conventions

  • API stability

  • Static Analysis

  • Explicit Types

  • return Parameters

  • Interface Receivers

  • const

  • Constructing and Using Bitmasks

  • Documentation

  • Use of context

  • Deadline Timers and Timeouts

  • pprof

Beliefs

Build Robust, Trusted Systems

  • Data Integrity

  • Predictability

  • Security

Operability
Maintainability
Availability
Debuggablity
Principles

1. Version of Go

Recommendation

  1. The latest released version or a current release candidate of Go SHOULD be used for day-to-day development.

  2. A released version of Go or a release candidate , until the next version becomes available, MUST be used for building released binaries.

  3. The latest version of Go SHOULD be used for release engineering.

  4. master MAY be used for development and is RECOMMENDED for Continuous Integration (CI) testing.

  5. Every time the version of go(1) changes the contents of $GOPATH/pkg MUST be cleaned out:

    $ rm -rf $GOPATH/pkg/

Rationale

The Go team and project are consistently advancing the state of the Go compiler and standard library. Released versions and release candidates are considered fit for production and should therefore be used in production. Upgrade early and often.

When release candidates are available, they should be used until the next finalized release is made available.

The Go team and contributors have a good track record of advancing the state of the compiler in terms of stability and performance. Tendencies toward risk aversion frequently have a higher cost (e.g. security, correctness, performance, or stability) than absorbing the cost of any incremental upgrade.

Use of master is not peril free, however it is good way of staying current and doesn’t cause much grief during development, but frequently does improve both correctness and quality.

The version of go(1) is tightly coupled to the cached object files in $GOPATH/pkg and there is no stability contract. Cleaning out $GOPATH/pkg periodically is an ounce of prevention whose tiny cost is worth more than the pound of debugging.[1]

Use of tools like vg(1) MAY be useful or considered in the future but are NOT REQUIRED: most of these recommendations are intended to aid productivity and decrease friction, not constrain how someone goes about working.

2. Project Structure

Recommendation

  1. $GOPATH SHOULD be set to $HOME/go and SHOULD be incorporated into your shell’s environment.

  2. All development SHOULD be done within $GOPATH/src.

  3. $GOPATH/bin SHOULD be part of PATH and before /usr/local or /opt/local (i.e. before system or package manager managed binaries).

    $ export GOPATH=$HOME/go
$ export PATH=$GOPATH/bin:$PATH

  4. Where appropriate, it is RECOMMENDED to make use of monolithic repositories (mono-repo).

  5. Publicly consumable libraries or programs SHOULD be pushed to a distinct canonical public location and automatically synchronized to the internal codebase.

Rationale

Starting in Go 1.8, go(1) defaulted to $HOME/go as its default GOPATH. It is not strictly necessary to set GOPATH, however it is still advised to make this implicit default explicit. Many tools or pieces of software test for the environment variable GOPATH instead of using using go env GOPATH.

In Go 1.8, the Go project defaulted to $HOME/go as the default value for $GOPATH. Use of one-workspace per project is counter-productive and establishes a workflow that is orthogonal to the ethos of the prevailing Go ecosystem. This isn’t to say there are times where this is necessary (i.e. clean-room verification or maintenance of vendor/), but the default practice SHOULD be to work inside of a single $GOPATH workspace [2].

Go’s tooling makes it especially productive to move all libraries and programs into the same codebase so that refactoring can commence in atomic units of work. In particular, making sweeping changes via gofmt(1) -r is easy to accomplish in a single repository and commit. Breaking apart individual libraries into discrete repositories fragments the codebase with no isolation guarantees that Version Control System (VCS) doesn’t already provide. Contrast that with having all libraries and programs in the same codebase, it is now possible to move the entire codebase forward in an atomic transaction [3]. Additional arguments in support of monorepos include:

Publicly reusable components, however, SHOULD be discretely usable.

3. Workflow

Recommendation

Engineer workflow changes based on whether or not you have write-privileges to the target repository.

If you HAVE write access to a repository and it is Github-like:

  1. Checkout the repository:

    $ go get -d my.git.server/my_org/my_project

  2. Create a branch for your change:

    $ cd $GOPATH/src/my.git.server/my_org/my_project
$ git checkout -b my-branch-name

  3. Commit your change(s):

    $ git commit

  4. Push your change to origin:

    $ git push -u origin my-branch-name

  5. Submit a Pull Request (PR).

  6. You SHOULD obtain a review. For all changes deemed to be trivial this is not necessary, however the change MUST be made through a PR in order to to aid in a quick backout commit.

  7. Automated regression tests MUST complete and pass.

  8. If the velocity of change for the repository is low enough, a CHANGELOG entry for the project SHOULD be committed to the PR as the final step before merging the PR. If the velocity of the repository is too high, the CHANGELOG entry for the project MAY be added after the PR has been merged.

  9. Merge the PR. If the history of the PR is messy with unhelpful commits (e.g. "fix typo", "update test"), perform a squash merge with a detailed, high-quality commit message that has been approved by the rest of the team. Detail that can’t be expressed in the commit message should be outlined in code comments.

  10. Pull the latest changes:

    $ git checkout master && git pull origin master

  11. Delete your local branch:

    $ git branch -d my-branch-name

  12. Delete your branch from the server (e.g. my-branch-name).

If you do NOT HAVE write access to a repository the workflow is largely the same except you MUST create a fork of the repository:

  1. Checkout the original repository:

    $ go get -d -v my.git.server/my_org/my_project

  2. Fork the upstream repository to your individual user account.

  3. Add the remote for your repository:

    $ cd $GOPATH/src/my.git.server/my_org/my_project
$ git remote add me my.git.server/my_user/my_project

  4. Create a branch for your change:

    $ git checkout -b my-branch-name

  5. Commit your change(s)

  6. Push your change to me:

    $ git push -u me my-branch-name

  7. A CHANGELOG entry SHOULD be incorporated into the PR unless the upstream project will write the CHANGELOG entry for you.

  8. Submit a Pull Request (PR).

  9. Wait for the upstream provider to merge your PR.

  10. Pull the latest changes:

    $ git checkout master
$ git pull origin master

  11. Delete your local branch:

    $ git branch -d my-branch-name

Work MUST be completed within the same directory as the upstream source and not the path to your fork of an upstream module (i.e. CORRECT: $GOPATH/src/my.git.server/upstream_org/my_project WRONG: $GOPATH/src/my.git.server/my_user/my_project). Instead use the path of your upstream source, but use a different git remote URL.

If you HAVE write access to a repository and it is Gerrit:

  1. Checkout the repository:

    $ git clone --origin gerrit https://my.git.server/my_org/my_project.git

  2. Create a branch for your change:

    $ git checkout -b my-branch-name

  3. Commit your change(s):

    $ git commit

  4. Push your change to origin:

    $ git push gerrit HEAD:refs/for/master

  5. You MUST obtain a review.

  6. Automated regression tests MUST complete and pass.

  7. A CHANGELOG entry MUST be committed to the PR as the final step before merging the PR.

  8. Merge the PR. If the history of the PR is messy with unhelpful commits (e.g. "fix typo", "update test"), perform a squash merge with a detailed, high-quality commit message that has been approved by the rest of the team. Detail that can’t be expressed in the commit message should be outlined in code comments.

  9. Pull the latest changes:

    $ git checkout master
$ git pull origin master

  10. Delete your local branch:

    $ git branch -d my-branch-name

4. Style

Recommendation

  1. All code MUST pass through gofmt(1). gofmt(1) SHOULD be executed with the -s flag.

  2. Lines SHOULD wrap at 80 characters.

Rationale

The particular brand of tribal fascism that extends from gofmt(1) increases the overall productivity of the entire Go community by creating a single dialect of Go that is universal across projects, teams, and organizations. Being able to drop into any arbitrary Go project, regardless of the copyright, and be able to understand the codebase quickly is a universal boon.

The only observable consequence to adhering to `https://golang.org/cmd/gofmt/[gofmt(1)]’s set of style norms is the cost of shedding the sentimental attachment to a preference for "my way of doing things". Developing a personal or project-wide coding style takes discipline to adhere to, an understanding of the style guide’s rules (including their rationale), and an eagle-eye to enforce. Investment in such skills and the pride attached to that skill-set is near-zero in the Go community. Shedding personal preference - justified or not - in favor of a prescribed doctrine is a tangible hurdle to overcome.

The computing industry has been well served by project-wide style guidelines in part because this created a sufficiently high barrier to entry which acted as a litmus-test to ensure tribal norms were understood and communicated to new members of the tribe. With many of the original industrial programming languages being riddled with undefined behavior (e.g. C or C++), style guides helped communities of engineers ship more reliable code and with fewer bugs because project-wide idioms had a tendency to be put in place for a reason.

Even before go(1) adopted gofmt(1) to enforce Go’s single-style guideline, ident(1) existed as a crude tool for enforcing style (crude to the point that ident(1) was eschewed because it was unable to perform at the levels required for a developer tool). In no way should clang-format(1) or clang-tidy(1) be lumped into the same league of correctness as ident(1) because clang-format(1) and clang-tidy(1) [4] recreate the AST before rewriting code (vs the brute-force text-level tokenization performed by most ident(1) implementations).

The value and merit of individual or project preferences with regards to the artistry stemming from style guides has been eclipsed by the value generated from participating in the open, code-sharing world of the Go Open Source ecosystem. Go came into the world with a lack of legacy, fragmentation, or tribalism and has largely remained an unfragmented community in large part due to its fungability of both Go developers and code that can be readily shared across either projects or organizations.

gofmt(1) SHOULD be used in place of the go(1) tool’s fmt command because:

  1. gofmt(1) supports the -s flag to simplify code where possible.

  2. go fmt calls gofmt(1): src/cmd/go/internal/fmtcmd/fmt.go L42-L71

  3. gofmt(1) supports programmatic rewriting of the code base via the -r flag.

  4. Code SHOULD be fungible. Go’s simple syntax, emphasis on readability, and "side-effect"-free code largely make this a reality.

Additional rationale is included in Robert Griesemer’s talk on The Cultural Evolution of gofmt.

5. Developer Tools

Recommendation

The following tools are RECOMMENDED for development:

  1. goimports(1):

    $ go get -u golang.org/x/tools/cmd/goimports

  2. guru(1):

    $ go get -u golang.org/x/tools/cmd/guru

  3. godoc(1):

    $ go get -u golang.org/x/tools/cmd/godoc

  4. gorename(1):

    $ go get -u golang.org/x/tools/cmd/gorename

  5. gomvpkg(1):

    $ go get -u golang.org/x/tools/cmd/gomvpkg

Rationale

goimports(1)

Manually maintaining import declarations is a tedious waste of time. goimports(1) gets this right 99% of the time and increases productivity significantly once integrated into your $EDITOR. goimports(1) does periodically get the package wrong when there is ambiguity, but with a nudge in the right direction it doesn’t go off the rails again for a particular source file.

guru(1)

guru(1) SHOULD be integrated into your $EDITOR because it enables quick, authoritative traversal of codebases. guru(1) is a huge productivity bump and can’t have enough good things said about it. Watch Navigating Unfamiliar Code with the Go Guru and read Using Go Guru: an editor-integrated tool for navigating Go code. Spiritually guru(1) could probably attribute a material portion of its inspiration with ctags(1), cscope(1), and LXR, however guru(1) is much more sophisticated.

The list of guru(1)s functionality includes (as of July 2017, and taken from golang.org/x/tools/cmd/guru):

Identifier Queries
what

The what query describes the current source position as rapidly as possible. It is not intended to be invoked directly by the user, but it allows editors to provide immediate feedback in the UI whenever the cursor position changes. It can be used to highlight all identifiers that are equivalent to current one.

definition

The definition query finds the declaration of the selected identifier. In some editors, it may jump the cursor directly to that location.

referrers

The referrers query finds references to the selected identifier, scanning all necessary packages within the workspace.

freevars

The freevars query enumerates the free variables of the selection. "Free variables" is a technical term meaning the set of variables that are referenced but not defined within the selection, or loosely speaking, its inputs.

Type Queries
describe

The describe query shows various properties of the selected syntax: its syntactic kind, the type of an expression, the value of a constant expression, the size, alignment, method set, and interfaces of a type, the declaration of an identifier, and so on. You may describe almost any piece of syntax, and guru(1) will print all the useful information it can.

implements

The implements query shows interfaces that are implemented by the selected type and, if the selected type is itself an interface, the set of concrete types that implement it. An implements query on a value reports the same information about the expression’s type. An implements query on a method shows the set of abstract or concrete methods that are related to it.

Call Graph Queries
callees

The callees query shows the possible call targets of the selected function call site. The cursor or selection must be within a function call expression; the selection need not be exact.

callers

The callers query shows the possible callers of the function containing the selection.

callstack

The callstack query shows an arbitrary path from the root of the call graph to the function containing the selection. This may be useful to understand how the function is reached in a given program.

Alias Queries
pointsto

The pointsto query shows the set of possible objects to which a pointer may point. It also works for other reference types, like slices, functions, maps, and channels.

whicherrs

The whicherrs query reports the set of possible constants, global variables, and concrete types that may appear in a value of type error. This information may be useful when handling errors to ensure all the important cases have been dealt with.

peers

The peers query shows the set of possible sends/receives on the channel operand of the selected send or receive operation; the selection must be a <- token.

godoc(1)

godoc(1) SHOULD be installed in order to have quick access to formatted documentation. Before committing a new body of work, the documentation for the package should be inspected. A strong cofactor in determining the reusable value of software is its documentation (see also: network effect).

gorename(1) and gomvpkg(1)

gorename(1) SHOULD be used for renaming package, function, and method members (i.e. const, func, var, and type).

While less commonly needed, gomvpkg(1) SHOULD be used when moving packages around because it updates the necessary import declarations in a given scope.

6. $EDITOR Integration

Recommendation

This section is NOT making a recommendation regarding any particular $EDITOR.[5] This section is, however making a strong recommendation that your $EDITOR include the following interrogations in order to aid in maximal productivity:

  1. goimports(1) is SHOULD be added as a save hook. $EDITOR instructions are found at: godoc.org/golang.org/x/tools/cmd/goimports.

  2. guru(1) SHOULD be integrated into your $EDITOR as a plugin. Binding "jump-to-definition" to an easy-to-access keybinding is strongly RECOMMENDED. Instructions can be found at https://golang.org/s/using-guru. See guru(1) rationale.

  3. gorename(1) SHOULD be integrated into your $EDITOR as a plugin (instructions for emacs, vim).

Specific editor integrations (alphabetically sorted):

  • emacs users SHOULD look at go-mode.el and MAY OPTIONALLY investigate integrating gocode. With guru(1) installed, enabling go-guru-hl-identifier-mode is RECOMMENDED when navigating code.

  • JetBrains users SHOULD look at Gogland.

  • vim users SHOULD look at vim-go and MAY OPTIONALLY investigate integrating gocode.

Rationale

$EDITOR preferences and configuration is an intensely personal subject. Integrating gofmt -s -w $FILE may be a benefit to your individual workflow. Some people who use emacs really like flymake or flycheck or go-guru-hl-identifier-mode, whereas others don’t like the extra background CPU they incur. The list above is both lightweight and common. Additional suggestions or tips to improve $EDITOR productivity are welcome.

7. vendor/ Management

Recommendation

  1. Forked and cached libraries in vendor/ MUST be managed via the dep(1) tool:

    $ go get -u github.com/golang/dep/cmd/dep   (1)
$ dep status                                (2)
$ dep ensure                                (3)
$ dep ensure -update                        (4)
$ dep init                                  (5)
    1 Install dep(1)
    2 Checking the status of dependencies
    3 ensure, the main subcommand
    4 Update Dependencies
    5 dep(1) Setup

  2. In a monorepo, whomever wants to update the bits in vendor/ and Gopkg.lock MAY update the version, however they MUST:

    1. take responsibility for making the change (and updating code as necessary).

    2. testing the change.

    3. communicate the change with consumers of the library.

    4. receive approval from teams receiving the update.

  3. Gopkg.toml SHOULD NOT lock a version to a specific version without reason. Valid reasons include:

    1. Upstream did in-fact change something that requires local attention and the cost of fixing the change locally is currently too high.

    2. Upstream did in-fact commit something that is materially broken and the cost of fixing the bug upstream is too high.

  4. Release CI runs MUST use the version specified in Gopkg.lock.

  5. Non-release CI SHOULD be able to report vendor drift via dep status. If the CI environment is safely isolated to the extent that you’re willing to run uninspected code from upstream, dep ensure -update SHOULD be run and report breakage caused by upstream changes.

  6. vendor/ and Gopkg.lock SHOULD be updated regularly in order to prevent forklift upgrades.

  7. Understanding the difference between the Gopkg.toml and Gopkg.lock files is REQUIRED.

Rationale

As of Gophercon 2017, dep(1) is on track to becoming the community defacto vendor/ management tool (this is also on track according to their roadmap). If a project is using either godep(1) or govendor(1), please make plans to upgrade to dep(1).

See also github.com/golang/dep/issues/281 and the dep(1) FAQ. Gopkg.toml documentation is RECOMMENDED reading, too.

CI automatically updating dependencies in non-release builds provides a motivation for vendor/ to more closely track the upstream’s most recently tagged version or master. Tracking more frequent, small changes is less error prone than large "#yolo updates."

Confusingly named, dep(1) is not the same as godep(1). dep(1) is the future, not godep(1).
CI systems can only run dep ensure -update if the CI systems are capable of running untrusted, foreign code (or some other compensating control is in place).

8. Naming Conventions

Recommendations

  1. Go software SHOULD conform to the recommendations outlined in the following resources:

    1. Package Names

    2. Style guideline for Go packages

    3. Effective Go

    4. Organizing Go Code

    5. Organizing Go Code

  2. Package authors MAY deviate from these conventions IF they have sought feedback from engineers who have sufficient experience writing Go libraries.

  3. When working with a forked copy of a package, package import paths MUST continue to use the canonical, public import path. See comments in workflow.

  4. Package aliases SHOULD be used when necessary and there are two libraries with the same package name.

  5. Programs SHOULD NOT explicitly import a package into the current namespace (i.e. do not use import . "lib/math" Sin).

  6. Programs MAY import a package for their side effects using the black identifier (i.e. a package’s init() MUST run). For example:

    import (
	"database/sql" (1)
	"fmt"

	_ "github.com/lib/pq" (2)
)
    1 imports database/sql
    2 Invoke’s github.com/lib/pq's init() method because it registers itself with the sql package.

Rationale

Naming is one of the hard things in software. The package semantics of Go help with this dilemma and minimize the blast-radius of poorly chosen names. With tools like guru(1) commonly in use, the practice of encoding extraneous type and package information into variable names is non-idiomatic and frowned upon. The burden for good naming and exported functions falls on library authors.

9. API Stability

Recommendation

  1. APIs within a single project SHOULD use tightly-coupled function signatures.

  2. Refactoring APIs within a single project SHOULD use gofmt(1)'s -r flag to migrate function signatures.

  3. External APIs that are loosely coupled across projects AND potentially unstable SHOULD use struct inputs. For example:

    package mypkg
struct MyFuncInputs {
  ArgA string
  ArgB int
  ArgC bool
}
func MyFunc(args MyFuncInputs) {
  // ...
}

    on the caller’s side:

    mypkg.MyFunc(MyFuncInput{
  ArgA: "foo",
  ArgB: 0xba72,
  Argc: true,
})

  4. Required arguments SHOULD be extracted from the input struct.

  5. Optional arguments or parameters that are subject to change by the authors of the library SHOULD be included in the input struct in order to provide loose coupling between the library and its consumers.

  6. Where input arguments are not reused across API calls, use of stack-initialized (e.g. MyFuncInput{}) input structs SHOULD be used (vs heap initialized, e.g. &MyFuncInput{}).

Rationale

Tightly coupled interfaces within the same project SHOULD be treated as local where possible. The onus for maintaining the API MUST be on the author changing the function signature. Tools that programmatically rewrite the codebase SHOULD be employed to make the change. The entire change SHOULD be merged as a single operation. Sweeping mechanical changes SHOULD be committed independent of either functional or behavioral changes.

External APIs that are loosely coupled where consumers of a library are apt to not update all of their call sites need to acknowledge that it is a maintenance cost to enforce tight coupling between a project and an external library. Use of struct input arguments allows:

  1. library maintainer to advance the functionality of their library independently

  2. consumers of the library to update without fear of breaking their API

This recommendation stems from the following hypothetical:

Imagine a function signature is:

func MyFunc(a string, b int) { /* ... */ }

and the authors of MyFunc() decide the function signature needs to be updated to:

func MyFunc(a string, b int, c bool) { /* ... */ }

All consumers of MyFunc() must update to the new signature. In some cases this compile-time breakage may be desirable in order for MyFunc()'s authors to communicate a breaking change or semantic change that requires some level of understanding by the consumer. In other cases, the authors of MyFunc() may have added new functionality without changing the semantic meaning of the contract API. In the latter case, adding functionality to MyFunc() requires source-code level API flexibility with a permissive interfaces in order to minimize the maintenance cost incurred by consumers.

This could be achieved by adding an additional variadic function argument:

MyFunc(a string, b int, args ...interface{}) { /* ... */ }

but that approach would require runtime checking of the variadic argument, args, and would eschew compile-time safety guarantees (and subsequent optimizations). If the consumers of MyFunc() span team or organizational boundaries, it is effectively impossible to force callers to update their interface to match the new function signature.

This recommendation is to introduce a static function signature with an "append-only input structure":

type MyFuncInputs struct {}
func MyFunc(ctxt context, dnode uint64, MyFuncInputs{}) {
  //
}

The function signature for MyFunc() can now be effectively frozen and stable from the perspective of the consumers of the library. MyFunc()'s new signature acknowledges that both ctxt and dnode are required arguments yet still allows the library author to extend the API in the future by appending members to the MyFuncInputs struct.

If an API is performance sensitive, this approach MAY NOT be appropriate. Use of this technique is an exercise in forethought where the cost of maintenance burdened by the author is weighed against the runtime performance impact of passing an optional struct input to a function. It is difficult to imagine the case where the execution cost of thousands of requests per second would outweigh the engineering burden of maintaining a frequently updated or loosely coupled interface that spans repositories.

This technique must adhere to similar rules as those suggested when updating a protobuf message type, notably:

  • *Input struct member names are permanent and MUST NOT change or have their meaning altered in a way that changes their contract.

  • Obsolete *Input struct member names MUST:

    1. be automatically mapped to an updated struct member(s)

    2. ignored (a discouraged practice)

    3. removed thereby explicitly breaking any existing code

    4. never be reused for the life of the interface (and therefore the *Input struct.

      A phased approach to evolving a *Input struct is an acceptable strategy.

Again, this is a recommended technique for providing stable interfaces where the runtime and diminished readability has been weighed against the cost of maintenance (most notably engineering time or runtime breakage).

10. Static Analysis

Recommendations

  1. Use and integration of "baseline static analysis checks" SHOULD be integrated into the CI.

  2. An inventory of "optional static analysis checks" is RECOMMENDED but not necessary for a second tier of checks to be added to list of suggested static analysis checks (e.g. "noisy, but useful" or "mostly accurate, but still throws false-positives").

Rationale

reviewdog stands out as a pragmatic way to programmatically raise the bar of quality within a given Go project by automatically executing and providing inline annotations in PRs with the results of baseline checks. If a particular type of error occurs more than a few times, write a static analysis check and incorporate it into reviewdog.

For offline development, use of gometalinter(1) is RECOMMENDED:

$ go get -u github.com/alecthomas/gometalinter
$ gometalinter --install

Regardless of the tool, incorporating a baseline of static analysis of commonly identified issues frees up reviewers to focus on the content of change versus the mechanics of the change. Time invested in static analysis checks usually pays dividends with respect to preventing bugs (e.g. scopelint, go tool vet --shadow, errcheck, safesql, staticcheck), reducing sub-optimal code (e.g. ineffassign, unparam), or reducing engineering time wasted pointing out nits that could be identified consistently by bots (e.g. go vet, lll (long line linter), misspell).

Several recommended static analysis checks include (most come from gometalinter(1), alphabetically sorted):

Several optional linters include (alphabetically sorted):

11. Explicit Types

Recommendation

  1. Explicit types SHOULD be used within a project.

  2. Libraries or public APIs MAY export types where it helps readability.

  3. Where the meaning or intent of a fundamental type would benefit from explicit type checking by the compiler, explicit types SHOULD be used.

  4. Type Conversions SHOULD be deferred as long as reasonable.

  5. Where explicitly typed variables are employed, the lifecycle of identifiers referencing underlying types SHOULD be reduced to the smallest reasonable scope possible.

  6. Use of gorename(1) to maintain type names is RECOMMENDED. The RECOMMENDED use of gorename(1) extends to all package, function, and method members (i.e. const, func, var, and type).

Rationale

Go is an explicitly typed language. The compiler does not perform any implicit type conversions of named types. Exported functions, interfaces, and types SHOULD make use of explicit types in order to enable the compiler to detect and enforce a package’s specified type system. It is NOT RECOMMENDED to deprive the compiler of the necessary type information it requires in order to prevent developers from incorrectly and abusively overloading Go’s underlying types (e.g. string vs RandomStringID, or uint64 vs inode).

As an example, a string SHOULD be thought of as an immutable slice of runes that is missing its type information (i.e. a string is a container, not a type).

Go’s fundamental or underlying types (e.g. string, int*, []byte) are containers that crudely answer the question "how is a variable going to be stored efficiently." Use of underlying types do not answer the question "what bits are in a given container." Go does not permit any implicit type conversions of named types.

Go’s explicit type system prevents variables backed by the same underlying type from fraternizing. Use of fundamental types at formal interface boundaries is discouraged because use of variable names to indicate the intended use of a variable is only enforced by the reader, not by the compiler. If variable names are sufficient to guard against variable misues, you MAY rely on variable names to convey type information.

Where type intent information SHOULD be enforced by the compiler, use of explicit types is RECOMMENDED. The Go type system is a compile-time cost, not a runtime cost. Use types.

Examples:

1
2
3
4
5
6
7
8
type ID string                (1)
type ID uint64                (2)
type CookieID string
type UUID []byte
type Index uint
type Key string
type Value string
type Lookup map[Key]Value
1 ID may have started out as a string
2 ID could be easily changed to a uint64 and the consequences easily observed. NOTE: this wouldn’t compile due to the ID identifier being reused in the same package.

12. Return Parameters

Recommendation

  1. When deciding if a function or method should return an argument by value or pointer, returning a value SHOULD be your default position except in the following situations, in which case it is RECOMMENDED to return a pointer to a value:

    1. the API contract you want to establish with the caller is to force them to deal with errors by returning nil AND the construction of the zero-value is onerous or expensive (i.e. return "" for a string).

    2. ownership of the variable may change throughout the course of the variable’s life.

    3. the expense of copying the variable is measurable.

Rationale

Go uses pass-by-value semantics and employs variable escape analysis.

Embrace the pass-by-value nature of Go, be productive, and let the compiler do work for you.

Much of the above reading was shamelessly borrowed from a Stack Overflow article which is a good read on its own merits.

13. Interface Receivers

Recommendation

  1. When deciding if a receiver should be a value or a pointer, a pointer SHOULD be used by default except in the following situations, in which case it is RECOMMENDED to use a value:

    1. the value of the receiver is a simple underlying type (i.e. an int)

    2. invocation of the given interface method SHOULD result in a copy of the receiver.

Rationale

This is simple: use a pointer to a receiver in nearly all cases. Item 1b is very rare in practice.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
type Foo struct {
  bar string
}

// Baz assigns "bur" to f.bar.  Without the pointer, this the instance of Foo
// would have been copied and the assignment would have been not visible to
// the caller (a nice source of frustration when first learning Go).
func (f *Foo) Baz() {
  f.bar = "bur"
}

In practice, use of non-pointer receivers is limited to the following example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
type MyEnum int

func (e MyEnum) String() string {
  switch e {
  case 0:
    return "zero"
  case 1:
    return "one"
  default:
    return "something not one or zero"
  }
}

var myEnum MyEnum = 0
fmt.Println("%s", myEnum)

Where the important takeaway is that in String(), it doesn’t matter if the value is copied.

14. const

Recommendation

  1. Use of const is RECOMMENDED.

  2. Create explicitly typed consts is RECOMMENDED.

  3. consts with type information SHOULD should be exported (both the type and the const values).

  4. Periodically using static analysis checks like goconst is RECOMMENDED but OPTIONAL.

Rationale

By creating a const, you give the Go tooling an identifier which you can search for referrers of the given const. See the referrers section of the Using Guru document (this document SHOULD be required reading).

15. Bitmasks

Recommendation

  1. Bitmasks SHOULD be created using const and iota.

  2. Bitmasks SHOULD be explicitly typed.

  3. The meaning of bits in a bitmask MAY change if it is documented in the interface that the meaning of individual bits may change.

  4. The meaning of bits MUST NOT change if the bitmask is exported and the position of individual bits is part of the contract.

  5. A new type, removal of the bitmask as a type, or other form of compile-time breakage MUST be introduced in order to communicate the change in behavior.

  6. Manual manipulation of bitmasks SHOULD NOT be performed without explicitly named bits.

Rationale

Go provides a convenient trick to automatically creating bitmasks:

1
2
3
4
5
6
7
8
type MyBitmask int

const (
      FlagA MyBitmask = 1 << iota  (1)
      FlagB                        (2)
      FlagC                        (3)
      FlagD                        (4)
)
1 FlagA == 0x01
2 FlagB == 0x02
3 FlagC == 0x04
4 FlagD == 0x08

Leverage this trick.

16. Documentation

Recommendation

  1. Projects MUST use godoc(1) to document their project.

Rationale

Read the Godoc: documenting Go code blog post.[6]

17. Context

Recommendation

  1. Projects MUST the context pattern for passing state along request-scoped state information (e.g. deadlines, cancelation signals, or request-specific information).

Rationale

Read the Go Concurrency Patterns: Context blog post.

Misc Resources

18. Deadline Timers and Timeouts

Recommendation

  1. In-process timers and timeouts MUST use time.Duration.

  2. In-process timers and timeouts using the context package MUST use context.WithTimeout.

  3. Inter-process timeout enforcement MUST communicate using an absolute time reference using time.Time.

  4. Inter-process timeouts using the context package MUST use context.WithDeadline.

Rationale

Starting in Go 1.9, Go’s time.Time package uses monotonic time.

19. gRPC

Recommendation

  1. gRPC SHOULD be preferred as the RPC framework for communicating between discrete Go processes locally or on the network.

  2. JSON MAY be used as the RPC framework when necessary to interoperate with non- gRPC clients.

Rationale

Read the Go Concurrency Patterns: Context blog post.

20. pprof and gops(1)

Recommendation

  1. Long-running daemons MUST have either a gops(1) or pprof endpoint available on a secure endpoint (a loopback interface, the default, is considered a secure endpoint).

  2. The gops(1) endpoint is RECOMMENDED and can be added to code like:

    import (
	"github.com/google/gops/agent" (1)
)

	// Somewhere after configuration, start the agent listener.  Exit if the agent
	// is unable to listen.
	if err := agent.Listen(nil); err != nil { (2)
		log.Fatal(err)
	}
    1 Import the agent
    2 Listen early on after configuration

  3. The pprof endpoint SHOULD use a standardized URL anchors.

  4. The pprof endpoint MAY be made available via alternate libraries other than the canonical net/http/pprof listener. See [agent] for a RECOMMENDED way of enabling the pprof interface. If [agent] CAN NOT be used, the following fragment is sub-optimal SHOULD NOT be used to create a local listener:

    import (
	"fmt"
	"net/http"         (1)
	_ "net/http/pprof" (2)
	"time"
)

// In a goroutine listener somewhere downstream of main() (3)
var pprofErr error
go func() { (4)
	// Listen on port 6060 for `localhost`.  This could be [::] or `127.0.0.1`
	pprofErr = http.ListenAndServe("localhost:6060", nil) (5)
}()

// Sleep for 1s to give the backend a chance to fail.
time.Sleep(1 * time.Second) (6)
if pprofErr != nil {
	return fmt.Errorf("pprof endpoint failed to initialize: %v", pprofErr)
}
    1 The net/http package is only needed for the standalone listener used in callout 5.
    2 Initialize the net/http/pprof package.
    3 init() SHOULD NOT be used to spawn this listener in order to allow for proper configuration to take place and to allow the program to respond to errors in the event the pprof endpoint fails to initialie. The pprof endpoint SHOULD be initialized before this service enters its duty-cycle.
    4 Start a detached thread to handle pprof requests.
    5 Listen on a local endpoint and port
    6 "One second ought to be enough time for any service to fail."

    This is a BAD example of how because: the code does not handle errors well; assumes the listener will start and fail within one second, and this introduces an arbitrary one second delay to program initialization. See the next example for a preferred way of initializng the pprof endpoint which addresses each of these concerns. The example snippet in net/http/pprof's documentation is convenient and uncomplicated but SHOULD NOT be used in production for the aforementioned reasons.

Rationale

TODO.

Misc Resources

21. External Resources

  1. The Go Programming Language

  2. Less is exponentially more

  3. Go Proverbs

    1. Don’t communicate by sharing memory, share memory by communicating.

    2. Concurrency is not parallelism.

    3. Channels orchestrate; mutexes serialize.

    4. The bigger the interface, the weaker the abstraction.

    5. Make the zero value useful.

    6. interface{} says nothing.

    7. gofmt’s style is no one’s favorite, yet gofmt is everyone’s favorite.

    8. A little copying is better than a little dependency.

    9. syscall.* must always be guarded with build tags.

    10. Cgo must always be guarded with build tags.

    11. Cgo is not Go.

    12. With the unsafe package there are no guarantees.

    13. Clear is better than clever.

    14. Reflection is never clear.

    15. Errors are values.

    16. Don’t just check errors, handle them gracefully.

    17. Design the architecture, name the components, document the details.

    18. Documentation is for users.

    19. Don’t panic.

  4. Go wiki

  5. Awesome Go: A curated list of awesome Go frameworks, libraries and software.

Appendix A: Go 101 Resources

If you are completely new to Go, "I haven’t written a line of Go in my life before today," this section is a quick primmer to get you up and running in a hurry.

Five Minute Jump Start

Suppose you know nothing and just need to get started. The following steps SHOULD get you to a functioning "Hello World" in ~5min.

  1. Install a stable copy of Go. This can be done in a number of ways, either from the official Download Page or via your preferred package management system:

    1. pkgsrc: sudo pkgin -y install go

    2. MacpPrts: sudo port install go

    3. Homebrew: sudo brew install go

  2. Open a terminal and add the following environment variables to your shell (as long as your GOPATH is set and PATH includes $GOPATH/bin at the end of this step how you get this done doesn’t matter):

    $ echo 'export GOPATH=$HOME/go' >> ~/.profile
$ echo 'export PATH=$GOPATH/bin:$PATH' >> ~/.profile
$ export GOPATH=$HOME/go
$ export PATH=$GOPATH/bin:$PATH

  3. Ensure git(1) has already been installed.

  4. Install gometalinter(1): go get -u github.com/alecthomas/gometalinter

  5. Install gometalinter(1)'s linters: gometalinter --install

  6. Install gops(1): go get -u github.com/google/gops

  7. Install expvarmon(1): go get -u github.com/divan/expvarmon

  8. Install go-torch(1): go get -u github.com/uber/go-torch

  9. Install FlameGraph: go get -d github.com/brendangregg/FlameGraph

  10. Add FlameGraph's scripts to your PATH: export PATH=$PATH:$GOPATH/github.com/brendangregg/FlameGraph

  11. Change into an existing sample program: cd $GOPATH/src/github.com/google/gops/examples/hello

  12. Edit main.go and make the following edits:

     package main

 import (
+       "fmt"
        "log"
-       "time"

        "github.com/google/gops/agent"
 )

+func getFactorial(n int) int {
+       if n < 0 {
+               return 0
+       }
+
+       if n <= 1 {
+               return 1
+       }
+
+       return n * getFactorial(n-1)
+}
+
 func main() {
        if err := agent.Listen(nil); err != nil {
                log.Fatal(err)
        }
-       time.Sleep(time.Hour)
+
+       fmt.Println("Hello World!  Please debug me.")
+       for i := 1000000; i > 0; i-- {
+               getFactorial(i)
+       }
 }

  13. Build the program: go build

  14. Run the program:

    $ ./hello
Hello World!  Please debug me.

    This program will run in the foreground for the next hour or until you interrupt the program (e.g. ctrl+c).

  15. The program is available to be inspected via gops(1):

    $ gops
44326	gops	(/Users/seanc/go/bin/gops)
44319*	hello	(/Users/seanc/go/src/github.com/google/gops/examples/hello/hello)
$ gops stats 44319
goroutines: 5
OS threads: 7
GOMAXPROCS: 8
num CPU: 8
$ gops memstats 44319
alloc: 96.28KB (98592 bytes)
total-alloc: 96.28KB (98592 bytes)
sys: 67.72MB (71012352 bytes)
lookups: 7
mallocs: 354
frees: 7
heap-alloc: 96.28KB (98592 bytes)
heap-sys: 32.69MB (34275328 bytes)
heap-idle: 32.22MB (33783808 bytes)
heap-in-use: 480.00KB (491520 bytes)
heap-released: 0 bytes
heap-objects: 347
stack-in-use: 32.31MB (33882112 bytes)
stack-sys: 32.31MB (33882112 bytes)
next-gc: when heap-alloc >= 4.27MB (4473924 bytes)
last-gc: -
gc-pause: 0s
num-gc: 0
enable-gc: true
debug-gc: false
$ gops stack 44319
goroutine 8 [running]:
runtime/pprof.writeGoroutineStacks(0x12094e0, 0xc4200c8020, 0x0, 0x0)
	/Users/seanc/go1.9/src/runtime/pprof/pprof.go:608 +0xa7
runtime/pprof.writeGoroutine(0x12094e0, 0xc4200c8020, 0x2, 0x0, 0x0)
	/Users/seanc/go1.9/src/runtime/pprof/pprof.go:597 +0x44
runtime/pprof.(*Profile).WriteTo(0x1219060, 0x12094e0, 0xc4200c8020, 0x2, 0x0, 0x0)
	/Users/seanc/go1.9/src/runtime/pprof/pprof.go:310 +0x3ab
github.com/google/gops/agent.handle(0x12094e0, 0xc4200c8020, 0xc4200ca000, 0x1, 0x1, 0x0, 0x0)
	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:171 +0x1a3
github.com/google/gops/agent.listen()
	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:119 +0x2c5
created by github.com/google/gops/agent.Listen
	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:100 +0x457

goroutine 1 [runnable]:
main.getFactorial(0xefb92, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:14 +0x82
main.getFactorial(0xefb93, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb94, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb95, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb96, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb97, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb98, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb99, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb9a, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb9b, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb9c, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb9d, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb9e, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefb9f, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba0, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba1, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba2, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba3, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba4, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba5, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba6, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba7, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba8, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefba9, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbaa, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbab, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbac, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbad, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbae, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbaf, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb0, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb1, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb2, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb3, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb4, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb5, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb6, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb7, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb8, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbb9, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbba, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbbb, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbbc, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbbd, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbbe, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbbf, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc0, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc1, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc2, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc3, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc4, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc5, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc6, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc7, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc8, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbc9, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbca, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbcb, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbcc, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbcd, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbce, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbcf, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd0, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd1, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd2, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd3, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd4, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd5, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd6, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd7, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd8, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbd9, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbda, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbdb, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbdc, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbdd, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbde, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbdf, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe0, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe1, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe2, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe3, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe4, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe5, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe6, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe7, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe8, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbe9, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbea, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbeb, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbec, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbed, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbee, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbef, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbf0, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbf1, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbf2, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbf3, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbf4, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
main.getFactorial(0xefbf5, 0x0)
	/Users/seanc/go/src/github.com/google/gops/examples/hello/main.go:23 +0x4d
...additional frames elided...

goroutine 5 [syscall]:
os/signal.signal_recv(0x0)
	/Users/seanc/go1.9/src/runtime/sigqueue.go:131 +0xa7
os/signal.loop()
	/Users/seanc/go1.9/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
	/Users/seanc/go1.9/src/os/signal/signal_unix.go:28 +0x41

goroutine 6 [select, locked to thread]:
runtime.gopark(0x1166cb8, 0x0, 0x115e4e1, 0x6, 0x18, 0x1)
	/Users/seanc/go1.9/src/runtime/proc.go:277 +0x12c
runtime.selectgo(0xc420040f50, 0xc42007c180)
	/Users/seanc/go1.9/src/runtime/select.go:395 +0x1138
runtime.ensureSigM.func1()
	/Users/seanc/go1.9/src/runtime/signal_unix.go:511 +0x1fe
runtime.goexit()
	/Users/seanc/go1.9/src/runtime/asm_amd64.s:2337 +0x1

goroutine 7 [chan receive]:
github.com/google/gops/agent.gracefulShutdown.func1(0xc420066060)
	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:132 +0x34
created by github.com/google/gops/agent.gracefulShutdown
	/Users/seanc/go/src/github.com/google/gops/agent/agent.go:130 +0xb5
$ gops trace 44319
Tracing now, will take 5 secs...
Trace dump saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/trace902953220
2017/08/03 14:17:23 Parsing trace...
2017/08/03 14:17:23 Serializing trace...
2017/08/03 14:17:23 Splitting trace...
2017/08/03 14:17:23 Opening browser
# gops will block in the foreground - push ctrl+c when done
# This program is uneventful so there's nothing to see
$ gops pprof-heap 44319
Profiling dump saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile786664192
Binary file saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/binary663659615
File: binary663659615
Type: inuse_space
Time: Aug 3, 2017 at 2:19pm (PDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) help
  Commands:
    callgrind        Outputs a graph in callgrind format
    comments         Output all profile comments
    disasm           Output assembly listings annotated with samples
    dot              Outputs a graph in DOT format
    eog              Visualize graph through eog
    evince           Visualize graph through evince
    gif              Outputs a graph image in GIF format
    gv               Visualize graph through gv
    kcachegrind      Visualize report in KCachegrind
    list             Output annotated source for functions matching regexp
    pdf              Outputs a graph in PDF format
    peek             Output callers/callees of functions matching regexp
    png              Outputs a graph image in PNG format
    proto            Outputs the profile in compressed protobuf format
    ps               Outputs a graph in PS format
    raw              Outputs a text representation of the raw profile
    svg              Outputs a graph in SVG format
    tags             Outputs all tags in the profile
    text             Outputs top entries in text form
    top              Outputs top entries in text form
    topproto         Outputs top entries in compressed protobuf format
    traces           Outputs all profile samples in text form
    tree             Outputs a text rendering of call graph
    web              Visualize graph through web browser
    weblist          Display annotated source in a web browser
    o/options        List options and their current values
    quit/exit/^D     Exit pprof

  Options:
    call_tree        Create a context-sensitive call tree
    compact_labels   Show minimal headers
    divide_by        Ratio to divide all samples before visualization
    drop_negative    Ignore negative differences
    edgefraction     Hide edges below <f>*total
    focus            Restricts to samples going through a node matching regexp
    hide             Skips nodes matching regexp
    ignore           Skips paths going through any nodes matching regexp
    mean             Average sample value over first value (count)
    nodecount        Max number of nodes to show
    nodefraction     Hide nodes below <f>*total
    output           Output filename for file-based outputs
    positive_percentages Ignore negative samples when computing percentages
    prune_from       Drops any functions below the matched frame.
    relative_percentages Show percentages relative to focused subgraph
    sample_index     Sample value to report (0-based index or name)
    show             Only show nodes matching regexp
    source_path      Search path for source files
    tagfocus         Restrict to samples with tags in range or matched by regexp
    taghide          Skip tags matching this regexp
    tagignore        Discard samples with tags in range or matched by regexp
    tagshow          Only consider tags matching this regexp
    trim             Honor nodefraction/edgefraction/nodecount defaults
    unit             Measurement units to display

  Option groups (only set one per group):
    cumulative
      cum              Sort entries based on cumulative weight
      flat             Sort entries based on own weight granularity
      addresses        Aggregate at the function level.
      addressnoinlines Aggregate at the function level, including functions' addresses in the output.
      files            Aggregate at the file level.
      functionnameonly Aggregate at the function level.
      functions        Aggregate at the function level.
      lines            Aggregate at the source code line level.
      noinlines        Aggregate at the function level.

  type "help <cmd|option>" for more information
(pprof) exit
$ go-torch -b /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile786664192
INFO[18:44:38] Run pprof command: go tool pprof -raw /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile786664192
INFO[18:44:38] Writing svg to torch.svg
$ open -a /Applications/FirefoxNightly.app torch.svg
$ gops pprof-cpu 44319
Profiling CPU now, will take 30 secs...
Profiling dump saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile127087150
Binary file saved to: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/binary771063221
File: binary771063221
Type: cpu
Time: Aug 3, 2017 at 6:46pm (PDT)
Duration: 30.16s, Total samples = 26.21s (86.90%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) exit
$ go-torch -b /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile127087150
INFO[18:47:35] Run pprof command: go tool pprof -raw /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/profile127087150
INFO[18:47:36] Writing svg to torch.svg
$ open -a /Applications/FirefoxNightly.app torch.svg
$ cat $HOME/.config/gops/${PID_OF_HELLO_PROCESS} ; echo
62023
Scheduler Latency
Figure 1. Output from gops trace and the scheduler latency profile

It’s obviously not required to step through and enumerate all of the capabilities of gops(1) and the pprof endpoints when writing a "Hello World" app but it shows off some of the built-in capabilities. Additional information is covered in the section on pprof.

Books

The following is a list of books that people have found helpful for getting started. One of the problems with learning by dead tree is the disparity between inches of shelf-space occupied to a given subject versus the mental real-estate a given subject occupies. Because a language feature is address - sometimes at length - doesn’t mean it is a primary concern or something you need to become a subject-matter expert on.

The following are "approved" resources ("approved" in this context only means "this wasn’t awful and will point you in the right direction" and doesn’t constitute as a explicit endorsement #JoyentEmployee). There may be other good resources that exist, but they haven’t been reviewed (pull requests welcome).

Extra Resources

make(1)

make(1) comes in many shapes and forms. GNU make(1) is the predominant make(1) implementation. GNU make(1), however, has its own dialect and its dialect is not universally supported. If you use GNU make(1)-specific extensions, you MUST spell the Makefile like GNUmakefile. The same rule applies to BSD make(1) and BSDmakefile. It is unfortunate that there is poor interopoerability between the various make(1) implementations and it’s equally unfortunate that the prevailing attitudes is to be conformist (versus taking a zero-cost principled stand for correctness).

$ cat GNUmakefile
test:
    echo $(shell echo bar)
$ bsdmake -f GNUmakefile
echo

$ gnumake -f GNUmakefile
echo bar
bar

In the end, this fragmentation has led many to adopt make(1) as the universal human UX and system interoperability layer to front tools like bezel(1) or goreleaser(1).

make(1) is like Latin. Many western languages decend from make(1), however no one speaks Latin anymore. Instead people speak Spanish, Italian, French, English, etc. If you don’t speak French and call it Latin, I won’t speak Spanish and call it Latin. Incorrectness has consequences. In software it most oftens in deferred cost.

Glossary

CI

Continuous Integration

Bibliography

  1. A. A. A. Donovan and B. W. Kernighan, The Go Programming Language (Addison-Wesley Professional Computing Series). Addison-Wesley Professional, 2015 [Online]. Available: smile.amazon.com/Programming-Language-Addison-Wesley-Professional-Computing-ebook/dp/B0184N7WWS

  2. C. Doxsey, An Introduction to Programming in Go. CreateSpace Independent Publishing Platform, 2012 [Online]. Available: www.golang-book.com/public/pdf/gobook.0.pdf

  3. M. Summerfield, Programming in Go: Creating Applications for the 21st Century (Developer’s Library). Addison-Wesley Professional, 2012 [Online]. Available: smile.amazon.com/Programming-Go-Creating-Applications-Developers-ebook/dp/B007Y6KDTG

1. While not an official recommendation, one could easily be forgiven for symlinking $GOPATH/pkg to someplace in /tmp/ in order to take advantage of the automatic pruning of old .a files that happens periodically and after every reboot on many *NIX platforms (including macOS).
2. Tools similar to bazel could influence this recommendation in the future however there are no plans to augment the workflow presented by the go(1) tool.
3. Monorepos can be justified by either their productivity gains, by Parkinson’s law, or by blurring blurring the natural organizational lines stemming from Conway’s law by embracing the egalitarian nature of software.
4. clang-format(1) SHOULD be considered for C and C++ codebases alike. clang-format - Automatic formatting for CXX and code::dive 2016 conference – Chandler Carruth – Making CXX easier, faster and safer (part 1).
5. See Real Programmers Hottest Editors
6. It would be nice, however, if godoc(1) supported a richer markup like asciidoc.