Liam Clarke brand
Back to the top
What is a narrow integration test?
Why use narrow integration tests?
How to implement a narrow integration test?
Creating the Storage layer
Creating some stub data
Creating the testing main function
Creating the narrow integration testbed
What next

Narrow integration testing with Go

Test
11 December 2021 6 minute read
GoGolangIntegration TestTesting

Integration tests determine if individual units of code work together as expected. In this article, we will take a look at narrow integration testing and how to implement them in Go.

What is a narrow integration test?

A narrow integration test is a test that is composed of 2 parts:

  1. An I/O stream the service needs to communicate with; such as a database, external service, message queue, etc
  2. A method that communicates with the stream; such as a repository, client, consumer, etc.

What makes an integration test narrow is the scope of what is being tested. We only test the communication between the I/O stream and the communicator to/from that stream.

In Martin Fowler's article, a narrow integration test is defined with the following criteria:

  • exercise only that portion of the code in my service that talks to a separate service
  • uses test doubles of those services, either in process or remote
  • thus consist of many narrowly scoped tests, often no larger in scope than a unit test (and usually run with the same test framework that's used for unit tests)

source: Martin Fowler, integration test

Why use narrow integration tests?

There are a number of benefits to using narrow integration tests, to name a few notable ones;

  • The speed at which tests are complete is significantly increased as they are much smaller
  • The scope of the tests are narrower for greater control over testing edge cases
  • They are decoupled from other parts of the service making them more reliable

Later on, we will discuss the drawbacks of narrow integration tests and how we can action against them to make these tests more reliable.

How to implement a narrow integration test?

For this implementation, we will use a PostgreSQL database as the integration the service needs to communicate with and a storage layer that communicates and runs the query.

Creating the Storage layer

We provide a database when initialising our Storage implementation, this will allow us to pass in the local database during tests.

// Storage is the Postgres storage implementation.
type Storage struct {
	DB *sql.DB
}

// New initialises a new Storage instance.
func New(db *sql.DB) Storage {
	return Storage{
		DB: db,
	}
}

Next, we create a model of what we will be inserting, in this example, it will be a basic User

// User is the storage representation of a User.
type User struct {
	ID    uuid.UUID
	Name  string
	Email string
}

We then create an Insert method that inserts a User into our table. Squirrel builds the query with the arguments and then the query is run with the standard library's SQL interface.

// Insert inserts a new User into the users table.
func (s Storage) Insert(ctx context.Context, user User) (uuid.UUID, error) {
	query, args, err := squirrel.
		Insert("users").
		Columns("id", "name", "email").
		Values(user.ID, user.Name, user.Email).
		PlaceholderFormat(squirrel.Dollar).
		ToSql()
	if err != nil {
		return uuid.Nil, fmt.Errorf("%w: %v", ErrUnableToBuildQuery, err)
	}

	if _, err = s.DB.ExecContext(ctx, query, args...); err != nil {
		return uuid.Nil, fmt.Errorf("%w: %v", ErrUnableToExecuteQuery, err)
	}

	return user.ID, nil
}

Creating some stub data

To test our storage implementation against different use cases, we add some stubbed data. In a production environment, there will more than likely be a more robust migration executable that we will get our tables from but for the sake of this example, it will be part of the stubbed data.

CREATE TABLE IF NOT EXISTS users (
    id uuid PRIMARY KEY,
    name varchar NOT NULL,
    email varchar NOT NULL UNIQUE
);

-- stubbed users
INSERT INTO users
VALUES
       ('a6acab82-2b2e-484c-8e2b-f3f7736a26ed', 'foo', 'foo@bar.com'),
       ('20105afa-9a44-4f10-b50d-69884fbdd32c', 'bar', 'bar@bar.com'),
       ('1908007f-d7bc-4c39-845d-f11f436b579c', 'fiz', 'fiz@bar.com'),
       ('6f6b2530-bcf5-4208-b94a-3430533ec46e', 'buz', 'buz@bar.com');

Creating the testing main function

To spin up and tear down a docker container in code, ory/dockertest is used and this comes with some good examples for different database drivers, the one we are interested in, is the Postgres driver that most of the below implementation is based on.

// db is at the global scope to pass into our test functions
var db *sql.DB

func TestMain(m *testing.M) {
	pool, err := dockertest.NewPool("")
	if err != nil {
		log.Fatalf("Could not connect to docker: %s", err)
	}

	path, err := os.Getwd()
	if err != nil {
		log.Fatal(err)
	}

	options := &dockertest.RunOptions{
		Repository: "postgres",
		Tag:        "12.3",
		Env: []string{
			"POSTGRES_USER=user",
			"POSTGRES_PASSWORD=secret",
			"listen_addresses = '*'",
		},
		ExposedPorts: []string{"5432"},
		PortBindings: map[docker.Port][]docker.PortBinding{
			"5432": {
				{HostIP: "0.0.0.0", HostPort: "5432"},
			},
		},
		Mounts: []string{fmt.Sprintf("%s/stub:/docker-entrypoint-initdb.d", path)},
	}

	resource, err := pool.RunWithOptions(options, func(config *docker.HostConfig) {
		config.AutoRemove = true
		config.RestartPolicy = docker.RestartPolicy{Name: "no"}
	})
	if err != nil {
		log.Fatalf("Could not start resource: %s", err)
	}

	err = resource.Expire(30)
	if err != nil {
		log.Fatalf("Could not expire resource: %s", err)
	}

	if err := pool.Retry(func() error {
		var err error
		db, err = sql.Open(
			"postgres",
			"postgres://user:secret@localhost:5432?sslmode=disable",
		)
		if err != nil {
			return err
		}
		return db.Ping()
	}); err != nil {
		log.Fatalf("Could not connect to database: %s", err)
	}

	code := m.Run()

	if err := pool.Purge(resource); err != nil {
		log.Fatalf("Could not purge resource: %s", err)
	}

	os.Exit(code)
}

Creating the narrow integration testbed

Now the testing main handles the spin up and tear down, we can add the testbed for our Insert method and test any use cases we have.

func TestStorage_Insert(t *testing.T) {
	tests := []struct {
		name    string
		db      *sql.DB
		ctx     context.Context
		user    database.User
		want    uuid.UUID
		wantErr error
	}{
		{
			name: "expect success when inserting a new User",
			db:   db,
			ctx:  context.Background(),
			user: database.User{
				ID:    uuid.Must(uuid.Parse("5db0da2f-170f-4912-b43c-e2e2da009bdd")),
				Name:  "foo",
				Email: "new@bar.com",
			},
			want:    uuid.Must(uuid.Parse("5db0da2f-170f-4912-b43c-e2e2da009bdd")),
			wantErr: nil,
		},
		{
			name: "expect fail when inserting an email that already exists",
			db:   db,
			ctx:  context.Background(),
			user: database.User{
				ID:    uuid.New(),
				Name:  "foo",
				Email: "foo@bar.com",
			},
			want:    uuid.Nil,
			wantErr: database.ErrUnableToExecuteQuery,
		},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			s := database.New(tt.db)

			got, gotErr := s.Insert(tt.ctx, tt.user)

			if !cmp.Equal(got, tt.want) {
				t.Error(cmp.Diff(got, tt.want))
			}

			if !cmp.Equal(gotErr, tt.wantErr, cmpopts.EquateErrors()) {
				t.Error(cmp.Diff(gotErr, tt.wantErr, cmpopts.EquateErrors()))
			}
		})
	}
}

Finally, we can now run our narrow integration test completely decoupled from any other part of our service.

What next

Narrow integration tests provide a number of benefits but they do come with their shortcomings, most notably is that narrow integration tests can become out of sync with the production stream they are testing. This is inevitable when testing against test doubles.

What if an external service updated their HTTP API or a consumed message body changed, this would break our production application without any noticeable impact on our CI or local environment.

Where these concerns arise, we can solve them by combining narrow integration tests with Contract tests to ensure the integrity of our test doubles.

In the next article to this series, we will take a look at how we can use a Contract test to improve the integrity and reliability of a narrow integration test for an HTTP client communicating with an external service.

You can find a complete example of each testing strategy in the go testing examples repository