Testing HTTP service in Go

Photo by Markus Spiske on Unsplash

In this article, I would like to share the techniques and tools I use to test services. I will not surprise you if I say that most of the tests I write are unit tests. Second and biggest (by volume) group of my tests - integration tests. Nothing new here, just classical Test Pyramid.

Speaking about tests it is impossible not to mention stubbing or mocking. Therefore in this article, I would like to highlight some of the test doubles generation tools available in the golang ecosystem.

I’ve created a basic service go-calc in Go, including different types of tests. Despite the golang specific features used in the implementation, testing principles remain the same for any language.

Example service

The sample application shows the features of a typical service. It provides a REST interface, implements business logic, and orchestrates a third-party service call.

go-calc service has two endpoints:

• POST /do - handles requests to calculate arithmetic operations (addition, subtraction, etc)
• POST /remote - makes a call to the third-party service

Figure 1. The high-level view of “go-calc” service

The next snippet shows the routes builder:

Snippet 1. Service routes map

The application has a dependency on external service. Such dependencies are not only HTTP services. They could be databases, file system, etc. To avoid coupling with the dependencies, I inject external services clients to the methods that work with these services. Therefore, I provided an HTTP client to remoteHandler and defined HTTPClient interface for the testing reasons (more about it in Testing HTTP calls).

Now I can initiate HTTP service and define necessary endpoints:

Snippet 2. HTTP server bootstrap

Every route handler should satisfy http.Handler interface. There are multiple ways to achieve it: declare ServerHTTP method on your handler type, or utilize typehttp.HandlerFunc that allows usage of ordinary functions as HTTP handlers.

For this example, I defined all route handlers as func(http.ResponseWriter, *http.Request) or as a higher-order function returning http.HandlerFunc.

Snippet 3. Route handlers

Testing route handlers

To test route handlers I need to invoke them with the required parameters: http.ResponseWriter and *http.Request. It is possible to write custom code to initialize these parameters. But in most cases, it is not necessary. Go comes with net/http/httptest - the package that provides tools to test HTTP clients and servers.

I use httptest.NewRequest to create an incoming request and httptest.NewRecorder to initialize ResponseRecorder. ResponseRecorder implements http.ResponseWriter interface and records response changes for later inspection. The following snippet shows tests for notFoundHandler and doHandler:

Snippet 4. Local handlers tests

Because of notFoundHandler and doHandler do not call external services, the tests are quite easy: initiate request and response writer, invoke handler, assert response.

Testing HTTP calls

Testing remoteHandler is a bit more complicated because it has an external dependency - HTTP call to a third-party service. There are a few ways to test such code: stub external service or mock external service client.

Testing using client mock

Having created HTTP client interface - HTTPClient, allowed me to create client mock and use it in tests. To generate client mock code I used mockery.

Snippet 5. External call test using client mock

At lines 6 and 7 I initialized a client mock and defined its behaviour. I used a basic parameter assert mock.AnythingOfType("*http.Request"). Such mock will always return a predefined response on any request that is an instance of *http.Request.

For stricter validation, it’s possible to assert parameters to have the exact value. In such a case, the test will fail when unexpected parameters sent to the mocked method.

Testing using stub service

The other way to test remoteHandler is to use external service stubs. net/http/httptest provides NewServer - a method that allows creating services with canned responses.

Snippet 6. Using stub server to test remote call

This approach is useful when mocking a client is not possible, or when testing the client implementation itself.

Testing service using Newman

Testing with mocks and stubs is enough to cover most of the use cases. But sometimes bugs happen in integration itself. For the full coverage its necessary to run integration tests, where the application launched, and tests run in a black-box manner (requests sent to the application and responses compared with the expected outcome). There are tools to help you with this: curl , postman , insomnia are the names of few. My preference is Postman. Not only it provides a comprehensive application, that ticks all the boxes that you might require, it also has a CLI version - newman.

I use Postman application to create a collection of requests and quickly run them. Every request could be accompanied by the tests. Then I export collection and commit it to the source code. After that, I can run the same requests in CI/CD environment.

The following figures show how the request collection and tests look like in Postman and Newman.

Figure 1. Sending a request in Postman

Figure 2. Test response in Postman

Figure 3. Running request collection in Newman

Testing on the local machine

On the local machine, I run all services in docker containers using docker-compose.

Snippet 7. Dockerfile

Snippet 8. Docker-compose

There are some of the reasons why I use docker and docker-compose:

• I don’t want to pollute local machine with the “zoo” of services
• it’s easy to add more services (DB, external services mocks, etc) in future as the development growth
• it’s easy to maintain services dependencies
• flexible network and ports configuration
• containers is a lightweight way to solve “it works on my machine” issue

Having a containerized application, I can test it as:

$ docker-compose up # launch application in docker container
$ newman run postman_collection.json # run request collection

Conclusion

Golang has an outstanding testing ecosystem. For most cases, tools provided by the language itself - testing, net/http/httptest will be enough. Interfaces, when used properly, allow creating robust and testable code.

I primarily use client mocks where possible. Otherwise, I create external services stubs. Finally, for the full integration tests, I use containerized services and Postman/Newman to send requests to the service.

Thanks for reading. Please visit go-calc repository for more information and code examples.

References

• http and httptest - Go HTTP package and test utilities
• testify - a package with common assertions and mocks
• mockery - mock generator for Go interfaces
• Postman and Newman - API client and test tools
• Test Pyramid, test doubles - testing theory and terminology
• go-calc - sample HTTP service in Go