Like most of the web services in the current software development trends, the service that our team is currently building is also based on the idea of microservice architecture using the Go programming language. Right now we have around 10, in theory, independently deployable services and these services are often communicating with each other using RESTful APIs with over 100 of HTTP endpoints. Also, we have a feeling that these numbers will keep increasing over time.
To add, manage, or maintain the APIs, developers have to deal with the handler functions (e.g. routers, validations, etc.); add or update the client codes, API documents what are being used by the consumers of the APIs in the format of JSON Hyper Schema. These codes are actual very similar but yet have differences among each APIs which make them very hard to refactor. As anyone can imagine, doing the tasks above is not fun at all and very error-prone.
Last year at our 20th Monthly Technical Session (MTS), @lestrrat talked about his approach to the similar problem using code generation. He also gave a similar talk in Go Conference 2016 Spring titled Auto-Generating builderscon server, validator, and client with JSON (Hyper)? Schema where we can find the tools that he built to help him builds builderscon. Our team started to realize the problems in our project too but did not invest much time on looking for the right solution to improvement our own developing experience as everyone is pretty occupied in solving the tasks and issues that have higher priority.
goa
I first found out about goa from the 7th Go Time podcasts where its creator, Raphaël Simon appeared on the show as special guest. “goa is a holistic approach for building microservices in Go.” It lets us design our APIs with its own design language and then generate almost everything, the boilerplate and glue code, documentations (JSON schema / swagger), command line application, JavaScript library, client code to interact with the API, etc.
Simple demo
For a simple demo, lets us design the APIs to register and list the users in a service.
We start from a clean Go project github.com/example/simple-goa-demo.
By convention, the design lives in the design directory and for the design we
are using the goa’s DSL, not Go itself.
Therefore, please pretend that you don’t see the dot imports.
In design.go, we define the User type which will be used as the payload
of the request.
UserMedia media type is used to build the response body. We can reuse the payload attributes in the media type by referencing the payload.
The media type should have at least
one default View("default", ...).
The two API endpoints are described with Action inside the Users resources.
The documentations for the DSL can be found at here.
Now that we completed the design, we can generate the other stuffs
in the directory one level above design/:
$ goagen bootstrap -d github.com/example/simple-goa-demo/design
and the following will be generated:
├── app/ ├── client/ ├── main.go ├── swagger/ ├── tool/ └── users.go
which includes the client codes, swagger files, CLI, etc.
If we explore the generated contents, we can find that the generated Go codes are
beautiful and idiomatic.
By default, main.go and users.go are boilerplates and will only be generated once.
To get the API server running, we need to implement the logic in users.go.
Notice that the filename is given by the resource name.
For simplicity, our database is defined as the following:
type user struct { name string email string } // Simple DB var users = make(map[string]user)
The contents of the payload can be retrieved from the context (*app.AddUsersContext) generated by goa
and the response body is constructed using the media type (app.MyUserCollection) defined in the app package.
Now that we have the implementation, we can build and run the server.
$ go build -o server
$ ./server
To test it with the correct payload,
$ cat <<EOF | curl -i -d@- -H 'Content-Type: application/json' -X POST localhost:8080/users/add { "name": "test user", "email": "test_user@example.com" } EOF HTTP/1.1 201 Created Date: Fri, 14 Apr 2017 06:07:59 GMT Content-Length: 0 Content-Type: text/plain; charset=utf-8
If we do not provide the email address of the user, we will get “400 Bad Request” with a proper error message.
$ cat <<EOF | curl -i -d@- -H 'Content-Type: application/json' -X POST localhost:8080/users/add { "name": "test user" } EOF HTTP/1.1 400 Bad Request Content-Type: application/vnd.goa.error Date: Fri, 14 Apr 2017 06:11:57 GMT Content-Length: 187 {"id":"RVzMUBbS","code":"bad_request","status":400,"detail":"[4/TWYx++] 400 invalid_request: attribute \"email\" of response is missing and required, attribute: email, parent: response"}
And we can also get a list users from the service:
$ curl -i -X GET localhost:8080/users/list HTTP/1.1 200 OK Content-Type: vnd.my.user; type=collection Date: Fri, 14 Apr 2017 06:12:57 GMT Content-Length: 55 [{"email":"test_user@example.com","name":"test user"}]
This basically sums up my very simple demo of goa. As we can see, goa allows us to only write the codes that matter, and it generates the boring stuff as idiomatic go codes for us. The documentation is also very complete (I did not need to Google nor refer to Stackoverflow when building this demo) and the community seems very friendly. Of course this is only the first step. We need more investigation on the framework itself to actually integrate it into our code base; especially in the area of security using JWT and the implementation of role based access control. With its extensibility, I have high hope that we can achieve a better API design in our services with goa.
