HDE BLOG

コードもサーバも、雲の上

Designing APIs with Goa

Like most of the web services in the current software development trends, the service that our team is currently building is also based on the idea of microservice architecture using the Go programming language. Right now we have around 10, in theory, independently deployable services and these services are often communicating with each other using RESTful APIs with over 100 of HTTP endpoints. Also, we have a feeling that these numbers will keep increasing over time.

To add, manage, or maintain the APIs, developers have to deal with the handler functions (e.g. routers, validations, etc.); add or update the client codes, API documents what are being used by the consumers of the APIs in the format of JSON Hyper Schema. These codes are actual very similar but yet have differences among each APIs which make them very hard to refactor. As anyone can imagine, doing the tasks above is not fun at all and very error-prone.

Last year at our 20th Monthly Technical Session (MTS), @lestrrat talked about his approach to the similar problem using code generation. He also gave a similar talk in Go Conference 2016 Spring titled Auto-Generating builderscon server, validator, and client with JSON (Hyper)? Schema where we can find the tools that he built to help him builds builderscon. Our team started to realize the problems in our project too but did not invest much time on looking for the right solution to improvement our own developing experience as everyone is pretty occupied in solving the tasks and issues that have higher priority.

goa

I first found out about goa from the 7th Go Time podcasts where its creator, Raphaël Simon appeared on the show as special guest. “goa is a holistic approach for building microservices in Go.” It lets us design our APIs with its own design language and then generate almost everything, the boilerplate and glue code, documentations (JSON schema / swagger), command line application, JavaScript library, client code to interact with the API, etc.

Simple demo

For a simple demo, lets us design the APIs to register and list the users in a service. We start from a clean Go project github.com/example/simple-goa-demo. By convention, the design lives in the design directory and for the design we are using the goa’s DSL, not Go itself. Therefore, please pretend that you don’t see the dot imports.

In design.go, we define the User type which will be used as the payload of the request. UserMedia media type is used to build the response body. We can reuse the payload attributes in the media type by referencing the payload. The media type should have at least one default View("default", ...). The two API endpoints are described with Action inside the Users resources. The documentations for the DSL can be found at here.

Now that we completed the design, we can generate the other stuffs in the directory one level above design/:

$ goagen bootstrap -d github.com/example/simple-goa-demo/design

and the following will be generated:

├── app/
├── client/
├── main.go
├── swagger/
├── tool/
└── users.go

which includes the client codes, swagger files, CLI, etc.

If we explore the generated contents, we can find that the generated Go codes are beautiful and idiomatic. By default, main.go and users.go are boilerplates and will only be generated once. To get the API server running, we need to implement the logic in users.go. Notice that the filename is given by the resource name.

For simplicity, our database is defined as the following:

type user struct {
    name  string
    email string
}

// Simple DB
var users = make(map[string]user)

The contents of the payload can be retrieved from the context (*app.AddUsersContext) generated by goa and the response body is constructed using the media type (app.MyUserCollection) defined in the app package.

Now that we have the implementation, we can build and run the server.

$ go build -o server
$ ./server

To test it with the correct payload,

$ cat <<EOF | curl -i -d@- -H 'Content-Type: application/json' -X POST localhost:8080/users/add
{
  "name": "test user",
  "email": "test_user@example.com"
}
EOF
HTTP/1.1 201 Created
Date: Fri, 14 Apr 2017 06:07:59 GMT
Content-Length: 0
Content-Type: text/plain; charset=utf-8

If we do not provide the email address of the user, we will get “400 Bad Request” with a proper error message.

$ cat <<EOF | curl -i -d@- -H 'Content-Type: application/json' -X POST localhost:8080/users/add
{
  "name": "test user"
}
EOF
HTTP/1.1 400 Bad Request
Content-Type: application/vnd.goa.error
Date: Fri, 14 Apr 2017 06:11:57 GMT
Content-Length: 187

{"id":"RVzMUBbS","code":"bad_request","status":400,"detail":"[4/TWYx++] 400 invalid_request: attribute \"email\" of response is missing and required, attribute: email, parent: response"}

And we can also get a list users from the service:

$ curl -i -X GET localhost:8080/users/list
HTTP/1.1 200 OK
Content-Type: vnd.my.user; type=collection
Date: Fri, 14 Apr 2017 06:12:57 GMT
Content-Length: 55

[{"email":"test_user@example.com","name":"test user"}]

This basically sums up my very simple demo of goa. As we can see, goa allows us to only write the codes that matter, and it generates the boring stuff as idiomatic go codes for us. The documentation is also very complete (I did not need to Google nor refer to Stackoverflow when building this demo) and the community seems very friendly. Of course this is only the first step. We need more investigation on the framework itself to actually integrate it into our code base; especially in the area of security using JWT and the implementation of role based access control. With its extensibility, I have high hope that we can achieve a better API design in our services with goa.

Further readings