Hi, this is Shi Han from HDE Inc. Before joining HDE Inc. as a software engineer, I was involved in the field of computational physics mainly focusing on computational fluid dynamics and granular materials simulations. The simulations are commonly developed in a monolithic style where the whole simulation is treated/built as a single unit. As for setting up the development environment, usually it only requires the developer to deal the the makefile. As a newcomer in the field of software engineering, I found that things are not quite the same over here. The "recent" trend is that the application can be divided into several small services (microservice architecture) and communicating via HTTP resource API (API-centric architecture). This practice is often applied on applications that are being deployed on the cloud as it allows each services to be deployed and scaled independently.
However, setting up the development environment which involves these microservices is tedious and time consuming (at least for a newcomer like myself). After poked around to see how others are dealing with this "problem", I've found that everyone has their own way of doing it (even for the same project):
- Launch the services in local host using containers i.e. Docker.
- Develop the application inside a virtual machine (VM).
- Sometimes it is easier to just push the code to the repository and let team's continuous integration software (e.g. CircleCI, etc.) to run the test for you...
- Instead of setting up / running a local services, some would use the cloud services directly during development...
The "solution" of setting up the local development environment which I am looking for should fulfill the following requirement:
- Can be properly "documented" i.e. in such way that it can be stored in a version control system.
- The environment should be as close as the one in continuous integration, staging, or even production.
- The environment should be isolated, can be deleted and start over easily, in case if I screw up.
After some digging, I found that these can be done by combining Vagrant (which is commonly used to creates and configures virtual development environments) and Docker. Note that none of these is my original idea. They are the results from what I've found and rewritten/documented here in this blog to show how it can be done for Golang development.
Source
For demo purpose, I've created a simple "web app" in GitHub. The source of the Go-app with a simple test-file is located in hello/ directory. Basically what it does is displaying the information of the system on which it is running. In order to demonstrate possibility with multi-envoriment, we also include an "access counter" in the app which will be incremented on every access. The value of the counter is being stored in Redis.
Proxy
First of all, if you are under a corporate proxy, you need to configure the virtual machine to use the specified proxies by using vagrant-proxyconf. After installing the plugin do:
$ export VAGRANT_HTTP_PROXY="http://proxy.example.com:1234"
Getting Started
The key of this approach is to use Docker as provider in Vagrant. Using Vagrant, we can spin up a Linux virtual machine to run the Docker containers. Which also means that, we can customize the Docker host so that it is as close as to the one in production.
Let's start by examine the Vagrantfile in the root of the repository. Here we are building two environments:
- The "redis" environment is built from the Docker Index.
- The "hello" environment is built from a
Dockerfilein./hellodirectory and link to the "redis" environment. The.volumesparameter is for sharing directory between the "hello" environment and the Docker host (not the local host).
Note that for these two environments, we use .vagrant_vagrantfile to specify a Vagrant-managed machine (defined in docker-host-vm/Vagrantfile) as our Docker host:
By using docker as a provisioner in our Docker host, the docker provisioner will automatically install Docker in the Docker host machine. We also configure the share-folder between the local machine and the Docker host machine over here.
The following line terminates all ssh connections. Therefore Vagrant will be forced to reconnect. That's a workaround to have the docker command in the PATH
config.vm.provision "shell", inline: "ps aux | grep 'sshd:' | awk '{print $2}' | xargs kill"
Vagrant Up
Let's startup our environments. Because we are linking to another container in the same Vagrantfile, we need to call vagrant up with the --no-parallel flag.
$ vagrant up --no-parallel
Then visit http://192.168.33.10:8000 or http://localhost:8080 to check if it is working.
Running Test
Of course we can edit our Go source file in our local machine using our favorite editor. Then, test can be run using the vagrant docker-run command. Vagrant will spin up another container to run the test.
$ vagrant docker-run hello -- go test hello
And as a "bonus" you can view the log of each environment with vagrant docker-logs: e.g.,
$ vagrant docker-logs hello
Summary
This sums up how to setup a development environment by combining Vagrant and Docker which allows us to have more control on the Docker host itself. Of course there are many other ways for achieving a similar result such as Docker Compose and the successor to Vagrant: Otto (also by HashiCorp). Finding one which suit your development style might need some experimenting and fiddling.
