In September I was fortunate to be able to give a talk at DjangoCon US, the largest annual conference about the Django web framework. The topic of the talk was on Vagrant, a free open-source tool facilitating the manipulation of virtualized environments, and how it may benefit the development of Django applications. This talk was aimed at Django developers of all levels who were interested in getting an overview of the great possibilities Vagrant offers to support teamwork and quality assurance. In this article, I provide the same overview of the issues and solutions as addressed in the talk, albeit in broader terms that should be compelling to all kinds of web development teams, regardless of their framework or programming language of choice.
Common issues for web development teams
Over the last couple of years, a number of things have contributed to making the lives of developers easier, in particular the emergence of cloud computing, of both backend and frontend testing frameworks, and of distributed version control systems such as Git. However, the web stack specifications have also become more complex in order to provide the features that are now commonly expected by web users and to respond to ever increasing levels of traffic on modern web sites and applications. Indeed, it is now often necessary to use components such as RabbitMQ, Hadoop, Solr or Redis, just to name a few. As a result, web teams nowadays tend to be recurrently confronted with two types of issues:
Vagrant to the rescue
The issues described above would not occur in an ideal world where all developers of a same team worked on the same, pre-built platform: there would be no need to manually install anything and the working environment would automatically be assembled so that developers may solely focus on their development tasks. In that ideal world, the development and production platforms would also closely match so that no more bad surprise would occur when new code got pushed live. The good news is that this world already exists — and it’s called virtualization.
Virtualization is nothing new, really. It has been around for a long time. However, in the last couple of years it has become much more approachable and useful to developers, especially thanks to a project called Vagrant. Essentially, Vagrant is a developer-friendly interface for managing virtual machines (VM). Currently the only VM manager supported by Vagrant is VirtualBox, but there are plans for it to support VMWare in the future as well. In a nutshell, with Vagrant the workflow for a given developer is as follows:
The developer interacts with Vagrant by running a few simple commands. The main command is ‘vagrant up’ (1) for booting the VM up, that is the equivalent of pressing the ‘on’ button of a physical computer. The developer also has to provide a configuration file named ‘Vagrantfile’ (1), which contains some basic directives to specify the characteristics of the VM that needs to be built. Then Vagrant will talk to VirtualBox (2) and instruct it to actually create the VM (3). Vagrant will also talk to some provisioners (4), which are responsible for provisioning the VM, that is, installing and configuring all the software required for running the project’s application (5). Once the VM has been created, booted and provisioned, the developer may directly access the VM (6), for example via SSH, exactly in the same way as they would access any remote server, except that in this case the server actually lives in a local VM inside the developer’s host machine.
Multiple types of provisioners may be used in conjunction with Vagrant. Chef and Puppet, both ruby-based projects, are probably the most popular ones. Salt Stack, a more recent project, is also quickly gaining popularity, in particular amongst the Python community. Regular shell scripts can also be used, so if you are familiar with writing shell commands then you will feel right at home. Chef, Puppet and Salt Stack can be regarded as provisioning frameworks in that they abstract out the logic for installing and configuring software on multiple types of platforms (e.g. Ubuntu, Debian, Solaris or AWS), allowing you to reuse the same provisioning scripts across any platforms needed for your project. Those provisioners use different syntaxes and follow slightly different paradigms but they all are very powerful. So, choosing a provisioner for your project really comes down to personal taste. Personally I have mostly used Chef as it has worked really well for me so far. I recommend trying them all out and selecting the one that feels the most comfortable to work with.
The Vagrantfile is what glues everything together. It references all the provisioning scripts and lists the specifications for the VM’s platform, for example what operating system the VM should be based on or how much memory it should be allocated. It also specifies the folders to be shared between the VM and the host machine — this is very important as it allows in particular to share the folder containing the project’s source files so that you may modify the code using your favourite editor on your host machine and let the VM access those same files to serve your application.
New team workflow
So, you have decided to give Vagrant a try and you are now wondering about how your team workflow will evolve. Here is how it could be summarized:
Finally, the individual workflow for each team member will remain similar to what they were used to in the past, as they may continue to edit code with their favourite editor or IDE and continue to browse their development copy of the web application using their favourite browser on their host machine.
When I asked Mitchell Hashimoto, the creator of Vagrant, how he chose the term “Vagrant” to name this tool, he replied, “The idea is that developers would be working in these transient, portable boxes. They would move around and would have no permanent residence. Like a vagrant.” This is a really important concept as the process driven by Vagrant allows the development environment to be automated and reproducible. In particular, this system scales really well as the steps for creating the development environment will remain simple for all team members over time, regardless of the current or future size of your team. Also, in the case where something accidentally breaks in any given person’s VM (for example, some software get corrupted or some important configuration files get deleted), then it is easy to tear down the VM and re-create it from scratch to start again with a pristine working environment. Having the entire software installation’s instructions coded in the provisioning scripts is also an opportunity to ensure that every team member works in an environment that reflects that of the production server, since the same provisioning scripts may be used for building the production platform.
Vagrant has completely transformed the way I approach teamwork for web development and I now systematically use it in all my projects. I recommend you try it and see what benefits this tool can bring to your own projects!