unfoldthat.com

Using virtual machine (vagrant) for you web development

In my opinion, environments on the development machine and production machine must be as close to each other as possible. First of all, you should use the same versions of Python, Django and everything else. If you are going to use Postgres, don’t test everything on Sqlite (especially if you’re dealing with Unicode, or changing schema, or using unique indexes, or just writing raw SQL). If you want to debug your search, you can’t avoid installing search server. Then there will be a bug on production which is related to these difference between it and your cool laptop, you’re gonna have a tough time debugging with vim (or something else, but unconvenient anyway). So we at r00 have standards, and one of them is that we trying to make dev, test and production machines look same.

But there is a problem. First of all, installing Postgres on my local machine sucks. Installing Solr and setting it up somehow to work with several projects even worse. A week ago I’ve bought a shiny new Macbook Pro and installing Ubuntu (our production default) on it would suck too. And if someone new would be going to develop some of our projects, he would be going to install it all again, by hand, and he would be going to fail at at least one single step.

So here is the solution: a virtual machine! If you never used a virtual machine during the last couple of years, I want to assure you that they’ve become fast, good, stable and you won’t have to deal with GUI at all. One great guy have written a ruby gem named Vagrant, but since I’m blogging about Django and you’re my reader you probably either never heard of it or isn’t used to deal with gems¹, so I’d like to help you a little bit.

Vagrant is a virtual machine manager with a great (because its simplicity, not power) command-line interface. It works awesome with Ubuntu and Mac OS, it works good with Gentoo, and I didn’t dare to try it on Windows but it seems to work there too. It’s fast, it almost never crashes horribly, it depends on VirtualBox which is actually pretty good and free, so you should try even if you’re in doubt.

As a bonus section, I’ll show you some Chef, which serves Vagrant machines better than a regular (e.g., ec2) machines. You’re going to like it. At least you should know about it. It’s actually a horrible piece of software, but on the other hand it’s cool, it can do a lot and it gives me a cute sense of archievement since I one of these rare people who is able to deal with it.

Installing Vagrant

Okay, here is the disclaimer: Vagrant is written in Ruby, so you have to be careful. It’s distributed using rubygems, so please be very patient. And always understand which version you are installing, because they fuck up everything on every upgrade². Never use your system package manager to install gems, because they are really outdated³.

sudo gem install vagrant --no-ri --no-rdoc -V

• if it says that there is no such command as gem, install rubygems.
• --no-ri --no-rdoc saves your time, because for some reason installing docs is slow with gem. You really don’t need docs for Ruby software, they are bad, not interesting and outdated.
• -V is for “verbose”. You should use it, because gem is very slow, and it just sits in silence for a long time, and you’ll be getting nervous without -V.

Also don’t forget about VirtualBox. It’s actually simpler: try apt-get on Ubuntu or download it for Mac OS (I’m sure you’re able to figure out how to install it for any other OS).

Creating your virtualmachine

You don’t need a specific folder for it, so you can just go the main folder of your current project and type:

vagrant init

This creates a config file named Vagrantfile. Take a look in it. The very first option is config.vm.box, this is a template for your virtualmachine. You’ll actually have to download such box. Vagrant can do it, but the box is big and the internet crashes, so I prefer wget:

wget http://files.vagrantup.com/lucid64.box # just run this command again with "-c"
                                            # to continue downloading if it stops 

I’m using 64 bits because more is better :-) And add this box to vagrant:

vagrant box add lucid64 lucid64.box

You can remove the downloaded file now. As you can see, I added it a more meaningful name than “base”, and you should reflect it in the Vagrantfile:

config.vm.box = "lucid64"

Nothing else is needed to tweak now, just run “vagrant up”, wait a bit and connect to your virtual machine using ssh:

vagrant ssh

Wow, that’s it! You can now do things. Don’t forget to tell your VCS about vagrant:

git add Vagrantfile
echo ".vagrant" >> .gitignore

Provisioning

You can install everything you need by hand on your virtual machine, but you don’t want to. You don’t want it, because you want to share this environment with people so they can install it on their virtual machines, and because if you create a set-it-all-up script for your VM, it all of a sudden will work on your “real” (actually, just as real as your VM) VDS box, or even on your really real dedicated server, so you can experiment and don’t be afraid of setting it all up again. Also it becomes simpler to create a new server (almost) instantly, that means that you’re web scale now.

So that’s what Chef for: provisioning, creating setup scripts. It’s integrated in Vagrant, so you can just start to use it right now without buying server from cloud hosting companies.

Chef’s configs are a collection of files. Templates are avaiable as repositories. One of them is blank, others aren’t (but you’ll have to google them). Please clone such repository somewhere in your project folder, remove chef-repo/.git/ folder and add chef-repo/ to your version control:

git clone https://github.com/opscode/chef-repo.git
rm -rf chef-repo/.git
git add chef-repo/

The very thing you need now are cookbooks. They are stored in the chef-repo/cookbooks/, but if you cloned the blank repository, there aren’t any now. There are several collections of cookbooks, you can use these:

• http://community.opscode.com/cookbooks (they say these are better than other ones)
• https://github.com/opscode/cookbooks
• https://github.com/37signals/37s_cookbooks
• http://github.com/engineyard/ey-cloud-recipes

You can try to use several ones like nginx or something, just to play with it. There almost no cookbooks for Django, but they’re simple to write (maybe I’ll show you how in the later blog posts). So, download now a cookbook or two and tell vagrant about it by editing Vagrantfile:

config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = "chef-repo/cookbooks/"
    
    chef.run_list = ["recipe[postgresql::server]",
                     "recipe[nginx]"]
end

Syntax for referencing a recipe is cookbook::recipe. If you omit the second part, it will use default as a recipe name.

Now you can ask Vagrant to install things for you:

vagrant provision

Next time: how to set up and use Chef for deployment. In the meantime, try running Django’s runserver from inside the VM. You’ll have to forward a port (take a look at Vagrantfile for an example) and run runserver with a parameter to bind it to 0.0.0.0.

Update: Fixed links to cookbook repositories, thanks to Seán B. Powell for noticing.

6th of May, 2011

Please send your comments, ideas, rants and job offers at v.golev@gmail.com.

Made with Nginx, Jekyll, Git, EC2 and Emacs.

Regarding all the advertising and readability money-related stuff: I'm sorry. I'm really interested in whether one can make money by showing ads in his blog. Seems like I can't. If someone is really annoyed by adwords, drop me a line, I'll reconsider.