Keep your project running locally

Every web project starts out with the best intentions, running locally, unit testing, easy deployments, provisioning of servers. As a project moves on, some of these niceties drop by the wayside, specifically: The ability to run locally.

Never let this happen to a project. You cannot afford the cost of it.

It happens to all projects for the same reason. You have some bug you need to fix, or are working on some feature that NEEDS to get out and you decide to just fire up vim on staging/production, fix it and make the change later.

Eventually, (and it doesn't take long) this creates an environment that cannot be run on a local machine. It's even more likely to happen if you're on Windows.

There may be other things to consider though. Servers are almost always a different operating system and have different dependencies. They probably aren't capable of provisioning themselves automatically. The dev environments also might need to run a separate process (like a worker thread, or solr or something)

Keeping the project running on the dev's boxes is no easy task.

Why is a project running locally so important?

When a project can only be run on a server, you have to do a few things you wouldn't have to do on a local box.

• (Generally) you can't have your own text editor setup, especially if the text editor of choice isn't a console one
• You're probably not using automated tests
• Someone else deploys to that server, all your work is gone
• You have to SSH in
• Somehow you have to transfer files back and forth. Most devs will use their own memory or copy-paste, better devs will use rsync, but even rsync sucks here
• Restart the server after a change (or enable code-reloading)

In the end, everything takes more time. When I code like this, I'm probably less than half as efficient as I could be. Not so much the extra steps, this is the sort of thing that makes an engineer's life miserable.

Most companies I've worked at have had this problem, and it's one of the things that ultimately made me quit working for them. Perks and money are nice, but having to go through this everyday is something that I personally cannot deal with for very long. It tears apart my sanity.

This will immensely help with onboarding. Since new engineers won't have to be brought up to speed with as much. They can also be comfortable messing around in their own environment.

How can you avoid this problem?

I'll be honest, it's not easy to avoid this problem. It takes serious dedication and willpower. Again though, it's not worth it to give up on.

Here's what you need to do:

• Be able to create a new dev environment very easy
• Be able to provision new servers easily
• Make production as close to the development environment as possible
• Use code-reloading on the development environment
• Use git

There's some tools out there to help with this. The ones I'm going to mention are all ruby-focused, but I know many python and php developers that use them and they work great.

Vagrant

Vagrant is sort of a command-line wrapper around virtualbox. Sounds totally boring and useless, but it's designed specifically to tack most of the problems I just mentioned. It can not only start up a vanilla linux distro in 2 commands, but it can provision them as well!

So if you have a Vagrantfile in the root of your project, a new developer only has to type 'vagrant up' into the command line, and they'll have the ability to run a fully provisioned, ready to go production-like environment.

It will also link the local project directory to one at /vagrant on the box, so any changes made locally take immediate effect on the box. It also sets up port forwarding so you can just hit 127.0.0.1:3000 and you'll be hitting the vm.

It can use either a bash script, or chef or puppet (or a combination) for the provisioning.

I don't use it on every project, it does take time to setup. Most of my projects will run just fine over osx. However, if you're on Windows, you NEED this.

You can also mess around with crazy things like updating all your gems, trying out the next version of django, whatever you want. You feel comfortable knowing you can destroy the whole environment and regenerate it.

Another bonus: Whether you use chef, puppet or a straight file, you've got the ability to provision a production machine as well. Load up a new ec2 install, run the provisioner against it and you've got a production ready machine that works just like the dev environment.

Chef + Puppet

Chef and Puppet are the new big deals in the unix community. They provision servers. I've used both, and hated both. They're difficult to learn in my opinion. I have used both enough to get by, but it's not an enjoyable experience like I feel like it should be.

I probably just need to learn them better.

Still, they're the best out there I know of for you're provisioning servers.

Capistrano + Fabric

Capistrano and Fabric are deployment tools. When your server has been provisioned, run these to deploy/start/stop/migrate dbs, whatever you might need. Use one of them.

Foreman

Foreman is VERY simple. All it does is let you have a file like this

web:    bundle exec thin start -p $PORT
worker: bundle exec rake resque:work QUEUE=*
clock:  bundle exec rake resque:scheduler

And if you run 'foreman start' at the command line, it'll run like this:

Image from David Dollar's Introducing Foreman

Easy way to ensure that the server is being started the same way by everyone, and all the dependencies get started as well. This is so you don't have to have an extra command window open for your worker server.

Summary

As many people know I am a big fan of Heroku. One of my favorite things about Heroku is you cannot access the server. I know a lot of engineers that would see this as a drawback, but I see it as prevention against me shooting myself in my foot.

So there is some tips on the worst thing a project can do to an engineer. Like I said, these are not easy problems to solve, which is why I only told you what the tools you need are and where to find more information on them.

Spend the time to use them and learn them, it's worth it.

Source: http://jeffdickey.info/keep-your-project-running-locally