Location Independent Software

When I started working for the university, everything I developed was designed to work on a single, well known set up. I didn’t have production, pre-production and testing set ups. I just hacked away and if I made a mistake I fixed it quick. Some of the ECS Intranet still runs ike that and it’s not a terrible system for a small team with a userbase who care more about getting new features than they do about 100% uptime.

However, I’ve learned over time that there’s lots to be said for being able to develop and test software separately to the live setup, and more importantly just set up a new install for someone to develop on on their desktop. Many of the new services we develop are set up like this, not as many as I’d like.

Sysprog vs Developer

I’ve been trying to train myself out of the habbit of “hard-wired-paths” in my code. It’s as easy to avoid as unnamed magic constants, if you clean up as you go. It’s also pretty easy to retrofit to an existing setup, if you can use grep well enough.

Basically:

BAD:  open( “/home/systemx/v2/etc/config.json” )

GOOD: open( $BASEDIR.”/etc/config.json” )

I should note at this point that I generally design applications for db and web, and they almost always have a /var and /etc directory under the base path of the install, rather than scatter the install around the filesystem. I like stuff which I can “git clone” or “tar xzvf” and have it work in place, it makes it easier to commit changes back to the git repository.

Basepath

If it’s a web-based app then you can put configuration into the apache vhost configuration, to set the base path of where the application was installed, but you don’t need to. Most languages I use can work out the absolute path of directory the script you are running was in, and from that the relative path to the base install. eg. if the script is /where-you-installed-the-app/bin/process then you can get that path from within the “process” script and then just add “/..” to get the base directory. If the script/library is deeper in the install path then just add a “/..” for each additional level.

For my own benefit, as much as anybody else’s, here’s a cheat sheet of my most-used-language’s ways of doing this:

Perl

use FindBin;
use lib "$FindBin::Bin/../lib/perl";
my $BASEDIR = "$FindBin::Bin/..";

PHP

Thanks to Andrew Milsted for a much more concise version than mine.

$base_dir = dirname( __DIR__ );
require_once( "$base_dir/lib/php/utils.php" );
require_once( "$base_dir/settings.php" );

BASH

This one I learned more recently (bless stack overflow and all who sail in her):

BASE_DIR=`dirname "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"`

Others?

If you can contribute similar nuggets for other languages; Java, node.js, Ruby, Python, C, etc. please leave them in the comments.

Secrets and local stuffs

This may sound obvious, but you don’t want to commit your passwords, and other secrets, into the git/svn repository. Especially if it’s on github.com!

There’s also other things which may vary between two copies of the same tool; locations of libraries, the URL (if it’s a web thing) and maybe some other stuff. The way to do this is to move all of this to one config file and then carefully make sure that it is never checked into the version control system. To make life easier, you can make a fake version, usually called something like config.ini.template which has a reminder of all the config elements needed, but not real passwords, and you check that into the repository so when you do a new install, testing copy or whatever, you just copy that to config.ini and edit it.

Hostnames

A good example of me cocking this up recently was we cloned the entire virtual machine that a service was on, so that we could have a development environment. That meant we clone the mysql database, right along with it’s username and password. All of that would have been fine if I’d used “localhost” as the hostname, but I used the absolute domain name of the machine… so for a little while the scripts on the test server were updating the live database. We caught the error and fixed it and no damage was done, but it added to the ever growing list of “things to look out for next time”.

URLs

The URL of a website should also be configuratble. We’ve had a few occasions where we did this, but the template file had absolute URLs linking to javascript or css which had to be unpicked when discovered causing confusion.

For extra bonus points, allow the software to be installed so that it runs on a subpath of a domain, eg. http://example.org/myapp/ rather than just assuming it’s going to get domain name to itself, which is easy for some people but a showstopper for others. If you don’t design everything from the ground up with an adaptable URL path in mind then this can be one of the most time-consuming things to retrofit.

An advantage of just instisting your application runs a the base path of the doman: http://myapp.example.org/, is that you can write the code without the domain name at all, just make all the links relative to the base of the site “/foo/bar.css”. There is a reason this can suck though; if you produce any kind of API which exports HTML to other sites (many of our tools at the uni package up HTML chunks to be used elsewhere), then the HTML links will be broken when shown on the new server. This is most commonly manifest in missing javascript or images.

bin/ directory style

These are petty, but I’ve decided I have some thoughts on good practice in /bin/ directories.

The first is to use hyphens in command names, not underscores. My team tends to use them interchangably, but the command name is a verb, not a name. Underscores are fine for passive cattle like config files. I admit this is pretty arbitrary, so the biggest no-no should be mixing in the same directory, something I am guilty of myself.

I also suggest not adding filename suffixes to bin (command-line) scripts. They are not needed on UNIX, Linux, OSX setups, as the script uses the #! to determine what to run. There’s a good reason for this. I’ve had people be nervous of using a command I wrote in Perl because they “don’t know Perl”. It sounds silly, but by removing the suffix you are indicating to the user of the script that they don’t need to care about how it works, just what it does.

Following on from the bit about absolute paths, don’t put absolute paths in the #! line if you can help it. There’s incantations for working out the correct location of tools like bash & perl. Also, it’s really a pain to have something like “#!/usr/bin/perl -I/opt/eprints3/perl_lib”. That EPrints example was one of mine, and checking the EPrints project commit logs, it was tidied to using “FindBin” by Tim Brody in 2010 (thanks, Tim).

#!/usr/bin/env xyz

There’s a technique I’ve seen used, which is to not use

#!/usr/bin/perl

but rather

#!/usr/bin/env perl

…which seems to use the PATH to work out where to find the perl command. While this is kind of clever, I’m not convinced it’s 100% safe (what if the user had a weird copy of perl in their path?), but I’d be interested to know if people can tell me how safe it is.

Other  tips?

Have I missed any obvious, or not-so-obvious, best practice or handy tips?