Building a local drupal development environment with multihosting

Ever wish you could work on coding a Drupal site without the hassle of the constant edit/ftp/test cycles? Or how about making changes when you have no Internet access available. Here are a few tricks many professional developers I know use; if you aren't aware of them yet, they could end up saving you a great deal of time and frustration. And it's always nice to be able to make some changes when inspiration strikes you, even when you're offline. Here's an approach I use with Drupal 6 sites.

The key is that Drupal is pretty much location agnostic; it is extremely easy to move a site from one hosting environment to another with just a few lines of code. Once you have a new environment configured, it will automatically use the correct configuration based on the URL.

The feature of Drupal that allows this is multihosting. Take a look under your sites directory; here you'll see a few subdirectories; typically the "all" folder, which will include a modules and themes folder, which will contain modules and themes available to all sites. Wait a minute; "all sites?" - isn't this a single site we're building with Drupal? Yes, but just because it's a single site, that doesn't mean it has to be recognizable at a single URL.

When you first set up a new site, you'll also see a sites/default folder, containing a settings.php file. The little trick we are going to use is the fact that if you create a folder under sites whose name watches your target URL, that's where Drupal is going to try to look for information about that specific URL.

Confused yet? I know I was at first, but it's actually quite simple. Suppose you have a site called http://www.mysite.com, and would also like to set up a test version of the site called http://test.mysite.com; this is remarkably simple to do. Assuming your www site is already running, create a place to host your test site. If you want to set it up on a remote server, go you your domain control panel (typically cpanel for most ISPs; this is where you also set up things like email accounts and databases), and create a new subdomain for your domain, calling it test. You'll also want to create a new database specific to the new subdomain.

You'll need to make a copy of your database; the easiest way for most folks is to use phpmyadmin to export your current database to a text file, then import it into the new database.

Create these two directories:

• sites/www.mysite.com

• sites/test.mysite.com

Next, copy sites/default/settings.php to each of them. Now here's the magic; edit the file

sites/test.mysite.com/settings.php

and look for the line near the top that looks like this:

$db_url = 'mysqli://DBUSER:DBPASSWORD@localhost/DBNAME';

where DBUSER is the database user name, DBPASSWORD is the password, and DBNAME is the name of the database. Of course these will be real values for your particular setup.

Since these are from your default settings, these values will be for your current database. Change them to reflect the new database you just created for your test site, and save the file.

If your new site is a remote site, copy the files there, using ftp or a version control system.

Now try going to the remote site you just set up. If everything went right, you'll be seeing your site at this new URL. If you get an error message that the site is offline, or mentions a database setting, make sure your settings.php for the new site is correct, with $db_url set ot the proper values; that's usually the issue.

Getting Local

Cool, we now have a test version of our site. You probably did this on a remote server. That's fine, until you're offline, or somewhere that you can't connect to the internet. To deal with this, let's take the internet out of the equation and set things up on a local machine. This can be your desktop box, but more and more I see people doing this on a laptop. Given the choice, I find a Macbook a little easier to deal with than Windows, simply because it is Unix under the skin.

Setting up a local web server is pretty easy these days. A one-click solution that also installs the latest version of Drupal for you is by using the Acquia Drupal stack installer. If you want a little more control, I'd suggest Xampp for Windows, and MAMP Pro for the Mac (the free MAMP works fine; I like the Pro version because it makes setup and dealing with lots of sites easier).

The naming convention I use is to create a subdomain on my local server, substituting "local" for "www". For example,

www.mysite.com

would become

local.mysite.com

on my local server. I would then create:

sites/local.mysite.com/settings.php

And access it from the URL http://local.mysite.com

This will not affect your www site, since the DNS changes are only made on your local. But you will need to add your host name to your local DNS settings; with MAMP Pro, this is just a few clicks, but otherwise you'll need to edit your local hosts file (Mac) or Windows.

More Sites/Settings Magic

So what about the sites/default directory, now that we've moved our seetings out of there? It still has an important role; it is the default place to look if something can't be found in a site-specific folder. It is considered good practice to have your Drupal file system path located in sites/default, rather than off the document root. That way, all your sites will look to this one single place for files.

But what if you want a separate set of files for your test server? Simple; just create a sites/test.mysite.com/files directory, and Drupal will look there on your test site (you'll also need to set the file location config setting; this is covered a little further down).

Similarly, if you have a module specific to a URL, you can create a sites/test.mysite.com/modules folder, and any module found there will take precedence over the ones in sites/default.

The order in which things are searched in Drupal is that it first looks for a sites folder matching the URL, and if found, searches it for a relevant subfolder. If none is found, it next checks sites/all, then sites/default.

Fine and dandy, but suppose I want to export the database from one subsite to another? Most of the stuff in it is not particular any subsite - with the exception of the variables table. When you change preferences for a site, it is stored in variables, not much you can do about that. Wouldn't it be nice to be able to somehow override these values for specific sites? Well, there is; take a look at the bottom of your settings.php file for the site, and look for the $conf array. Here you can override any configuration setting. The only downside of this is that it will always take precedence, so any setting you specify there cannot be changed via a configuration page on the site. But that's usually not the issue; this is needed only for things that are very site specific.

As an example, the place where temporary uploaded files are stored is usually somewhere like /tmp on a Unix system, but this will make no sense to a Windows server, and cause big problems. To remedy this, here's the $conf file for sites/www.mysite.com/settings.php:

$conf = array(
  'file_directory_temp' => '/tmp',
);

and the corresponding setting in my local settings, running under Windows:

$conf = array(
  'file_directory_temp' => 'C:\Xampp\tmp',
);

Any configuration variable can be controlled this way. A few more I find useful are:

file_directory_path - the file system path

• preprocess_css - whether to aggregate CSS (nice to have off in a developemtn environment)
• javascript_aggregator_aggregate_js - whether to aggregate javascript
• javascript_aggregator_optimize_js - whether to optimize javascript

Keeping Things in Sync

The last issue to face when maintaining multiple versions of the site is how to keep code and data in sync. The code part is pretty simple; I maintain a master copy on my local, where I make and test changes. When I'm done, I push them up to the remote servers.

In ye olden days, most people used ftp to do this. The difficulty with this is that you have to somehow keep track of which files have been changed, and be sure to copy only those that have. And you have to remember the various directories they are all in.

An improvement over this is using the Unix utility rsync, which can look at your local and remote directories, and figure out which files are in need of copying, and copy only those. This is certainly faster and will ensure all files are copied, but there is one issue with this; suppose you changed ten files, but only six are ready to be pushed?

This is where a good version control system comes in. I use Subversion for this; many developers are moving to git or Mercurial. What's important is that you do use something, since Drupal projects often involve hundreds of files spread across many directories, and you'll eventually mess up by not copying the right files over. A version control system allows you to check out a master set of files, modify them, and check revised version back in. So in the above scenario, I would check in only the six files that were ready to go, then go to my remote server and update the local repository. The remote would then be exactly in sync with my test server.

Version control gets even more useful if you like to take a few risks. If you make some changes that didn't quite work as expected, you can delete the file and update it from the repository, and you're back to the original. There are some cool advanced tricks too, like being able to tag all the files in a release set to a version that you can specify, in case you need to get a copy of a snapshot at a given place in time. But just using it as a smart way to synchronize sites is very easy and will make you more productive.

You can set up Subversion on your own computer and create repositories there, but I think it's a good idea to have your repository hosted at a company that specializes in this. The main advantage is you let them worry about backups, privacy, and networking issues. Plus all your eggs aren't in one basket that way. The company I use is sourcerepo.com; they are inexpensive, reliable, and have excellent support.

The Problem with Data

Database backups are a different story, and one that isn't easy to solve. I've gone to presentations every year at Drupalcon that deal with this, and even the experts admit this isn't anything easy to solve. My method is to do frequent database dumps, using phpmyadmin, or the backup/migrate module, that allows you to set up automated backups. Before starting work on my local, I will do a dump of the remote and import it to my local, so things are in sync.

But what happens when you're ready to push your updated code back to the remote, and have already made some database changes? If you simply export your local database and import it on the remote, you will lose any database changes on the remote from the time between your remote export and local export. This may not be an issue for a personal blog site, but could be a disaster for a busy ecom or social site. Unfortunately, you will probably have to manually make changes to things like menus and configuration settings. There are tools to export/import views and nodes that usually work reasonably well. Just be sure to make plenty of backups!

Version control doesn't really help us much here, since there is no way to directly transfer a database into vc; you still need to go through an import/export. Plus, version control gets its efficiency from storing just the difference between two revisions of a file. Database tables tend to be large, with lots of changes, and don't lend themselves well to delta analysis. If you try versioning a database dump, you're going to end up with large files, and spend a good amount of time on it.

I do sometimes archive a database dump in version control, simply because it can be tied to a known state of the site. When I do this, I carefully name the SQL dump file to a unique name based on the date, so I know when it was made (i.e. mysite_2010_07_04.sql). That way, I always start with a fresh file, and the repo doesn't need to compare it to earlier versions.