Someone else used to maintain a really good “ideal” WP requirements page, but it seems to have vanished from their site. I thought I’d start my own, for my reference, to point potential clients to, and maybe of use to others.

The server should be running:

• Linux
• Apache
• PHP 5+
• MySQL 4+

Also, these factors are very important:

• The mod_rewrite Apache module should be installed and enabled.
• PHP safe_mode should be OFF. Turning it on is a bad attempt to manage security on shared hosting.
• The PHP memory_limit should be 32 MB at least. If it’s not at least this by default, it should be possible to increase it to at least this.

Here’s my code:

RewriteCond %{SCRIPT_FILENAME} /wp-content/uploads/.*\.pdf$ [NC]
RewriteCond %{HTTP_COOKIE} !^.*wordpress_logged_in_.*$
RewriteRule .* http://domain.com/ [F,L]

The first line matches requests for PDF files in the uploads folder. You can change that however you want. The key is the second line, which is the match for the WP cookie you’ll have if you’re logged in.

Note that the name of the actual cookie has a string of random characters at the end, which I assume WP generates to make the login cookie hard or impossible to fake. I don’t know a way to access this value outside WP. I’d be interested if anyone knows how; I’d also suspect this would be a security hole in WP!

In the above example, if someone knew the PDF’s URL, didn’t have a login to your site, and really wanted to download the file, I’m sure they could fake the login cookie easy enough. If you need tighter security than this on downloads, you should probably look at a plugin like Download Monitor, which provides “mask” URLs for files, and thus can process login checks via PHP code from within the WP framework before returning the download.

Note also that I’m not sure that http://domain.com/ is necessary in the last line. The F flag after it returns a 403 status code, so you get the browser’s “Forbidden” page instead of any URL that you specify as the redirect.

Any suggestions for improvement to this quick-and-dirty trick welcome!

Until tonight.

I was forced to solve a situation involving a WordPress site which had some legacy flat HTML content. Way back when, I hacked it to use .html suffixes on the WP URLs, so we didn’t lose any search engine juice for the old URLs. This necessitated changing core WP code—always a bad idea—and thus blocked any automatic WP upgrade process. Which is a pain.

So, I’ve removed the .html hack, and tried to pull flat HTML content in for the old stuff, under the guise of smooth new .html-free URLs.

Say we have something that was at /news_2004_09.html. First, we redirect to new URLs:

RewriteRule ^news_([0-9]{4})_([0-9]{2})\.html$ /news/$1/$2/ [R=301,L]
RewriteRule ^(.*)\.html$ /$1/ [R=301,L]

The first bit redirects (301 = permanently) pages with the format news_[year]_[month].html to news/[year]/[month]/. The second line deals with all other pages that used to have .html endings.

Now to pull in flat HTML:

RewriteRule ^news/([0-9]{4})/([0-9]{2})/$ /html-news/news_$1_$2.html [L]

Note the absence of the R=301 flag. Without this, an “internal rewrite” instead of an “external redirect” happens. For the end user, what this amounts to is that the URL you typed in or clicked on will stay the same—just the way the resource used by the server for that request will alter. R=301 (or any other HTTP status code) will make the URL in the browser change—meaning this URL will get bookmarked, indexed, etc.

All well and good, but (and this is where my last couple of hours have gone)… Even though no R flag means it’s an “internal rewrite”, and the L flag indicates this is the last rule to be processed, Apache still effectively “makes another request”.

I don’t think this involves more traffic between the server and browser, and I think it’s related to the fact that the rewriting is being set up in .htacccess (rather than at the server level, in httpd.conf). I’m not quite sure.

The end result is, all your other rules get processed again, even though you gave it that L flag! So for me, the old URL was going through the following rewrites:

1. http://example.com/news_2004_09.html (initial URL)
2. http://example.com/news/2004/09/ (permanent redirect)
3. http://example.com/html-news/news_2004_09.html (“internal rewrite”, not visible to browser)
4. http://example.com/html-news/news_2004_09/ (another permanent redirect)

See what happened? My catch-all code for changing old .html URLs was processing the “internal rewrite” URL. That last step shouldn’t have happened. The browser’s URL should have stopped at stage 2, with the content coming from the rewrite in stage 3. Instead, I got the URL in stage 4, with no content at all.

The solution? Well, thanks to Richard K on the Mod_Rewrite forums, here it is:

RewriteRule ^news_([0-9]{4})_([0-9]{2})\.html$ /news/$1/$2/ [R=301,L]
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^(.*)\.html$ /$1/ [R=301,L]
RewriteRule ^news/([0-9]{4})/([0-9]{2})/$ /html-news/news_$1_$2.html [L]

I worked out a while ago that I needed a RewriteCond before my catch-all rule, but I couldn’t for the life of me find the condition to test if the current request is a result of an internal rewrite already.

ENV:REDIRECT_STATUS is my new best friend.

There’s been a whole series of hoops to jump through. I’m no tech novice, but Windows and occasional Mac use has been my staple thus far. Of course you can’t do much in web development without knowing a bit of Linux at least; still, wrangling with various desktop issues can be a chore.

The web, or Google (if there’s a difference) is your friend. Finding a comprehensive tutorial that covers your precise situation is rare if not impossible; you need that familiar net skill of cobbling together varying perspectives and bits of advice into something that works for you. That said, I’ll point to this ACME Guide up-front as a fantastic source of information on getting web development stuff going.

I also picked up the O’Reilly Linux Pocket Guide. It’s a good little intro to the Linux environment as well as a useful reference point.

Anyway, here’s my summary of what I’ve learned. Hope it’s useful for you! (As ever, this information is used at your own risk. I can’t support you if stuff goes wrong—but do let me know if you know for sure I’ve, for example, mis-typed commands!)

Ubuntu Eee

I got a 40GB Eee with Linux pre-installed. Knowing I was going to shift to Ubuntu anyway (the default version of Linux on the Eee is the clunky Xandros), I probably should have gone for the 80GB Windows version—it’s cheaper!

Anyway, in either case, the good news for people wanting Ubuntu is that there’s a special distribution of this system especially tailored for the Eee: Ubuntu Eee. Follow their instructions.

My only real sticking point here was that the UNetbootin-created USB installation drive didn’t boot. The Eee doesn’t have a CD/DVD drive, so this utility unpacks the disk image (ISO) of the Ubuntu system onto a USB stick, making it bootable. I only solved this by getting a new USB stick for this purpose. The first one was oldish, and only 1GB. This is theoretically enough, but they say a 2GB stick is recommended. Indeed.

The /home directory

The Ubuntu Eee install is pretty slick. The only thing I think I missed was relocating the /home directory. This is where most user files are kept in Linux. The Eee seems to come with its main hard disk (actually a solid-state disk) partitioned into two—mine had one with around 8GB, one with around 32GB. I wiped the Xandros installation off the smaller one and installed Ubuntu there. It was only after installation that I got round to thinking my /home should be on the big partition for all my data…

There are other options of course—search and you’ll find myriad opinions. If you end up in my position, here’s what to do. I waded through a number of ad-riddled blog posts and forum threads, but this one seemed to nail it in the end.

Be aware that some of this can go wrong. I managed to make some drastic mistakes here, but Linux is quite sturdy and I managed to recover OK. This definitely isn’t for total beginners.

1. Close all applications. Print this out or write it down.
2. Open a command line terminal.
3. sudo su
   This shifts the session to “root” or “super-user” (after you enter the password for your own user account). You can prefix individual commands with sudo to temporarily shift from your own user account that was created with the Ubuntu installation to the root account, needed for many system operations. But it can get tedious. This trick keeps you operating as root until you type exit.
4. cd /media/HOME
   This shifts you into the large partition, which, in the default Ubuntu-Eee install, is mounted with the label “HOME”. Linux devices (like disks and drives) are generally registered in the /dev directory. My boot partition is /dev/sda, and my O’Reilly guide says sda is usually the “First SCSI” device (whereas the master hard disk should be hda). I guess this is down to the Eee having solid-state disks instead of hard disks? A detail to watch for. Anyway, this large partition’s mount is here in the /media directory, usually for external devices—there’s a /mnt directory for usual hard disk partition mounts.
5. Now, there’s probably some user directories in /media/HOME left over from Xandros. Obviously grab what you might need, but I felt like a clean slate. Beware, this next command will wipe that partition:
6. rm -r *
   Remove (rm) recursively (-r)—that is, all sub-directories and contents—everything in the current directory (*).
7. unmount /media/HOME
   Pretty self-explanatory. A useful command you can check this with is df—it’ll list all mounted filesystems.
8. mv /home /home.bak
mv “moves” (actually renames) a file or directory. Here, it basically backs up the home directory by renaming it.
9. mkdir /home
   This creates a new /home directory.
10. mount -t ext3 /dev/sdb1 /home
    Now this re-mounts the large partition, /dev/sdb1, under the new /home directory. So now /home points to the partition. Do double-check the partition’s reference first. Use fdisk -l, which will (for root) show all connected disks, mounted or not, with their details. /dev/sdb is the second (“b”) SCSI device, and /dev/sdb1 is its first partition. It’s usually easy to identify drives and partitions by their sizes. Note that ext3 is the filesystem format (like NTFS in Windows).
11. cp -a /home.bak/* /home
    This copies the contents of the old /home into the new partition.
12. You should be up and running now with /home located on the large partition. The only thing left to do is make sure this mount happens each time you boot up. This sort of stuff is managed by the /etc/fstab file. If you’re still root in the terminal, you can open it for editing by entering:
    gedit /etc/fstab
    This will open the file in the text editor, with root permissions (essential for saving it!). If you’re not root anymore, just prefix the above with sudo (you’ll get used to this).
13. At the bottom, add the line:
    /dev/sdb1   /home   ext3   nodev,nosuid   0   2
    By now I imagine you’ll roughly see what’s going on in this.

Automounting the USB stick

That’s hopefully it for moving /home. But while we’re editing fstab, there’s another thing. I found after installing Ubuntu, the USB stick I’d successfully used was no longer “automounting” when I plugged it in. After a bit of searching, someone suggested commenting out the lines referring to /media/cdrom. Just put a # at the start of each line. Seems to be something to do with how the system has to recognize the stick as a boot device initially, which is no longer the case. Anyway, worked for me.

LAMP

After all that, LAMP (or AMP at least—Linux is done!) brings good news.

In Ubuntu, open up Synaptic Package Manager. This is basically a nice GUI for the “Advanced Packaging Tool”. You’ll see plenty of command line examples for installing software (“packages”) on Linux with the apt-get program—the part of the APT family that grabs software from remote repositories. Synaptic makes it all a breeze.

Select Edit > Mark Packages by Task, and from that list select LAMP Server. This will check all the components of the LAMP bundle of packages. You may as well also search (with the Synaptic search tool) for “phpMyAdmin” (the trusty web interface for MySQL) and/or “mysql-admin” (the MySQL desktop GUI), and check one or both of those. Just click the box at the left of the package listing and select Mark for Installation.

If you want to install ColdFusion, too, search for “sun-java6-bin”. CF’ll need it.

Any time you use Synaptic, once you’ve selected everything you want (a process that might also auto-select other packages that the ones you’ve chosen are dependent on), click Apply.

One of the nicest parts of getting all this going was, after installing LAMP, opening up a browser and going to http://localhost/. In big letters I found that “It works!”. Hope you do too.

For more details, do refer to that great ACME Guide (though you might need to ignore the stuff about 64-bit versions).

Keeping www somewhere else

As I’d decided to keep all my data on that other partition, I also wanted to keep my www directory, for all the local files of websites I’m developing, there too. I was kind of dreading some trawl through Apache configuration, but thankfully Linux offers a simple solution: a symbolic link.

Also known as a symlink or soft link, symbolic links are just small files that direct operations from one part of the filesystem to another.

1. Create your web files directory in your user folder, e.g. /home/steve/www.
2. Apache’s default web directory is /var/www. If there’s anything in there you want, copy it to the new one. Then delete the directory.
3. In a terminal, as root (or using the sudo prefix), enter:
   ln -s /home/steve/www /var/www
   Obviously, change the steve bit! The -s specifies a symbolic link. The next bit is the target directory to point to, and the bit after that effectively recreates /var/www as a link to it.

Now you can keep your web files in /home/[wherever]/www, and anything referencing anything starting with /var/www will point there.

Oh, if you’ve wondered, in looking in the www folder, where the heck phpMyAdmin is—you’re not alone. Mine works fine at http://localhost/phpmyadmin/, but I’ve no idea how this is so, or where it points. Ah well.

ColdFusion

There’s not much I can add to the ACME Guide’s lowdown on this. It’s a good brief education in some basic Linux concepts, too. Thanks to Adrian J. Moreno.

Fonts

I’ve still got loads to learn, but one final tip that will hopefully help you enjoy Ubuntu more. This page pointed me to Microsoft’s core fonts for the web. They should make browsing a little more pleasant. You can install them from Synaptic (of course)—search for “msttcorefonts”.

That’s it for now. Have fun!

Ever wanted to have a system in place that allows you to easily “switch on” a holding page for the whole of your site for when you need to do some maintenance? Well, that’s relatively easy to do; but what about bots? Even if you’re only down for 10 minutes, what if your luck is such that Googlebot makes its random rounds at precisely that time? Depending on how you’re holding page works, it might register a load of “404 – Not Found” errors, or replace your indexed content with your holding page… Who knows? Not I.

Well, with yet another WordPress upgrade (2.1.3) just out, I thought I would try to get to the bottom of this holding page issue. WordPress upgrades might be made smoother in a future version; but for now, it’s a slightly brutal case of deleting your live files and replacing them. Even without discovering some hair-tearing plugin incompatibilities along the way, or accidentally overwriting a crucial hack you’ve coded into your installation, a proper upgrade can take 10-20 minutes.

The only really useful page I found on the issue of bots and holding pages was over at AskApache.com. You can head over there and work out your own adaptation; I thought I’d document mine here for reference.

Note that the basics of this require PHP running on an Apache web server, with mod_rewrite enabled. The rest assumes you’re using WordPress.

The holding page

I created a file, 503.php, sitting in my site’s root. The code looks something like this:

<?php
header("HTTP/1.1 503 Service Temporarily Unavailable");
header("Status: 503 Service Temporarily Unavailable");
header("Retry-After: 3600");
?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xml:lang="en" lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Site upgrade in progress</title>
<meta name="robots" content="none" />
</head>
<body>
<h1>Site upgrade in progress</h1>
<p>This site is being upgraded, and can't currently be accessed.</p>
<p>It should be back up and running very soon. Please check back in a bit!</p>
<hr />
</body>
</html>

Those first two lines of PHP set the request’s HTTP headers to the “503 Service Unavailable” status code. HTTP headers are invisible when you’re browsing the web, but are read by the browser itself and by bots crawling around (and can be checked out via tools like Web-Sniffer and the Firefox Web Developer Extension). This status code means:

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay.

Just what we want.

The Retry-After bit is used to indicate how long you expect the site to be down for (in seconds). If you are hit by a search engine bot while down, it’ll probably understand this, but of course it doesn’t mean it’ll return as soon as that time’s elapsed. It’s probably got better things to do. At least it knows not to return too soon.

The rest is a plain page for humans, letting them know in English what’s going on. It needn’t be as minimalist as this; you can add branding if you want, and maybe a link to another site to be polite and give people a “way out”.

Allowing yourself access

The next step is to alter your .htaccess file to make sure all requests go to the 503 page.

But hang on – what about your good self? How do you test out all the maintenance you’re doing if you’re just being sent to 503.php with everyone else?

Basically, you filter requests by IP address. Assuming you have a static IP address, you can test that, and only let your own IP through.

Here’s the code for your .htaccess file:

Options +FollowSymLinks
RewriteEngine On
RewriteBase /
RewriteCond %{REMOTE_ADDR} !^23\.23\.23\.23
RewriteCond %{REQUEST_URI} !^/503.php [NC]
RewriteRule .* /503.php [L]

Just replace those numbers on the fourth line with your own IP. (Keep the backslashes before each dot!)

Those two RewriteCond (“rewrite condition”) lines basically mean: only apply the following rewrite if the IP of the request (REMOTE_ADDR) doesn’t match the one given, and the file requested (REQUEST_URI) isn’t 503.php (preventing an infinite loop!). As long as it’s not you and they’re not already going to 503.php, the RewriteRule kicks in and serves that page up. (Though note that the URL in the browser address bar doesn’t change for the user; this is rewriting, not redirecting.)

Note that last [L] in the code. That means, if this rewrite rule is processed, make it the last one. This is handy if you have other rewrites (e.g. your WordPress rewrites) that still need to work when you access the site, but which shouldn’t intefere with this rule for everyone else. Of course, for this reason, the above code should go right at the top of your file.

The switch

You may have your own technique. I keep 503.php and 503.htaccess in the web root. When I need to switch my holding page in, I just rename my usual .htaccess to something like LIVE.htaccess, and (quickly!) rename 503.htaccess to .htaccess.

All should now be on hold for the rest of the world. Bots should be politely leaving your site be for the while. And you should be able to get in and muck around to your heart’s content.

Looking through other’s eyes

A pesky stumbling block I came up against developing this technique was my obvious inability to see what other people were seeing via all those other IP addresses. My friend Jim, ever abreast of tech developments, directed me to the Torpark browser. Built on the Firefox shell, this browser is designed to anonymize your web surfing by routing your requests through a labyrinthine series of IP relays (or whatever they might call them).

It’s sluggish, but it works. Whatever your IP ends up being seen as by servers while browsing with this nifty tool, it won’t be your actual IP.

Upgrading WordPress

So, with these tricks ready to go, here’s my revised WordPress upgrade guide. (Do also check the official guide if you’re new to this though!)

One thing to bear in mind with more recent WordPress versions is that if the .htaccess file doesn’t contain WordPress’ rewrite rules at the end, it may try to “fix” things – and break them. Make sure your 503 .htaccess file has your usual WordPress rewrite rules at the end.

1. Backup all your files and your database.
2. Make sure you’ve a record of which files you’ve customized.
3. Do the 503 holding page switch.
4. De-activate all your plugins.
5. Delete all files apart from wp-config.php, the wp-content directory, and of course .htaccess and 503.php
6. Upload all files in the new version of WordPress (excluding the wp-content directory)
7. Run the upgrade script (e.g. http://yourdomain.com/wp-admin/upgrade.php)
8. Re-activate the plugins. Pray.
9. Check the site out, make sure it’s survived OK.
10. Revert to the original .htaccess file.
11. Done!

Update

I’ve just tried implementing this technique to display a holding page for a site that’s yet to be launched. Pretty much the same situation, but an interesting issue came up when I tried to include an image on the holding page. The image wouldn’t appear. I spent a frustrating 5 minutes checking paths and files, but eventually it dawned on me: the browser’s request for the image file was returning a 503 error, not the image!

A little extra .htaccess magic remedies this. Insert the following line before your RewriteRule line:

RewriteCond %{REQUEST_FILENAME} !\.(gif|jpe?g|png)$

Of course you may need to add css or js to that list of filetypes, depending on your holding page’s needs.

Anyway, for new installs it seems like the way to go, with a few caveats. One involves the potent but potentially tricksy .htaccess file for Apache server configuration settings. If, like me, you prefer to maintain control over the file yourself instead of having WordPress automatically generate it, you might run into problems when 2.1 tries to “fix” it.

If you suddenly get a HTTP 500 error (Internal Server Error), it’s often due to slip-ups in .htaccess. If you’re using WordPress 2.1 and get this, download .htaccess and see if WP hasn’t slipped its mod_rewrite rules in at the end. If you’ve adapted the default mod_rewrite code, WP 2.1 thinks it’s not there at all, and slips the default in at the end again.

I’ve not found a way to stop WP doing this – there was a suggestion to change the write permissions on the file to stop WP accessing it, but you’d have to switch them back every time you re-uploaded the file. Pain in the arse. I’ve just reverted to keeping the default WP mod_rewrite at the end of the file, exactly as WP intends.