Making your downloads mirrorable

This is a step-by-step guide that shows what I have done to make Ant's releases mirrorable. The process should be the same for other Jakarta and XML projects, but if I have missed something, please tell me (and maybe submit a patch to this page).

Prepare for the move

• Read http://www.apache.org/dev/mirrors.html.
• Read http://www.apache.org/dev/mirrors.html again ;-).
• SSH to minotaur.
• Make sure your umask is set to 002.
• Create directories /www/www.apache.org/dist/jakarta/your-project as well as /www/www.apache.org/dist/jakarta/your-project/source and /www/www.apache.org/dist/jakarta/your-project/binaries. Maybe you need platform specific subdirectories under the binaries directory as well.
• Make sure the directories you've just created are owned by goup jakarta and are group-writable.
• Copy your release downloads to the "correct" directories. Don't touch your nightly builds.
• Create symbolic links in /www/www.apache.org/dist/jakarta/your-project as described in http://www.apache.org/dev/mirrors.html
• Make sure all files you have just copied have the correct group ownership and permissions.

Going live on www.apache.org

• Remove all lines matching your-project from /www/www.apache.org/dist/jakarta/.htaccess. There may be more than one.
• Visit http://www.apache.org/dist/jakarta/your-project/ as well as the subdirectories to ensure everything looks the way you want it to be.
• Change all download links you have control over to point to the new locations.
• Create a .htaccess file inside the "old" download directory for your-project. The contents are too dependent on your former release directory layout to give hard rules here. As an example, Ant's 1.5.1 release directory was /builds/jakarta-ant/release/v1.5.1 and had subdirectories src, bin and rpms. The .htaccess in /www/jakarta.apache.org/builds/jakarta-ant/release/ now looks like this:

  Redirect Permanent /builds/jakarta-ant/release/KEYS http://www.apache.org/dist/ant/KEYS
  Redirect Permanent /builds/ant/release/KEYS http://www.apache.org/dist/ant/KEYS
  Redirect /builds/jakarta-ant/release/v1.5.1/src http://www.apache.org/dist/ant/source
  Redirect /builds/jakarta-ant/release/v1.5.1/bin http://www.apache.org/dist/ant/binaries
  Redirect /builds/jakarta-ant/release/v1.5.1/rpms http://www.apache.org/dist/ant/binaries/rpms
  Redirect /builds/jakarta-ant/release/v1.5.1 http://www.apache.org/dist/ant
  Redirect /builds/ant/release/v1.5.1/src http://www.apache.org/dist/ant/source
  Redirect /builds/ant/release/v1.5.1/bin http://www.apache.org/dist/ant/binaries
  Redirect /builds/ant/release/v1.5.1/rpms http://www.apache.org/dist/ant/binaries/rpms
  Redirect /builds/ant/release/v1.5.1 http://www.apache.org/dist/ant
  

• Visit your old download URLs to ensure you get redirected properly.
• Tell Pier (pier at apache dot org) about the change so that he can adjust his rsync script for nagoya.
• Create a patch to log-analyzer-daily.pl and send it to Vadim Gritsenko (vgritsenko at apache dot org) so that your project doesn't get dropped from his statistics pages http://www.apache.org/~vgritsenko/stats/daily.html.
• Wait a few days to make sure mirrors have time to pick up your files.

Send your users to the mirrors

If your-project is a Jakarta project and has no download pages of its own, all you need to do is to add a link to the "Using a mirror" list under "Release" on /xdocs/site/binindex.xml and /xdocs/site/sourceindex.xml in the jakarta-site2 module, regenerate the HTML pages and commit everything. For the links you'd simply replace http://www.apache.org/dist with [preferred] in your download URL.

If you want something more sophisticated (or simply have download pages of your own or are not under the Jakarta umbrella), read on.

The old way to send people to the mirrors has been a link to http://www.apache.org/dyn/closer.cgivirtual-path where virtual-path would be replaced by your download URL minus http://www.apache.org/dist. This still works but has the drawbacks that you don't have control over the layout of that page and that users may use the signatures found on the mirrors to verify the releases instead of picking them up from www.apache.org.

To solve the layout problem, Justin Erenkrantz (and others?) have written http://www.apache.org/dyn/mirrors/mirrors.cgi which parses a template and replaces certain [expression]s with something useful. To write and use a template of your own, you currently need to do three things.

1. Write your template. It should be straight forward to do so by looking at examples like httpd's, Ant's or Jakarta's pages. These all use Anakia, but it should be possible to integrate it with different technologies as well. For example, tutorial on creating such templates with Forrest.
2. For each template, add a CGI script (with extension .cgi) to your site. This script must look like this:

   #!/bin/sh
   # Wrapper script around mirrors.cgi script
   # (we must change to that directory in order for python to pick up the
   #  python includes correctly)
   cd /www/www.apache.org/dyn/mirrors
   /www/www.apache.org/dyn/mirrors/mirrors.cgi $*
   

   Well, you could probably trim the comments. Ensure that the checked out file on minotaur is executable and has Unix line-ends. Subversion in general will not translate line endings for you unless you tell it).
3. Tell mirrors.cgi where to find your template. To do so, you need to add a line to /docs/dyn/mirrors/mirrors.conf in the site module (in https://svn.apache.org/repos/asf/infrastructure/site/trunk/) This file is under SVN control, don't edit the checked out copy on minotaur directly. If there is no section for your virtual host, add it. Next add the virtual path to your CGI script from above (replacing / with -) as key and the full physical path of your template as value. You will need commit access to the site module to do this (all ASF members have) - if you don't have karma for it, send a patch to infrastructure at apache.

After updating both /www/www.apache.org/dyn/mirrors/mirrors.conf and your site, your new download page should show up, when you access your CGI script.

How to deal with old releases

Old releases should be copied to http://archive.apache.org/dist/ - simply follow the same directory layout that your distribution under www.apache.org uses. You find the site in /www/archive.apache.org on minotaur.

Once you start distributing via www.apache.org you can forget carying for your old releases as archive.apache.org will automatically archive everything that has been on www.apache.org.