#######
Mirrors
#######

The ACM mirrors a lot of material from elsewhere in the world.  So much so
that we have a common automation host for keeping our mirrors up-to-date,
at the moment running on ``astrolabe.vm.acm.jhu.edu`` . (This is the same
machine currently serving mirrors over HTTP).

There is not a whole lot to say here except to point at the configuration
files, which are, conveniently, in AFS and public:
`<file:///afs/acm.jhu.edu/service/mirrors>`_ .

In addition to the Debian ``archvsync`` tool, which provides the ``ftpsync``
tool used for mirroring Debian-alike repositories, we have locally-written
tools ``plain-rsync`` and ``plain-git`` for maintaining mirrors of other
varieties of upstreams.  All of these utilities consult files in
`<file:///afs/acm.jhu.edu/service/mirrors/etc>`_ for details of their
execution; no configuraiton, other than the mirror name, is passed on their
command lines.

The ``mirrors-updater`` host has a user (``mirror``) whose crontab oversees
the actual invocation of these scripts.

Mirrors Updater
===============
The mirrors updater will occasionally get a corrupted filesystem. Now, there's an
easy way to fix this. Run ``/afs/acm.jhu.edu/groups/admins.pub/scripts/mirrors/destroy_mirror_updater_vm.py``
to destroy the old (and broken) vm. Then run ``make_mirror_updater_vm.py`` with an unallocated floating ip,
and the ID of the latest mirror_updater snapshot (look at ``cinder snapshot-list``). You'll need to follow
the instructions to finish setting it up (just run a command with your admin hat on to land a keytab on the
machine, and populate crontab).

Adding a Mirror
===============

To add a mirror, then, you should do the following steps:

* Run ``/afs/acm.jhu.edu/group/admins.pub/scripts/new-afs-mirror-volume MIRROR`` ,
  where ``MIRROR`` is the name of the thing you want to mirror.
* Make the appropriate entry in ``etc/`` and adjust the crontab on the mirrors
  updater (i.e. mirrors-updater; look it up by its floating ip, it won't have a name).

If your mirror uses, e.g. rsync or ftp, you are more or less done.

Git Mirrors
-----------

Mirrors using ``git`` are somewhat more interesting. For an example of a working
git mirror, the ACM mirrors a git repository containing the plan9 source tree
that can be cloned using the following command::

  git clone http://mirrors.acm.jhu.edu/plan9/

To actually set this mirror volume up properly, after running the new volume
script, you must run the following git commands (where, for plan9, $SOURCE is
``https://github.com/0intro/plan9``)::

  git clone --mirror $SOURCE
  git update-server-info

This will perform the first sync.

Nagios Monitoring
=================

We use nagios to monitor mirror statuses. This lets us stay alert when a mirror
sync fails because e.g. the person we are mirroring from has gone out of date.
Generally, there are two types of mirrors that we monitor: "fast" mirrors, which
are expected to sync at least daily and are checked every two days (in case of
transient failures, which caused significant problems), and weekly mirrors, which
are checked weekly.

To add a new mirror service to nagios, add the following stanza to one of nagios's
configuration files (i.e. ``trinidad-mirrors.cfg``)::

  define service {
          use generic-service
          host_name astrolabe.vm
          service_description Mirror CTAN
          _FILE /afs/acm.jhu.edu/service/mirrors/log/rsync-ctan.log
          check_command file_age_fast_mirror
  }

In this case, the example stanza is provided for our CTAN mirror, which is "fast".
For weekly mirrors, simply change the check command to ``file_age_weekly_mirror``.