.. This document is reStructuredText. If you are having a hard time reading the source, you may prefer to read the rendered version available on the web at https://www.acm.jhu.edu/~admins.pub/systems/users/acmer.html . ############################# Navigating the ACM's Services ############################# Welcome to the ACM and congratulations on your new system account. .. note:: You should feel free to delete this document from your home directory when you feel sufficiently comfortable with the contents. It can always be found at ``/afs/acm.jhu.edu/readonly/group/admins.pub/users/acmer.rst``. The ACM has a wide variety of technology set up for our (and that means your, too!) use. Among these, physically, are: * A data center full of reasonably powerful machines with large amounts of storage (more than 192 raw TB of disk!). * Desktops in the office, offering pre-configured integration into our computing environment. * A lot of embedded widgets for playing with hardware at a very low level. * :ref:`Our door controller` is a custom-made piece of kit, even. * A retrocomputing history museum! (http://www.acm.jhu.edu/~acm-museum) But at the moment you're probably interested in the software side of things. Software-wise, we have a truly large number of goodies * A globally-available network file system, called "OpenAFS". * An OpenStack setup for running VMs in the data center. * Quassel for detachable IRC sessions. * Our own email server, including mailing lists. * Desktops and shell servers are available for remote shells. First And Foremost ################## All of this is possible through the efforts of our system administrators and Volunteers Like You! To get someone's attention, try one of these approaches: * Use IRC: irc://freenode.net/#jhuacm * Use the admins mailing list: admins@acm.jhu.edu * File a bug on our bug tracker: https://ebola.acm.jhu.edu OK, OK. Tell Me About The Goodies ################################## Storage ======= Each user has: * A 20G home volume * A 5G mail volume * A 1TB "scratch" volume The former two will be guarded with extreme care and backed up regularly, including to archival storage. The "scratch" volume, on the other hand, will recieve a lesser degree of attention. As a general rule, while we will not actively expunge the contents of the scratch volume, we will not work to salvage it from catastrophe, and we may not attempt to deliver the same standards of availability as the main portions of your home directory. (If you find that you absolutely need more highly-reliable space, please come talk to a sysadmin. There may need to be some financial arrangement worked out, as the sizes above were chosen so that we could make the assurances of backups to all users.) .. note:: In the event of a catastrophic failure, the last snapshot of your files should remain available under ``/afs/acm.jhu.edu/readonly/user/YOU``. Now, that said, if you look around in your home directory, you will see that it is quite spartan, at the moment. There are, however, several interesting things laying about: * ~/acmsys -- all of the "ACM system" magic lives here * ~/acmsys/OldFiles -- this is a read-only copy of your home directory as of last night. (This does not count against your quota.) * ~/acmsys/Maildir -- your mail (eventually) * ~/acmsys/OldFiles.Maildir -- your mail, as of last night. * ~/acmsys/scratch -- your scratch volume. * ~/acmsys/OldFiles.scratch -- your scratch volume, again as of last night. * ~/acmsys/public_html -- not a special volume, but the directory wherefrom we will serve files on the web. See :ref:`readme-user_webhosting` below. .. note:: If you're joining us in the Brave New World but have come from the Scary Old World, you are probably very curious about your old home directory; keep reading or skip to :ref:`readme-user_old-homedirs`. .. note:: In addition to your private scratch volume, there is a system-wide scratch volume on a 10TB partition, mounted at ``/afs/acm.jhu.edu/scratch``. Because everyone has read&write access to this location, you should not store private data there. How does AFS work? ------------------ The technical details of AFS are not terribly germane to this conversation, but it is not quite like anything else you've ever used, so we have to talk a bit about it. This is, of course, only the most basic introduction; for details, consult the manual pages of the programs mentioned below (but especially ``fs`` and ``pts``) or just run ``fs help`` or ``pts help``. One very key thing to keep in mind is that AFS is a global system -- your files are available from anywhere in the world, not just computers on JHU campus. Notably, things you share publicly in AFS are truly public for the whole world to see (or change, if you allow them)! (You can, of course, restrict things to just properly-authenticated ACM users.) Quota ----- With us having just spent all that time talking about how much storage space you're given, you're probably curious to know how much you're using! To find out, use the ``fs listquota`` (or just ``fs lq``) command, as in :: fs lq ~ ~/acmsys/Maildir ~/acmsys/scratch -human which will show you something like :: Volume Name Quota Used %Used Partition user.YOU 20.0G 2.0K 0% 0% mail.YOU 5.0G 2.0K 0% 0% scr.YOU 1.0T 37.7G 4% 0% You can probably ignore almost everything except the %Used column. If you leave off the -human option, ``fs lq`` will show you the values in 1K blocks, which is more accurate but less immediately friendly. (Of course, you can run ``fs lq`` on other paths in AFS; for example ``fs lq /afs/acm.jhu.edu/mirror/* -human``, will tell you how much space our mirroring effort takes up, and what the various quotas on those mirrors are.) .. _readme-user_permissions: Permissions ----------- AFS does not pay (much) attention to the traditional UNIX ownership and permission bits (the things that ``ls -l`` will show you). Of interest to users, the 'w' bit is considered advisory (AFS may try to stop writes to a non-w file), and 'x' retains its traditional meaning. Instead, AFS has a richer vocabulary of permissions on a per-directory basis. To see an example, you might run :: fs listacl $HOME which will produce something like :: YOU rlidwka system:services l system:anyuser l which says that YOU have full permissions, and that the groups ``system:services`` and ``system:anyuser`` (that's everybody, even people on The Internet At Large) can list the contents of this directory. ``rlidwka`` stands for ============ ============================================ Read read the contents of files in this directory List display the contents of this directory Insert create new files in this directory Delete remove files from this directory Write mutate the contents of files in this directory locK exclude other clients from writing to files Administrate change ACL entries ============ ============================================ To change permissions, use the ``fs sa`` command. You can *always* make further changes (because you own your home directory, which is the top-level directory on its volume) even if you attempt to revoke your own right to do so (by mistake, one assumes). Whenever a directory is created, it inherits the permissions of its containing directory, but thereafter can be changed independently. To make a directory world-readable, use the special incantation :: fs sa $DIRECTORY system:anyuser rl If you want to refer to JHU ACM members, replace ``system:anyuser`` with ``system:authuser``. If you ever want to remove a user or group from an ACL entirely, give ``fs sa`` the empty string as an argument, e.g. :: fs sa $DIRECTORY system:anyuser "" To make changes recursively (if you've already filled out the contents of a directory and want to change all of their ACLs, you'll want the really-mysterious incantation (none of the punctuation is a typo) of :: find $TOPDIR -type d -exec fs sa {} system:anyuser rl ';' Of course, you can vary the ``system:anyuser rl`` part of that command to taste, and should replace ``$TOPDIR`` with the directory you want to recursively descend. If ever a directory's ACLs are so far different from what you desire that you'd like to just start over, ``fs sa`` can be given a ``-clear`` option which erases all permissions on the target directory. Special Permissions in Your Home Directory """""""""""""""""""""""""""""""""""""""""" Some ACM services need permissions to function: * Your home directory and ``~/acmsys`` directory should retain at least ``system:services l`` permissions in order for its magic to work, in general. * ``~/acmsys/Maildir`` and sub-directories should have at least ``system:mta wlid`` for us to be able to deliver your mail to this directory. If you choose to forward mail elsewhere, this is less important. * ``~/acmsys/public_html`` (and possibly sub-directories) should have at least ``system:www rl`` for our web servers to be able to serve them to the world. See :ref:`readme-user_webhosting` for additional details. Creating Groups --------------- One of the most compelling features of AFS is the ability for users to create their own groups and include whomever they like without requiring administrator attention. This is something that most other storage systems simply can't offer. This is done via the AFS ProTection Service, which is managed by the ``pts`` utility. To create a group, say ``pts creategroup $USER:example``. All groups you control are prefixed by your username and a ':'. To see a list of all groups you control, say ``pts listowned $USER``. Of course, having created a group doesn't mean anyone's in it! Members are added with ``pts add`` and removed with ``pts rem``, both of which (can) take a user name then a group name. So, for examples:: pts add nwf $USER:example pts add jay $USER:example pts rem jay $USER:example To see who is in a group, run ``pts members $GROUP``. This will produce something like :: Members of YOU:example [...] are: nwf jay And last, groups go in ACLs just like users:: fs sa ~/projects/cs201 YOU:example rlidw Note that for the above to be useful, at least the List ('l') permission must be available to YOU:example on ~ and ~/projects, as otherwise the members of this group cannot get to ~/projects/cs201 . (You probably don't want to pass out the Administration right to anyone who isn't you.) What Happens Late At Night? --------------------------- If you're on the ACM systems very late at night, you might notice your volumes temporarily stop responding (e.g., sometimes applications hang for a while). There is a very narrow window each day during which our systems snapshot your volumes, producing the OldFiles directories mentioned above. The time it takes to snapshot your volume is proportional to how much has changed since last time, but in general shouldn't take but a few minutes. It would be nice if AFS could do this seamlessly, but in the interim, stand up and stretch, go for a walk around the building, etc. If "dmesg" reports lots of "Waiting for busy volume" messages, this is almost assuredly what's going on. (Note that AFS achieves this backup on a whole-file basis: mutating a file that's been backed up causes the AFS server to make a whole-file copy of the file before allowing your change. This means that you probably do not want to store huge, dynamic files, like VM disk images or similar, in AFS. If you find yourself needing such things, we will be happy to arrange for you to have a new volume which does not get backed up automatically.) Mail ==== By default, mail to ``YOU@acm.jhu.edu`` ends up in ``~/acmsys/Maildir``. .. todo:: This section is not yet written. It should explain local MUAs, forwarding, and optional IMAP use. We have IMAP and authenticated SMTP available to point your MUA at, if you wish to use the mailbox we provide for you that way. IMAP is port 143 of ``imap.acm.jhu.edu`` with STARTTLS, or port 993 with TLS. SMTP is port 587 of ``smtp.acm.jhu.edu`` with STARTTLS. Use your ACM username and password. These services also accept GSSAPI (Kerberos ticket) authentication, if you prefer that. .. _readme-user_webhosting: Web Hosting =========== We will gladly serve files from your home directory on the web! Our webserver(s) look at ``~/acmsys/public_html`` for the root of ``https://www.acm.jhu.edu/~YOU``. You will need to ensure that ``system:www`` has ``rl`` permission on all directories from which you expect to serve files; this should be the default. You may wish to remove that right for directories containing data that should not be globally visible. See :ref:`readme-user_permissions` above for details. Additionally, despite AFS's disregard for UNIX permissions, web servers tend to care and manually validate permissions; you must ensure that all "other" would have "r" on files and "x" on directories you wish to serve. Again, this is probably the default depending on your environment. Should you want to run CGI scripts, let the admins know and we'll set up CGI (the scripts will go in ``~/acmsys/public_html/cgi``) for you (see :ref:`readme-user_webhosting_cgi` below). Kerberos Authentication ======================= Changing Your Password ---------------------- Use the ``kpasswd`` program; if it complains about a "bad address", that's a known bug -- it worked anyway (try!). Life without SSH Keys --------------------- SSH keys would get you a shell on the shell servers, but your session wouldn't have Kerberos tickets or the all-important AFS tokens, making it a rather useless shell. The easiest thing is to resign yourself to typing your ACM password whenever you log in; once you're signed in to one system, you should be able to move about from there without need of typing your password again. Failing that, you can set up Kerberos on your client machine and ``kinit YOU@ACM.JHU.EDU`` before logging in to the ACM. If you have the right knobs set, your tickets will come with you. If even that's not good enough, you can create a keytab file which can be used to manage your tickets and tokens without human intervention. Storing this in AFS would be unwise (AFS encryption is not, ah, robust); you should probably keep it only on your client machine. Long-running sessions --------------------- If you like, the ACM admins can set the "renewal lifetime" of your tickets arbitrarily high. (We prefer to keep short non-renewal lifetimes as good security hygene; the ``krenew`` program can keep your tickets and tokens refreshed within their renewal lifetime for when you actually do have a long-running session.) If even that's not good enough, you can create a keytab file which can be used to manage your tickets and tokens without human intervention. We would much rather give you a second principal along with that keytab; come talk to us. .. _readme-user_webhosting_cgi: CGI scripts ----------- .. todo:: (This is not yet set up) Periodic Tasks -------------- .. todo:: (This is not yet set up) A similar story holds for cron usage; if you'd like to automate some periodic task, we will again make a principal for you and will have a crond adopt that credential during execution of your task. You should give your alternate hat the minimal set of permissions it will need to complete its task. .. _readme-user_jhed: Using JHED Identities with the ACM Systems ========================================== .. note:: Possibly the simplest way to do this is to use :ref:`egg_for-users` which acts like a JHED-based UNIX system and automagically connects your JHED account to some ACM services. Keep reading if that isn't what you had in mind. You do not have to log in (or necessarily even be a member) to the ACM to use our services, but we still need to know something about you. One way for us to do this is to use JHU's JHED system: you log in to their systems, and they tell us that you're you. On any on-campus UNIX environment equipped with OpenAFS (which, notably, includes the CS department workstations in the ugrad and masters labs), it should suffice to run :: kinit ${your_jhed_id}@WIN.AD.JHU.EDU aklog acm.jhu.edu .. note:: If these commands error out for whatever reason, it may help to check that AFS is actually running on the machine you've logged into. We have had some "mysterious" failures that ultimately ended up being due to afsd not being plugged in, as it were. If ``ls /afs/acm.jhu.edu/`` succeeds, afsd is running; if not, try a different computer! .. note:: Note, too, that as you are likely running from a machine which is "foreign" to our AFS setup, you will have to add ``-cell acm.jhu.edu`` to all ``pts`` commands you issue. (``fs`` commands can be run as is.) If you are a member, you can grant your JHED identity (which will be ``${your_jhed_id}@win.ad.jhu.edu``, all lower case) rights on the system as you would any other user when you are logged in using your ACM credentials. You can allow SSH logins to ACM systems using your JHED identity by adding a line to your ``~/.k5login`` file containing ``${YOUR_JHED_ID}@WIN.AD.JHU.EDU`` [#readme-user_k5login]_. If you are not a member, someone else will have to grant you rights to the system. Ideally, if this is your first time here, you're following along because someone told you to and they are about to do the right thing. :) If you've been made a member and/or administrator of a group, you'll want to read :doc:`groups`. .. [#readme-user_k5login] Note that your ``~/.k5login`` file will have to be world-readable for this to work as the system daemons do not run with AFS credentials. If your home-directory is not world-readable but only world-listable, make ``~/.k5login`` a symbolic link into a world-readable sub-directory of your homedir, e.g. ``~/Public``. See :ref:`readme-user_permissions` above for details. Installing the OpenAFS client on Your Own Machine ================================================= You can install OpenAFS on your own systems and access your files from there. The ACM's systems already have these installed and are set up so that typing your password gets you access to your files. Linux, BSD, etc., systems ------------------------- Typically, your distribution will have packaged OpenAFS. On Debian, for example, you should install the following packages:: openafs-client openafs-krb5 krb5-user You will have to ``kinit ${your_user_name}@ACM.JHU.EDU`` and then ``aklog acm.jhu.edu`` to authenticate and grab AFS tokens for the ACM. Mac OS X systems ---------------- On Mac OS X, you can grab an installer from http://openafs.org. Windows ------- It is, in fact, possible to access AFS from Windows! (In fact, in a bit of an embarrassing situation, the Windows AFS cache manager is better than the UNIX ones.) `This guide `_ is a good starting point, more documentation may be added here as needed. Your default cell should be "acm.jhu.edu". Note that you do need to install Heimdal Kerberos for Windows, and that you will want to make sure you check the "AFS Authentication" feature in the AFS installer. If not, some testing suggests that while you'll be able to browse, you may not actually be able to authenticate. .. _readme-user_old-homedirs: Where Are All My Old Files? =========================== The contents of your old home directory have been landed at ~/acmsys/scratch/oldhome; please take the time to copy out what you desire, re-arrange your scratch volume, and delete any cruft you may have laying about. The ~/acmsys/scratch/oldhome directory itself is not special and may be removed once you are happy. Note that for various silly reasons, not all your files may have the right permissions. If that's the case, you might need to do some liberal adjustment.