How do I use the File system?

Important notes:

Each user has three base directories which can be accessed either by their name or by the their environmental variable:

Directory Environmental variable
Your home directory $HOME
$HOME/bins $BINS
$HOME/scratch $SCRATCH

In addition a group may have a $SETS directory which is designed for semipermanent data sets that will be used repeatedly by the group. $SETS can contain things like equations of state or velocity fields. It may also contain programs used by multiple members of a group. $SETS will be readable on the compute nodes. Not all groups have $SETS directories.

$HOME - Should be kept very small, having only start up scripts and other simple scripts. Output from parallel jobs can not be directed to $HOME. It should only be read from compute nodes.

$BINS - Should contain programs users have built for personal use and small data sets and run scripts. Output from parallel jobs can not be directed to $DATA It will be read only from compute nodes.

$SCRATCH - The main area for running applications. Output from parallel runs should be done to this directory.

FILE SYSTEM QUOTAS

Machine $SCRATCH $HOME + $BINS (Combined Total)
Aun/Mc22,000,000 Files 20 GBs
Mio 2,000,000 Files 20 GBs

Note: most unix style file systems will see a performance decrease as the number of files per directory increases, this will be noticeable as the number of files per directory gets into the hundreds. This will cause a performance hit for all users when a user access files in a directory that contains a large number of files. Please keep the number of files per directory reasonable.

The organizational structure of the file system is the same on Mio, AuN and Mc2; however, Mio has its own file system while AuN and Mc2 actually share the same file system. Also, from BlueM it is possible to see the AuN/Mc2 file system and the Mio file system. Technically we say that BlueM mounts the AuN/Mc2 file system and it mounts the Mio file system.

GETTING AROUND THE VARIOUS FILE SYSTEMS

When you first login to AuN or Mc2 you will see that you have the directories:

On AuN:
bins scratch mc2
On Mc2:
bins scratch aun

The Mc2 directory on AuN is a link to your home directory on Mc2 and the AuN directory on Mc2 is the reverse.

Scratch is shared directly across AuN and Mc2. This is where runs should be done, not in your home directory. The bins directory is distinct on the two machines. Files created in bins on Mc2 are not in bins on AuN. The bins directory is where you should store applications that you build.

When you log in to BlueM you will see a directory remote that contains:

remote/aun/:
bins home scratch
remote/mc2/:
bins home scratch

And possibly

remote/mio/:
bins home scratch

These "remote" directories have links to bins, home and scratch on the given machine. Thus, to copy a file from your desktop machine to AuN you only need to copy it to remote/aun on BlueM. The same holds for remote directories for Mc2 and Mio.

If you have an account on Mio the remote directory on BlueM will contain subdirectories for Mio. Thus it is possible to move files among Mio, AuN and Mc2 by doing a "cp".

On Mio, by default, you have only bins and scratch directories. There is no remote directory.