Filesystems¶
Usage of the filesystems is subject to the Disk Usage Policy.
/home
¶
Shared resource by all users of Boreal.
313 TB system.
Provides storage for source code, submission scripts, small input files and possibly executables.
/dtemp
¶
Each project has a
/dtemp/<project_id>
directory for storing project-specific files.Global 2.8 PB Lustre file system visible to all the compute nodes and the login nodes.
Users should direct program output to
/dtemp
and not/home
./dtemp
is the preferred file system to store input, output and executables./dtemp
has more space, higher bandwidth and lower latency than the/home
file system./dtemp
is for temporary storage it is not backed up. Files older than 60 days in the/dtemp
file system are subject to deletion without notification.
/archive
¶
/archive
is a 22PB disk/tape library shared by several PNNL systems.Boreal users have a
/archive/<login_id>
directory for storing input and output files and submission scripts.The way users store files can influence retrieval time (on the order of weeks compared to days).
EFFICIENT STORAGE: small numbers of large files is the most efficient way of storing files on
/archive
.INEFFICIENT STORAGE: large numbers of small files is the least efficient method. It can takes weeks to retrieve these files.
To store a large number of files, first
tar czf <name>.tar.gz <dir>
the files, then store the large file archive./archive
is intended for output files you want to save and large input files you intend to reuse.
For more information on /archive
, aka Aurora, see Data File Storage (Aurora).
/scratch
¶
Each Boreal node has a 246 GB local disk mounted as
/scratch
./scratch
is used to store intermediate results, input files, and executables to reduce access time.The
/scratch
file system has the lowest latency, highest per node bandwidth of all the file systems./scratch
is volatile memory:files on
/scratch
will be deleted after the job terminates;to preserve files, move them from
/scratch
to/dtemp
in the submission script (do not store them on/home
);do not store files you want to save (e.g. checkpoint files) on
/scratch
because if the job terminates before the copy completes, the data will be lost.
Please direct output files to the
/dtemp
file system.
/emslfs
¶
Each Boreal login and data mover node has the /emslfs filesystem mounted.
/emslfs is a shared filesystem that is pervasively mounted on EMSL systems. It is a convenient way to move data between systems. It should never be used for input or output of jobs.
/emslfs is mounted as a convenience for users who already have data there. Directories are not created for you automatically. Contact MSC Consulting if you need directories created.