Disk Usage Policy¶
Important
The processing, storage, or transmittal of sensitive data [e.g. Personally Identifiable Information (PII), Official Use Only (OUO)] is prohibited on the Boreal Information System. Due diligence must be used to prevent inadvertent disclosure of invention, patent, or other sensitive information. It is your responsibility to protect access to the information.
/home¶
Directory for storage of data and executables while users have an active EMSL project.
/home sub-directories are available for each individual user. When a user no longer has
active EMSL projects, the data will be deleted and the disk space reclaimed 90 days after
the end of the last project on which a user is a participant.
Jobs running from the /home file system can cause system degradation either
because too many files are being accessed simultaneously or by exceeding disk
storage capacity.
Important
Do not submit jobs from /home. Jobs must be run from /dtemp or /scratch.
/home is a shared resource without enforced quotas. When excessive amounts
of disk space is used, those who are using the most space will be asked to
reduce their use. Repeated excessive use of disk space can lead to loss of
system privileges. This directory is not a permanent storage system. Files to
be saved should be stored on your local machine or in /archive.
/dtemp¶
A staging area for launching jobs, /dtemp is for temporary storage of executables
and data. Each EMSL project has a /dtemp sub-directory available to the users who
are project participants.
While an EMSL project is active, users are responsible for managing data in their project
/dtemp sub-directory. While a project is active, users are requested to maintain their
/dtemp project sub-directory data storage below 1.0TB. The project sub-directories will be
reclaimed and all data deleted 90 days after the end of the project period.
For long-term storage of important data, users should move files from /dtemp to /archive.
Files should be compressed with tar and gzip prior to storage on /archive.
Important
Jobs must be run from /dtemp or /scratch. Do not submit jobs from /home.
/archive¶
The Aurora archive is for long-term archiving of data. Each EMSL project has a /archive
sub-directory available to the users who are project participants. Storing data in personal
sub-directories outside of the project sub-directories is discouraged. Data should remain with
the project where it is easily identifiable and accessible by other project participants.
Important
Do not store large numbers of small files on /archive. Instead, compress datasets
into fewer, larger files using TAR and GZIP, then move the .tgz file(s) into /archive
and delete the source data. TAR/GZIP Example:
tar -czvf filename.tgz /sub-directory/to/archive
Important
Single files larger than 2TB cannot be copied to /archive directly. Instead, use HSI
to transfer these large files. Instructions for using HSI can be found at:
https://www.emsl.pnnl.gov/MSC/UserGuide/compute_resources/aurora.html#transferring-files-with-hsi
/scratch¶
/scratch is 246GB of local space per node of volatile storage. It is
intended for storage of programs and data while a job is running. After
termination, the entirety of /scratch will be deleted and all data will be
unrecoverable.
Important
Jobs must be run from /dtemp or /scratch. Do not submit jobs from /home.