Usage of the filesystems is subject to the Disk Usage Policy.


  • Shared resource by all users of Boreal.

  • 313 TB system.

  • Provides storage for source code, submission scripts, small input files and possibly executables.


  • Each project has a /dtemp/<project_id> directory for storing project-specific files.

  • Global 2.8 PB Lustre file system visible to all the compute nodes and the login nodes.

  • Users should direct program output to /dtemp and not /home. /dtemp is the preferred file system to store input, output and executables.

  • /dtemp has more space, higher bandwidth and lower latency than the /home file system.

  • /dtemp is for temporary storage it is not backed up. Files older than 60 days in the /dtemp file system are subject to deletion without notification.


  • /archive is a 22PB disk/tape library shared by several PNNL systems.

  • Boreal users have a /archive/<login_id> directory for storing input and output files and submission scripts.

  • The way users store files can influence retrieval time (on the order of weeks compared to days).

  • EFFICIENT STORAGE: small numbers of large files is the most efficient way of storing files on /archive.

  • INEFFICIENT STORAGE: large numbers of small files is the least efficient method. It can takes weeks to retrieve these files.

  • To store a large number of files, first tar czf <name>.tar.gz <dir> the files, then store the large file archive.

  • /archive is intended for output files you want to save and large input files you intend to reuse.

For more information on /archive, aka Aurora, see Data File Storage (Aurora).


  • Each Boreal node has a 246 GB local disk mounted as /scratch.

  • /scratch is used to store intermediate results, input files, and executables to reduce access time.

  • The /scratch file system has the lowest latency, highest per node bandwidth of all the file systems.

  • /scratch is volatile memory:
    • files on /scratch will be deleted after the job terminates;

    • to preserve files, move them from /scratch to /dtemp in the submission script (do not store them on /home);

    • do not store files you want to save (e.g. checkpoint files) on /scratch because if the job terminates before the copy completes, the data will be lost.

  • Please direct output files to the /dtemp file system.


  • Each Boreal login and data mover node has the /emslfs filesystem mounted.

  • /emslfs is a shared filesystem that is pervasively mounted on EMSL systems. It is a convenient way to move data between systems. It should never be used for input or output of jobs.

  • /emslfs is mounted as a convenience for users who already have data there. Directories are not created for you automatically. Contact MSC Consulting if you need directories created.