SC2006 Lustre Scalability Research
EMSL Project ID
20909
Abstract
As seen in the past few years parallel storage subsystems have become larger, requiring tens to hundreds of storage servers, and high-speed networks. There is a growing need for these systems to scale to hundreds of physical disks, and have Input and Output rates in multi-terabytes per second. These new systems will require scalability in the ten thousand to hundred thousand client count range.We propose to re-task the local I/O subsystems on the HPCS2 cluster currently housed in the Environmental Molecular Sciences Laboratory at Pacific Northwest National Laboratory in Richland, Washington. Currently, the cluster has 978 Dual processor HP rx2600 Itanium2 systems, 570 of which have a .5 TB raid file system. We plan to re-configure these systems to become Lustre Object Storage Server nodes and serve the file system to the remaining nodes acting as clients for the I/O tests. There are three ways we plan to test this configuration:
1.Raw disks - we will use the 7*73GB disks in each system to present 3390 physical Object Storage Targets to the clients.
2.Raid 0 - Will we present a single OST from each OSS in a high-performance, low-redundancy configuration.
3.Raid 5 - We will present a single OST from each OSS in a raid-redundant manner, and a lower-performance I/O system.
These three methods will allow us to evaluate the large number of OST scalability, and evaluate the speed of our architecture.
We believe we can achieve in these tests I/O write rates in excess of 100GB/s to a single file and scale Lustre to thousands of OST targets in an efficient manner.
Project Details
Project type
Limited Scope
Start Date
2006-08-04
End Date
2006-09-25
Status
Closed
Released Data Link
Team
Principal Investigator