CEP4 in production

Image may be NSFW.
Clik here to view.© RO, RUG CIT

The RO is proud to announce our newest High Performance Computercluster: CEP4. It is used for the recording and offline processing of LOFAR data that is generated by the COBALT correlator. Production commissioning started last summer and in December we were ready for production. The project started halfway 2015 with the tender and the time thereafter was taken by installing the system according to the specs, designing, building and testing the software. ASTRON and CIT Teams of network and systems administrators, software engineers, telescope scientists and other specialists have worked hard to bring this to a successful conclusion.

The time spent was significantly more than estimated at the start of this project because a number of unforeseen major hurdles had to be overcome. All critical issues have been addressed and the result is a technically up-to-date “workhorse” that will process LOFAR data for the coming 5 years. We will continue to explore and harvest the new opportunities this new cluster is able to offer to LOFAR and its community (e.g. unlocking the power of the GPU nodes).

Specs for the tech-enthusiasts:

Nodes: 1 management, 2 heads, 2 filesystem meta-data, 18 storage, 4 GPU compute and 50 regular CPU compute.

Tflops (theoretical): 96 from CPUs, 68 from GPUs, Total 164 (CEP2 was 20)

Filesystem: 3.5 PB

New techniques and frameworks that have been incorporated in the LOFAR system as part of the migration to CEP4 are a.o.: Lustre (cluster file system), Docker (containerized applications), SLURM (batch scheduling), Qpid (message broker infrastructure), Ganglia (scalable distributed monitoring system), Spacewalk (systems management), Robin Hood (policy engine and reporting tool for large file systems) and a new standard for an OS: CentOS 7.