Регистрация / Вход
Прислать материал

ATLAS Production System in heterogeneous computing environment

Name
Dmitry
Surname
Golubkov
Scientific organization
Big Data Laboratory, National Research Centre "Kurchatov Institute", Moscow, Russia
Academic degree
M.Sc.
Position
Researcher
Scientific discipline
Information technologies
Topic
ATLAS Production System in heterogeneous computing environment
Abstract
The new generation of the ATLAS production system is an automated workload manager, which used by thousands of physicists in CERN to process and analyze the distributed exabyte-scale data using the power of 150 computing centers, supercomputers and cloud academic resources. The system was developed in accordance with the requirements from ATLAS: high efficiency use of computing resources, automated load balancing and scalability. We present a description of the main elements of the system, their interactions and solutions used in the development of the system architecture.
Keywords
Big Data; Grid-based Simulation and Computing; Parallel and Distributed Computing; Large Scale Scientific Instruments
Summary

ATLAS Production System in heterogeneous computing environment

M Borodin1,2, D Golubkov1,3, A Klimentov1,4, R Mashinistov1

1Big Data Laboratory, National Research Centre "Kurchatov Institute", Moscow, Russia

2National Research Nuclear University “MEPhI”, Moscow, Russia

3Institute for High Energy Physics, Protvino, Russia

4Brookhaven National Laboratory, NY, U.S.A

The new generation of the ATLAS production system is an automated workload manager, which used by thousands of physicists in CERN to process and analyze the distributed exabyte-scale data using the power of 150 computing centers, supercomputers and cloud academic resources. The system was developed in accordance with the requirements from ATLAS scientific community: a flexible web user interface, fast adaptivity to new working processes, high efficiency use of computing resources, automated load balancing controlled by configurable policies for different types of data processing, scalability. It’s achieved through the use of a scalable multilevel system architecture. The high level interface for workflow management provides the abstraction of data and generates sets of input parameters - computing tasks based on user requests. These tasks are transferred to the executive low-level system to run in a heterogeneous computing environment. The task monitoring system provides continuous access to information about tasks and necessary task management methods. We present a description of the main elements of the system, their interactions and solutions used in the development of the system architecture.