Регистрация / Вход
Прислать материал

Data Management For Exascale Experiments

Name
Mario
Surname
Lassnig
Scientific organization
European Organization for Nuclear Research (CERN)
Academic degree
Doctor of Natural Sciences
Position
Staff Scientist
Scientific discipline
Information technologies
Topic
Data Management For Exascale Experiments
Abstract
The High-Energy Physics Experiment ATLAS has processed more than an Exabyte of data during its operation, and is growing continuously. In this talk, I present the solutions we developed for the data management system that serves as the foundation for ATLAS. First, I give an overview of the global requirements. Then I present selected highlights, specifically (a) how we dynamically schedule competing requests over constrained storage and network, (b) how we deal with with data loss and recovery, and (c) how we dynamically adapt our data placement based on the analysis of system usage.
Keywords
data management, distributed systems, databases, networks, analytics
Summary

Part 1 - Introduction

  •     The ATLAS Experiment
  •     Workflow and Data Management

Part 2 - Data Management

  •     Requirements from workflow management
  •     Declarative data management with rules and subscriptions
  •     Global infrastructure and system operations

Part 3 - Highlights

  •     Request scheduling for transfer and deletion
  •     Data loss and recovery
  •     Smart placement using data access patterns
  •     System analytics: How we keep track of everything

Part 4 - Outlook and Summary

  •     Data management evolution: The challenge of serving single events
  •     Conclusions