Name: WP2 fortnightly meeting
Start: 2021-03-24T11:00:00+01:00
End: 2021-03-24T12:30:00+01:00
Location: No location set

- 11:00 → 11:10
  News 10m
  
  Orateurs: Rosie Bolton (Square Kilometre Array Organisation), Xavier Espinal (CERN)
  Activities
  
  EOSC-Future update. "last" consortium meeting on Monday, revised proposal to be sent today/tomorrow.
  
  During the last month we gathered information from experiments and partner sites to know individual goals and objectives. The roadmap is quite clear and the commonality of interests is big. Strong synergies across the experiments and partner sites.
  
  Program of Work for ESCAPE 2nd phase second phase almost finished. Focused and very challenging. To be cristalysed on a full scale exercise by end November, the codename is DAC21 (Data and Analysis Challenge)
  
  WP2/WP5 joint workshop: 6th and 7tf of April 13:00 to 17:00
  
  Commercial clouds, ephemeral resources, integration work ongoing: AWS and Google. Frederic from LAPP leading the activity, some preliminary results (presentation in two weeks)
  
  Data Lake and HPC: communication note is out [link] (credits to Tomasso, Diego, Lucia and all the INFN team)
  
  Full Dress Rehearsal/WP2 workshop communication note: [link]
  
  Synergies with CS3MESH4EOSC and LOFAR. Possibility to leverage the Data Lake infrastructure with end-user oriented and/or educational purposes through the Analysis Platforms and Notebooks Infrastructure.
  
  Preparation of a mini-workshop between Experiments and RUCIO, specially focused on the demands from not communities with less experience on data management, might have special needs, i.e.. metadata extension. Poll to identify possible dates to be circulated soon, Alba in charge.
  
  Two vCHEP contributions from CERN. Propose to add "thanks to ESCAPE GA" reference add the end of the paper and "on behalf of WP2" next to the main author(s).
- 11:10 → 11:50
  DIOS Second Phase, towards DAC21 40m
  
  Orateurs: Andrea Ceccanti (INFN), Paul Millar (DESY), Dr Riccardo Di Maria (CERN), Rosie Bolton (Square Kilometre Array Organisation), Xavier Espinal (CERN), Yan Grange (ASTRON, the Netherlands Institute for Radio Astronomy)
  
  Proposal DAC21
  T2.1 Plans for DAC21
  
  ----------------------------------------------------------------------------------------------------------------------------------------------
  
  Proposal to re-gather in the Data Injector Demonstrators forum with focus on:
  
  Experiments needs and interests (always open to sciences/partners proposals)
  
  Data registration and/or buffering injection
  
  Data life-cycle aka QoS (cross-task activity with T2.2)
  
  Deterministic vs. non-deterministic RSEs
  
  Data/dataset size and volume (effect on namespace, tape-disk moving behaviour, etc.)
  
  Exploring “not-Rucio-aware site” use case
  
  Data registration and/or buffering injection from non-Rucio sites (e.g. FTS)
  
  Data handling and management of Telescope or Astronomy facilities,
  and addressing scenarios from Photon and Neutron sciences
  
  HTTP-based storage solution → easy to use and to be adopted by communities
  
  Data Preparation
  
  Calibration, reprocessing, formatting, etc.
  
  Usually drives a conspicuous part of the computing model and data management
  (e.g. MonteCarlo campaigns, RAW→AnalysisObjectData→format-user-friendly, etc.)
  
  Experiment perspective drives the activity
  
  Data Injector+Access Demonstrators
  
  Proposal for a cross-task&WP forum → Data Injector+Access Demonstrators
  
  Expanding fortnightly forum hosted every other Wednesday at 1100 CET
  
  Joint effort with T2.3 and WP5
  
  Data Access and User Analysis Platform (e.g. JupyterLab-Rucio integration)
  
  Caching layer and integration
  
  … (T2.3 leading)
  
  Leveraging effort in EU-funded projects → ESCAPE+CS3MESH4EOSC
  
  LOFAR and CERN have already activities and effort in both projects
  
  Rucio + CERN + CS3MESH4EOSC = GSoC2021
  
  Data Lake Activities
  
  Outcome/takeaway of FDR shows room for improvements in view of DAC21
  
  Several partners started to deploy parallel Rucio instances
  
  Perfect opportunity to jump on challenging activities and improvements
  
  Motto should be: Try, Test, Assess, and Report
  
  Experiment interests/needs as keystones
  
  Explore and discover new orchestration tools and phase-spaces
  
  “metadata”, “multi-VO”, “Rucio Auth schema/policies for ESCAPE”, “automatix”, “bb8”, etc.
  
  Activities are already on-going
  
  Goal to demonstrate the full exportability of unique contributions by integrating them directly in the ESCAPE Data Lake for community-based testing
  
  ----------------------------------------------------------------------------------------------------------------------------------------------
  
  Task 2.3
  
  ----------------------------------------------------------------------------------------------------------------------------------------------
  
  The most important task from the T2.3 point of view is the actual integration. This touches a lot with many of the activities in the other tasks.
  
  Define end-to-end use cases, including cases that are part of the “not-Rucio aware” use case mentioned in Task 2.1
  
  Use those to define the ways in which data is to be accessed by software running on central processing facilities.
  
  AAI implementation is relevant here
  
  Access to RUCIO from within jupyter lab containers
  
  External data access (e.g. VO -> WP4; or external archives)
  
  ----------------------------------------------------------------------------------------------------------------------------------------------
  
  
- 11:50 → 11:55
  
  Depops hot topics 5m
  
  Orateur: Depops weekly chair
- 11:55 → 12:00
  
  AOB 5m