WP2 fortnightly meeting

Europe/Paris
    • 11:00 AM 11:10 AM
      News 10m
      Speakers: Rosie Bolton (Square Kilometre Array Organisation) , Xavier Espinal (CERN)
      • Mid-term review confirmed by the 27th of November 2020. 
        • Preparations: ~30mins/WP + ~20mins discussion draft to be circulated with project management by Friday 23 October 
          • Task leads: high level summary slide (or couple of them) by tomorrow
          • DepOps: nice data lake activity plots 
        • E-EB 3rd of November
      • Technical coordination meeting kick-off (date tbc)
        • To ensure good integration of the ESFRIs and RIs with the various technical work packages, and good understanding and feedback between what the scientific collaborations want to do and how ESCAPE implements the relevant infrastructure and services
        • WP leads and representatives from the ESFRIs and RIs. Technical experts invited when needed.
      • Milestones and deliverables:
        • M2.3 (M22): Second WP2/DIOS Workshop (virtual event) 
          • Date: 8th and 9th of December (half day)
        • M2.4 (M24): Expanded prototype. Verify experiment data access from compute platforms (including commercial clouds)
          • Heads-up to review storage resources deployed. To evolve data lake infrastructure towards a meaningful size, allowing to scale-up data ingestion and throughput challenges. 
      • FDR exercise
       
       
       
       
    • 11:10 AM 11:30 AM
      WP2 workshop preparation 20m
      Speaker: All

      WP2 2nd workshop agenda discussion

      • Format: we proposed 1 full day, comments?
      • Pilot assessment
        • Experiments perspective? 
        • Sites perspective? 
        • Task leads perspective about the exercise? what went well, what can be improved, plans for the 2nd FDR, etc... 
      • Next steps:
        • Preparations for the 2nd Dress Rehearsal
      • Integration with compute/WP5 joint discussion?
      • Next steps:
        • Evolution of the pilot towards the prototype
      • Action list review and highlights

      NB: Need to appoint a scientific secretary. Please drop an email to myself/Rosie if you want to cover this key role.

       
       
       
       
       
    • 11:30 AM 11:50 AM
      Data injection demonstrators (discussion) 20m
      Speaker: Dr Riccardo Di Maria (CERN)

      All the sciences/experiments are now on board, having provided reports for at least the 1st exercise.

      The reports for the 2nd exercise are attached to agenda https://indico.in2p3.fr/event/22501/ .

      CMS, EGO-VIRGO, and SKA reports for 2nd tasks are still missing.

       

      3rd Data Injection Demonstrators meeting next week, for which the 3rd task is due:

      accessing the data in the lake with an application/workflow that does use the data and produce a real output.

       

      If you are using a JupyterLab notebook, the work performed by our GSOC student (Rucio-JupyterLab Extension) could be handy. It is a simple extension to be installed.

      https://github.com/didithilmy/gsoc2020

      https://github.com/didithilmy/rucio-jupyterlab

       

      Several topics were raised at today's meeting that should be discussed with a broader audience (i.e. next WP2 Fortnightly meeting):

      - greedy option for RSEs;

      - Rucio multi-VO functionalities;

      - experiment-specific RSE.

    • 11:50 AM 12:00 PM
      AOB 10m

      Rob Barnsley - Rucio testing framework

      - Rob (SKAO): Data Lake automated transfer tests