ESCAPE QoS fortnightly meeting

Europe/Paris

Minutes for 2020-05-27 T2.2 meeting

Present

Marcelo V. P. Soares

Yan Grange

Paul Millar

Aleem Sarwar

Marek Szuba

Rohini Joshi

Apologies

Sorry, I could not attend the meeting today (Maisam).

Enrico Vianello

Frederic Gillardo

Pandey

Agenda

  • News

  • Round table

News

Paul will not be able to attend next week’s WP2 meeting so Marek will present the QoS update then in his stead. Slides will be prepared together in advance.

Recent versions of the Rucio client have got problems uploading files over GridFTP because they fail to create all the parent directories in advance of an actual file upload. Aleem has been discussing this with Aris, and Marek will report this to Rucio upstream this week. Meanwhile, if you have a Rucio installation which does work do not update!

  • (Paul) DESY-DCACHE also supports WebDAV, try that. Side issue: we want to demonstrate soon the use of OpenID Connect to authenticate with Rucio, which basically excludes GridFTP file transfers.

GSI RSE (using xrootd) finally added to the testbed, Paul Kramp is now working on adding SciToken (for now only X.509 has been available) authentication to the set-up.

Update on Running LOFAR use-case

From the last QoS meeting, Aleem and Pandey.  Aleem has been gaining access to DESY computing resources.  Starting with a small example of LOFAR analysis job.  The plan is to start small; once this is working, to recreate the process on other sites.

Aleem has been targeting DESY’s National Analysis Facility (NAF)/batch Infrastructure Resources at DESY (BIRD) Which is the central DESY facility for batch/high throughput computing using HTCondor as a job broker. Started with “hello world” jobs. Now this is working well. Next step would be run LOFAR workflow on DESY compute resources.
And then request other sites participating in ESCAPE project with compute resources to run on their sites.

 

Data was/is LOFAR long-term archive and copied using srmcp.

 

Round table
Streams info can be found at: https://indico.in2p3.fr/event/19937/contributions/75972/attachments/56927/75787/ESCAPE-T2.2-update.pdf

Stream A: prototyping and demonstration

Marek is now testing Aleem’s demo, given that DESY parent directory problems might be resolved through the use of a different protocol.  Trying some dummy root files, plan to move to more realistic data once teething troubles are fixed.


Stream B: engagement with experiments

Marek: Last week they have completed interviews with PANDA and CBM (have been updating the Google docs from Aleem), interviews with the laser community are upcoming. Proposals from ATLAS make sense for both. Issue brought up by CBM: can we reserve storage in advance, which would be crucial for on-line processing directly to the data lake?

  • (Paul) QoS could be used to request minimal upload bandwidth but space reservation is more tricky (and somewhat outside QoS scope). Details TBA by Paul. Suggested to add this as a use case.

SKA: Updating the document.  Added a data life-cycle diagram  The QoS policies likely a subset of the ATLAS policies.

Meeting on ESCAPE QoS Doc will be organised on this or preferably next week after discussing with the participants.
Aleem will circulate email today.


Stream C: software developments

No update from Aris and Martin.

AOB
The next meeting will be on 10th June 2020 at 2-3 PM CES

 
 
There are minutes attached to this event. Show them.
    • 2:00 PM 2:10 PM
      News 10m
      Speaker: Paul Millar (DESY)
    • 2:10 PM 2:50 PM
      Round Table 40m
      • Stream A: prototyping and demonstration

      • Stream B: engagement with experiments

      • Stream C: software development

    • 2:50 PM 3:00 PM
      AOB 10m