Minutes for 2020-05-27 T2.2 meeting
Marcelo V. P. Soares
Yan Grange
Paul Millar
Aleem Sarwar
Marek Szuba
Rohini Joshi
Sorry, I could not attend the meeting today (Maisam).
Enrico Vianello
Frederic Gillardo
Pandey
News
Round table
Paul will not be able to attend next week’s WP2 meeting so Marek will present the QoS update then in his stead. Slides will be prepared together in advance.
Recent versions of the Rucio client have got problems uploading files over GridFTP because they fail to create all the parent directories in advance of an actual file upload. Aleem has been discussing this with Aris, and Marek will report this to Rucio upstream this week. Meanwhile, if you have a Rucio installation which does work do not update!
(Paul) DESY-DCACHE also supports WebDAV, try that. Side issue: we want to demonstrate soon the use of OpenID Connect to authenticate with Rucio, which basically excludes GridFTP file transfers.
GSI RSE (using xrootd) finally added to the testbed, Paul Kramp is now working on adding SciToken (for now only X.509 has been available) authentication to the set-up.
From the last QoS meeting, Aleem and Pandey. Aleem has been gaining access to DESY computing resources. Starting with a small example of LOFAR analysis job. The plan is to start small; once this is working, to recreate the process on other sites.
Aleem has been targeting DESY’s National Analysis Facility (NAF)/batch Infrastructure Resources at DESY (BIRD) Which is the central DESY facility for batch/high throughput computing using HTCondor as a job broker. Started with “hello world” jobs. Now this is working well. Next step would be run LOFAR workflow on DESY compute resources.
And then request other sites participating in ESCAPE project with compute resources to run on their sites.
Data was/is LOFAR long-term archive and copied using srmcp.
Marek is now testing Aleem’s demo, given that DESY parent directory problems might be resolved through the use of a different protocol. Trying some dummy root files, plan to move to more realistic data once teething troubles are fixed.
Marek: Last week they have completed interviews with PANDA and CBM (have been updating the Google docs from Aleem), interviews with the laser community are upcoming. Proposals from ATLAS make sense for both. Issue brought up by CBM: can we reserve storage in advance, which would be crucial for on-line processing directly to the data lake?
(Paul) QoS could be used to request minimal upload bandwidth but space reservation is more tricky (and somewhat outside QoS scope). Details TBA by Paul. Suggested to add this as a use case.
SKA: Updating the document. Added a data life-cycle diagram The QoS policies likely a subset of the ATLAS policies.
Meeting on ESCAPE QoS Doc will be organised on this or preferably next week after discussing with the participants.
Aleem will circulate email today.
No update from Aris and Martin.
AOB
The next meeting will be on 10th June 2020 at 2-3 PM CES