WP3-WP5 Joint Meeting

mercredi 15 sept. 2021, 14:00 → 15:35 Europe/Paris

Zoom

Jutta Schnabel (FAU Erlangen (ECAP)), John Swinbank (ASTRON)

At the WP5 Workshop of 2021-08-05, Gareth raised a number of relevant questions:

• Do we want a “Find Software” page within ESAP?
• Do we want a dedicated “Find Workflow” page, where the user can add criteria to filter the list of workflows?
• If so what would they want to filter by, and are these parameters covered in the zenodo and codemeta metadata?
• Is it worth setting up a Hackathon session where we invite the publishers of the existing workflows and members from WP3 to add them to the OSSR?
• How do we handle complex/composite workflows?
• Which use cases are involved in DAC21?

This meeting will aim to:

• Answer the above, or define a process for answering the above;
• Identify further questions or uncertainties; ensure we share a common vision of what the OSSR/ESAP integration should ultimately look like.
• Identify and assign next-step tasks to move towards that goal.

---

Escape ZoomRoom is inviting you to a scheduled Zoom meeting.

Topic: WP3-WP5 Joint Meeting
Time: Sep 15, 2021 14:00 Amsterdam

Join Zoom Meeting
https://astron.zoom.us/j/89547566811?pwd=VEZ2ZWV6eUY5NFQ5akFKMTRhRFJqZz09

Meeting ID: 895 4756 6811
Passcode: 364657
One tap mobile
+31207946520,,89547566811# The Netherlands
+31207947345,,89547566811# The Netherlands

Dial by your location
        +31 20 794 6520 The Netherlands
        +31 20 794 7345 The Netherlands
        +31 707 006 526 The Netherlands
        +31 20 241 0288 The Netherlands
        +31 20 794 0854 The Netherlands
        +31 20 794 6519 The Netherlands
Meeting ID: 895 4756 6811
Find your local number: https://astron.zoom.us/u/kdL1xvlkLD

Join by SIP
89547566811@zoomcrc.com

Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 895 4756 6811
Passcode: 364657

Join by Skype for Business
https://astron.zoom.us/skype/89547566811

14:00 → 14:05 Introduction

Orateur: John Swinbank (ASTRON)

2021-09-15 — ESCAPE WP3+WP5.pdf

14:10 → 14:20 WP3 Perspective

Orateurs: Jutta Schnabel (FAU Erlangen (ECAP)), Dr Thomas Vuillaume (LAPP, CNRS)

20210915_escape_wp3_wp5.pdf

14:20 → 15:35 Discussion

Notes

• WP5 can use the Zenodo Sandbox, https://sandbox.zenodo.org, for testing: we can create test repositories etc in there, without “polluting” the true OSSR.
• The eossr library presented by Thomas provides a convenient Python interface for working with the OSSR through Zenodo. We should adopt this within ESAP.
• The goals of OSSR and ESAP are slightly divergent, in that OSSR has to be inclusive — it should include all of the software that ESCAPE participants want to upload — but ESAP can be selective — it's not plausible to think that ESAP can provide mechanisms for executing arbitrary software, but instead only certain types will be supported.
• It may be possible to support a wide-range of containerized software, including Jupyter notebooks, in BinderHub.
• It's clear that software cannot be expected to contain information about the services which can execute it. Rather, there are (at least?) two possibilities:
  • A “can you do this?” interface for services, as proposed by Dave. This would be provided with metadata describing the software in question, and would return a “yes/no” answer as to whether that service could execute the software.
  • A database  of service capabilities maintained centrally by ESAP, which could then map software (based on the OSSR metadata) to service (based on known capabilities).
• “Complex workflows” remain somewhat undefined.
  • We have seen at least:
    • Workflows which consist of multiple pieces of software in different containers, “flying in formation”, coordinated by e.g. Docker Compose.
    • Workflows which process data from multiple facilities in a single container (e.g. a notebook which process both SKA and KM3Net data at the same time).
  • Your humble author suggests the latter are much more plausible to execute through ESAP than the former!
  • A really clear taxonomy of complex workflows is required before we can make concrete statements about what ESAP will support.

Action Items

These actions will form the basis of our meeting on 28 September.

• WP5 members (Susana?) to push on getting the HCG-16 workflow fully onboarded into OSSR.
• WP5 members (Stelios?) to experiment with the eossr library in ESAP.
• WP5 members (who?)  to register existing example notebook repositories with Zenodo Sandbox for testing in ESAP.
• WP5 members (who?) to try representing existing workflows as containers, and interfacing those with Binder.
• Dave to experiment with his “can you do this” interface, and see if he has anything to report by 28 September.
• WP3 + WP5 members to review existing and potential future OSSR contents to identify a taxonomy of different types of software and workflow complexities that we need to represent.