WP5 monthly meeting 20 January 2020

→ Europe/Paris
Description

Meeting with all partners involved in WP5/ESAP

You have been invited to attend a VidyoConference taking place in virtual meeting room:
ASTRON Zoom Room 4 at https://astron.zoom.us

Meeting ID: 944 740 9884

Join Zoom Meeting
https://astron.zoom.us/j/9447409884

Join by web (Chrome only)
https://zoom.us/wc/join/9447409884

Dial by your location
        +31 (0) 20 241 0288 Netherlands
        +31 (0) 70 808 1008 Netherlands
Meeting ID: 944 740 9884
Find your local number: https://zoom.us/u/acRpGgV86D

Join by Skype for Business

https://astron.zoom.us/skype/9447409884

For using the full Zoom client, please download the latest client from https://astron.zoom.us/download

 

Contact
Cacher

ESCAPE WP5 Monthly Meeting notes 20 January 2020

1. Roll call

ASTRON: Zheng Meyer, Christine Werkhoven

CERN: Aristeidis Fkiaras, Riccardo Di Maria, Rizart Dona

CNRS: Enrique Garcia

CSIC: Susana Sanchez

CTAO: Matthias Fuessling

EGO: Pierre Chanial

FAIR: Raffaele Grosso

FAU: Jutta Schnabel

KIS: -

INAF: -

IFAE: Jordi Delgado Mengual

JIVE: Harro Verkouter

Nikhef: -

OU: Hugh Dickinson

RUG: Myroslav Kavatsuyk

SKAO: Rosie Bolton

UCM: Daniel Nieto

UEDIN: Stelios Voutsinas, Dave Morris

2. Last meeting minutes

-

 

3. Announcements, news and general updates

ASTRON

Each of the MVP (Minimum Viable Product) core team partners are leading the development of one or more service categories:

  • ASTRON is leading the development of the ESAP UI and API Gateway.

  • SKA and CTA lead the aspects of data lake and workload management integration.

  • Nikhef leads on the federated-AAI aspects.

  • UEDIN leads the IVOA integration and interactive data processing services.

  • INAF is leading the development of batch processing.

WP5 has been invited to give a presentation at the CS3 workshop, where the CS3MESH project will kick-off. We are also invited to the CS3MESH4EOSC Kick-off Event. Both events will take place in Copenhagen between 27-30 January.

Two web developers (1.2 FTE) are working on the ESAP UI and API Gateway development at ASTRON. We expect to have the first version ready before the developers busy week. 

CERN

  • Discussion with SWAN and Rucio this week for possible integration (still really early stages)

  • Preparing some presentations for Brussels meeting which will possibly be interesting to WP5. (Datalake usage and access showcase/tutorial)

  • Rucio workshop 10th of March at Fermilab https://indico.cern.ch/event/867913/

  • ISGC2020 March 8th in Taipei http://indico4.twgrid.org/indico/event/11/program

CSIC -

CTAO

  • CTAO is now hiring, the job advertisement can be found at: https://www.cta-observatory.org/jobs/#1579265699488-cce62d7a-7bc9 The position aims to support CTAO activities in all ESCAPE work packages (WP3, WP4, WP5) with a special focus on WP5.

  • Beyond that, we are currently coordinating the activities of the CTA partners within (and outside) ESCAPE on the topics of Rucio and DIRAC (CTA is active both in WP2 and WP5 on these topics).

FAU

Within ESCAPE: mostly activity related to WP3 - WP5 links:

  • Common phone call on distributed computing (Dirac) in tech call

  • note: ESCAPE workshop on innovative workflows 7-10 July in Salerno (tbc)

For KM3NeT: development of workflows ongoing (including Jupyter notebooks as interfaces, CWL integration), no additional hiring is planned.

INAF -

IFAE

Objective: use Big Data platforms/tools to analyze MAGIC/CTA (gamma-ray astronomy data)

 

The current status of our work is described in the following. Green as Done, Orange on Going. The rest, pending tasks

JIVE

The new, bi-directional, EVN archive is currently under analysis. A new/parallel database needs to be designed first to enable the WP5 goal of receiving and linking to derived data and updated workflows. We are investigating different pipeline schemes (Jupyter notebook, Common Workflow Language) to write our re-runnable pipeline(s) in.

KIS -

Nikhef

Working on exploring component integrations that might be of interest for the ESAP. Knowledge for design and development of large, long-lived and secure software systems is available. There will be Nikhef attendance at ISGC as well.

RUG -

OU -

SKAO

Not much to report from SKA - we still have to people actively working on WP5 activities - looking at platform technologies (K8s, LSST SW stack), using DIRAC workload management service on UK grid (under SKAtelescope VO as a testbed) and looking at ways to improve reproducibility in SKA's science data challenges - looking at open-sourcing some code for assessing submissions and at features possible through kaggle.

UEDIN

We've been experimenting with running a a Hadoop (Yarn/HDFS) / Spark cluster, with a Apache Zeppelin as the interactive analysis tool, and data stored in Parquet format to run analysis on Gaia DR2 data. Alongside this we're looking at different cloud technologies such as Kubernetes/Helm & Ansible. We're also running a JupyterHub service and working on Docker images that build environment (kernels) that allow users to access and discover data through VO tools.

 

4. WP5 Busy Week - end of March/beginning of April

We want developers to attend this meeting.

List of participants:

  • ASTRON: Nico Vermaas + Zheng Meyer-Zhao

  • CERN: Aris Fkiaras

  • SKAO: James & Rohini

  • UEDIN: Stelios Voutsinas

Date:

Zheng will set-up a doodle please let her know if she needs to invite any else.

Location:

The Netherlands, venue tbd

 

5. WP4 Tech Forum - 4-6 February

Participants from WP5

JIVE: Harro Verkouter

EGO: Pierre Chanial

UEDIN: Dave Morris

CTAO: Matthias Fuessling (not 100% sure yet)

ASTRON: Zheng Meyer-Zhao, Yan Grange, Mattia Mancini

Zheng will present the WP5 developments.

 

6. ESCAPE Progress meeting 26-27 February , Brussels, BE

The list of participants is now online available at: https://indico.in2p3.fr/event/20203/registrations/participants 

WP2/WP5 joint session topics will be discussed via email.

WP4/WP5 joint sessions will be a continuation of the WP4 Tech Forum meeting.

 

7. WP5 Tech meetings Topics

Matthias Fuessling

  • Talk/contribution by J. Colomé on a Multi-Messenger Platform prototype build up to coordinate observations, the collaboration and (to some lesser extend) the data analysis for multi-messenger use cases.

Pierre Chanial

  • ESA talk about the science platform

  • invite Microsoft representative

Rosie Bolton:

  • LSST Software Stack by James, Rohini

Stelios Voutsinas

  • I'd be interested as well in having a Zoom meeting to discuss the LSST stack as well, as I've recently been experimenting with setting up the LSP (LSST Science Platform) with K8s on Openstack.

  • Topic: Collaboration tools for MVP such as github. How do we collaborate remotely and what are the rules?

Susana Sanchez

  • MVP and how to implement a new use case

Riccardo Di Maria

  • SWAN and Rucio

8. Round table

FAU: WP3 will organize a workshop on innovative workflows 7-10 July in Salerno (tbc). We are looking for experts on ML DL please forward names to Jutta Schnabel.

UCM: Hiring is finalized, Daniel Neto will fill the post doc position and start the end of February.


 

Il y a un compte-rendu associé à cet événement. Les afficher.