WP4 Technology Forum 3

15 mars 2022, 09:00 → 16 mars 2022, 17:05 Europe/Paris

On-line

Dave Morris, François Bonnarel (CDS ObAS CNRS Université de Strasbourg), Hendrik Heinl, Marco Molinaro (INAF), Mark Allen (CDS/CNRS), Martino Romaniello

 

The WP4 Technology Forum 3 is organised as a major WP4 meeting of the ESCAPE project. All WP4 partners are encouraged to attend.  We plan to have cross-WP activities and a number of invited guests.

Goals

Update on the progress of all of the WP4 tasks.

• Task 4.1 - Integration of astronomy VO data and services into the EOSC
• Task 4.2 – Implementation of FAIR principles for ESFRI data through the Virtual Observatory
• Task 4.3 – Adding value to trusted content in astronomy archives

Prepare the ESCAPE input to the May 2022 IVOA Interoperability meeting.

Strengthen the cross-WP activities with WP3, WP5 and WP2.

Planning for the WP4 activities towards the end of the ESCAPE project

Work together on practicial implementations!

 

Location and Schedule

Due to the on-going covid-19 pandemic, this Technology Forum will be held as an on-line event.

The schedule is include below.  The presentation sessions are organised in the mornings and hack-a-thon interactions in the afternoons, with the final conlcuding session on Thursday afternoon.

Registration

Registration is open

Contributed presentations can be proposed in the registration form, and we also encourage you to propose hack-a-thon topics. Note that you can return to your registration form to add/modify the contributions, and hack-a-thon topics can of course be decided on-the-spot in the meeting according to the needs.

Links

Top level info for WP4 is maintained on the Wiki pages: https://wiki.escape2020.de/index.php/WP4_-_CEVO

ESCAPE project: https://projectescape.eu

 

mardi 15 mars

Welcome and WP status

Join Zoom Meeting
https://us02web.zoom.us/j/85476328170?pwd=bWdvTGFDTkRFaXByMDNqUE9oOHNQUT09

1 Welcome and Introduction

Orateur: Mark Allen (CDS/CNRS)

Allen_ESCAPE_TF3_Intro.pptx

2 CEVO Task 1: status

Orateur: Marco Molinaro (INAF)

Task-4.1_status.pdf

3 CEVO Task 2: status

Orateur: François Bonnarel (CDS ObAS CNRS Université de Strasbourg)

Task4-2StatusReport.pdf

Coffee Break

Welcome and WP status

Join Zoom Meeting
https://us02web.zoom.us/j/85476328170?pwd=bWdvTGFDTkRFaXByMDNqUE9oOHNQUT09

4 CEVO Task 3: status

Orateur: Martino Romaniello (ESO)

Task4.3.pptx

5 Schedule afternoon Hackathons

Lunch

Discussion: EOSC Architecture and Interoperability framework consultation

The EOSC Future project will have a consultation on their "EOSC Architecture and Interoperability Framework". the document is available at the link below.

We should prepare input to the consultation from ESCAPE WP4, in particular on where the Virtual Observatory fits in.

EOSC Architecture and Interoperability Framework - draft document

Hackathons

6 CTA/KM3Net VO DataModels progress

CTA/KM3Net VO DataModels progress "Mapping CTA datamodels and more specifically gammapy index files to IVOA standard in the overal perspective of using/defining interoperability datamodels and standards for KM3Net and CTA." Our objective is to properly describe CTA data products in the VO framework. To make this data findable, a first step is to associate the right metadata in the standardised fields of the VO ObsCore data model. We thus expect to find a mapping between the ObsCore data model and the general CTA data model. We can use as a base to the discussion the gammapy DataStore concept, where gammapy expects an index file or several (obs-index and hdu-index) that shortly describe each data product and its content. However, some information may not be found in the index files, and in that case it may be in the more general CTA data model, or one may need to extract the information from the data itself. The steps of the discussions would be as follow :

• review of use cases and objectives * discussion on each ObsCore keyword to create a mapping table

• identify important metadata not covered by ObsCore (may be the base for an ObsCore extension for VHE/HE) The discussion may be illustrated by Obspm prototype ObsTAP service that exposes HESS public DL3 data :
  https://hess-dr.obspm.fr/
  Use/science cases are listed below material needed to be read before the meeting for more fluent discussion

• presentation / documentation / links of the VO ObsCore mandatory and optional fields : see ObsCore UML diagram in
  https://cloud.projectescape.eu/index.php/s/verY6WXPnzWYTd4 or full specification here :
  https://www.ivoa.net/documents/ObsCore/20170509/index.html

• presentation / documentation / links of the CTA data levels, data content, and general data models : see
  https://cloud.projectescape.eu/index.php/s/1P5oTA1Zow8qn5e

• presentation / documentation / links of the gammapy DataStore structure and the content of the index files :
  https://docs.gammapy.org/dev/api/gammapy.data.DataStore.html https://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html
  https://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/hdu_index/index.html

Google doc to fill :

https://docs.google.com/document/d/1vVzQVfjlbSW9eCEBnRWrPP__E10v7s5lm95-Q-BzyTg/edit?usp=sharing

HackaThonCTAKM3Net.pdf

7 Metadata schemata for resources in the IVOA, in Europe and at global level

https://gather.town/app/PuCDQczUl8pfyXH3/ObAS

Discussion among Marco, Markus and Dave on the path to follow for the integration of VO resources into EOSC, both in terms of the remaining part of the ESCAPE project and the future.

Could be synthesised as follows.

• Keep collaborating with EUDAT B2FIND on extending what we map into their metadata schema.

Recently: Instrument, time.  Space coverage is probably unattractive (although we have quite a lot of records) because no other science uses our reference frame, and also because our geometries (MOCs) are probably too complex for them.  Spectral coverage would be cool, though.

• Don't follow "direct onboarding" as a sustainable solution for single Registry records for now.

The marketplace currently has hundreds of resources in "sciences" while on the VO side we have to onboard tens of thousands (2 orders of magnitude more). Single resources/providers might still want to onboard directly, suggesting all resources follow that paht looks not a suitable solution.

If in the future EOSC Marketplace switches to a metadata schema and architecture similar to the IVOA one, possibly using an extension of the metadata schema of B2FIND, then we could feed them with mapped VOResource if necessary.

Besides the above, discussion touched also:

• interaction with the Task Force on Semantic Interoperability: use cases discussion might be investigated alongside metadata mappings and crosswalks;
• a specific point was also made of reporting as a valuable solution the OAI-PMH harvesting and distributed architecture to the "EOSC Architecture and Interoperable Framework" document review;
• considerations on metadata structures for datasets stored as detached objects or in accompanying flies were brought up and led to considerations on the RUCIO metadata solution;
• finally some discussion on sustainability of federated resources led to something that reads "you need funding to sustain the resources that later have to be federated" not only funding for federations themselves.

mercredi 16 mars

Presentations

Join Zoom Meeting
https://us02web.zoom.us/j/83161947221?pwd=cVFOVkVvbER0bm5RTFFlQXluZEE1dz09

8 Feedback from the 2nd ESCAPE-VO School

Orateur: Stefania Amodeo (CDS, Observatoire Astronomique de Strasbourg)

2_1_stefania_amodeo_SecondESCAPE-VO-School_no_photo.pdf

9 Aladin Lite meets MOCWasm: online calculator and interactive viewer of credible areas of GW sky localizations.

Orateur: Giuseppe Greco (INFN-Perugia)

AladinLiteMOCWasm.pdf

10 LoTSS DR2 - A technical view

Orateur: Yan Grange (ASTRON, the Netherlands Institute for Radio Astronomy)

LoTSS-DR2.pptx

11 ExecutionPlanner - CanIDoThis?

Orateur: Dave Morris (University of Edinburgh)

20220316-02-ExecPlanner.pdf

Lunch

Hackathons

12 LineTAP Proposal

LineTAP is a proposal for a relational schema to describe spectral line transitions that can be queried using the TAP protocol. Current version can be found at https://github.com/mmpcn/slapvamdc An implementation is being prepared, and there are some open questions to be discussed.

The hackathon will happen in gather.town
https://app.gather.town/app/PuCDQczUl8pfyXH3/ObAS

13 Open discussion about the metadata and service interface for the ExecutionPlanner.

The hackathon will happen in gather.town
https://app.gather.town/app/PuCDQczUl8pfyXH3/ObAS

14 Mapping Single dish metadata to Obscore.

Some issues with specific attributes

Starting point : Alessandra's view on some obscore attributes issues with SD data (presented initaily in an IVOA Radio IG meeting)
the hacka a thon has been split in two parts . Initially during the TEch Forum and then a follow up on March 21st

Zoom link:
https://cnrs.zoom.us/j/97900825333?pwd=MitQRkoyZXg3czBHem8vWkxYQTIyZz09

INAF_SD-data-and-the-VO_v1.pdf

SDHaTTechForum2022

Coffee Break

Plenary

15 Summary of Hackathons

HackaThon-1-and4-Feedback.pdf

16 Open Discussion: What are the next technical steps for the Euro-VO aspects of ESCAPE?

What are the next technical steps for integration with WP2, WP3, WP5 ? -- What is feasible and realistic?

(We need some tests of the VO access in the ESAP platform - do the TAP services work correctly?)

There are 2 more IVOA interoperability meetings during the project - what topics should we address there?

(D4.8, the main final deliverable report for WP4 is scheduled for September 2022, so things to include there must be finished by ~end of June)

There will be one more ESCAPE school (WP3) - should we put a tutorial there?

WP 3/4/5

Data Model into codemeta OSSR ESAP.

Execution planner Note - longer term. 

'Wow' notebooks - feasible.

----

WP 3 

put MOCWASM in OSSR

HH interacting with FXP

Aladin / TOPCAT / astropy container -curation effort - done in Task 4.1

----

WP2 Rucio - issue can be closed. stays to be reported in D4.7

----

Data Cite, Dublin Core MD, Claudia already did the mapping

schema - not chosen yet for EOSC. We should comment on that.

----

TAP services

- moving from sync to async. TAP services check.

 

----

Interops:

MM - Registry session (like in 2019 with Claudia, MD). Feedback from Registry Group. 

FB:

FAIRisation of ESFRI using standards has been successful. not all groups at same. 

Work of specific data to VO standards and proposing updates to standards.

 

---

Example - CTA in ESCAPE. Real discussion on data models, coming at same time that CTA is putting its own data model group together. Provenance and VO data access. 

DaCHS implementations - VESPA, ASTRON, JIVE, KM3NeT, EST

----

Tech Forum like activity could be proposed to be included in Concluding event.

 

 

 

 

 

 

 

 

 

 

 

17 Wrap up