Focaliser sur:  Cacher les contributions
S'authentifier

WP4 Technology Forum 3

→ Europe/Paris
On-line

On-line
Dave Morris, François Bonnarel (CDS ObAS CNRS Université de Strasbourg), Hendrik Heinl, Marco Molinaro (INAF), Mark Allen (CDS/CNRS), Martino Romaniello
Description

 

The WP4 Technology Forum 3 is organised as a major WP4 meeting of the ESCAPE project. All WP4 partners are encouraged to attend.  We plan to have cross-WP activities and a number of invited guests.

Goals

Update on the progress of all of the WP4 tasks.

  • Task 4.1 - Integration of astronomy VO data and services into the EOSC
  • Task 4.2 – Implementation of FAIR principles for ESFRI data through the Virtual Observatory
  • Task 4.3 – Adding value to trusted content in astronomy archives

Prepare the ESCAPE input to the May 2022 IVOA Interoperability meeting.

Strengthen the cross-WP activities with WP3, WP5 and WP2.

Planning for the WP4 activities towards the end of the ESCAPE project

Work together on practicial implementations!

 

Location and Schedule

Due to the on-going covid-19 pandemic, this Technology Forum will be held as an on-line event.

The schedule is include below.  The presentation sessions are organised in the mornings and hack-a-thon interactions in the afternoons, with the final conlcuding session on Thursday afternoon.

Registration

Registration is open

Contributed presentations can be proposed in the registration form, and we also encourage you to propose hack-a-thon topics. Note that you can return to your registration form to add/modify the contributions, and hack-a-thon topics can of course be decided on-the-spot in the meeting according to the needs.

Links

Top level info for WP4 is maintained on the Wiki pages: https://wiki.escape2020.de/index.php/WP4_-_CEVO

ESCAPE project: https://projectescape.eu

 

Participants
  • Alessandra Zanichelli
  • Andre Schaaff
  • Andy Lawrence
  • carlo zwolf
  • Catherine Boisson
  • Dave Morris
  • Francois-Xavier Pineau
  • François Bonnarel
  • Gareth Hughes
  • Gilles Landais
  • Gisela Esplugues
  • Giuseppe Greco
  • Hendrik Heinl
  • Jutta Schnabel
  • Marco Molinaro
  • Margarida Castro Neves
  • Marjolein Verkouter
  • Mark Allen
  • Mark Kettenis
  • Markus Demleitner
  • Mathieu Servillat
  • Matthias Fuessling
  • Maximilian Nöthe
  • Mireille Louys
  • Paula Kornecki
  • Quentin Remy
  • Ricardo Rizzo
  • Sara Bertocco
  • Stefania Amodeo
  • Vincenzo Galluzzi
  • Yan Grange
Hendrik Heinl
  • mardi 15 mars
    • Welcome and WP status

      Join Zoom Meeting
      https://us02web.zoom.us/j/85476328170?pwd=bWdvTGFDTkRFaXByMDNqUE9oOHNQUT09

    • Coffee Break
    • Welcome and WP status

      Join Zoom Meeting
      https://us02web.zoom.us/j/85476328170?pwd=bWdvTGFDTkRFaXByMDNqUE9oOHNQUT09

      • 4
        CEVO Task 3: status
        Orateur: Martino Romaniello (ESO)
        Task4.3.pptx
      • 5
        Schedule afternoon Hackathons
    • Lunch
    • Discussion: EOSC Architecture and Interoperability framework consultation

      The EOSC Future project will have a consultation on their "EOSC Architecture and Interoperability Framework". the document is available at the link below.

      We should prepare input to the consultation from ESCAPE WP4, in particular on where the Virtual Observatory fits in.

      EOSC Architecture and Interoperability Framework - draft document
    • Hackathons
      • 6
        CTA/KM3Net VO DataModels progress

        CTA/KM3Net VO DataModels progress "Mapping CTA datamodels and more specifically gammapy index files to IVOA standard in the overal perspective of using/defining interoperability datamodels and standards for KM3Net and CTA." Our objective is to properly describe CTA data products in the VO framework. To make this data findable, a first step is to associate the right metadata in the standardised fields of the VO ObsCore data model. We thus expect to find a mapping between the ObsCore data model and the general CTA data model. We can use as a base to the discussion the gammapy DataStore concept, where gammapy expects an index file or several (obs-index and hdu-index) that shortly describe each data product and its content. However, some information may not be found in the index files, and in that case it may be in the more general CTA data model, or one may need to extract the information from the data itself. The steps of the discussions would be as follow :

        • review of use cases and objectives * discussion on each ObsCore keyword to create a mapping table

        • identify important metadata not covered by ObsCore (may be the base for an ObsCore extension for VHE/HE) The discussion may be illustrated by Obspm prototype ObsTAP service that exposes HESS public DL3 data :
          https://hess-dr.obspm.fr/
          Use/science cases are listed below material needed to be read before the meeting for more fluent discussion

        • presentation / documentation / links of the VO ObsCore mandatory and optional fields : see ObsCore UML diagram in
          https://cloud.projectescape.eu/index.php/s/verY6WXPnzWYTd4 or full specification here :
          https://www.ivoa.net/documents/ObsCore/20170509/index.html

        • presentation / documentation / links of the CTA data levels, data content, and general data models : see
          https://cloud.projectescape.eu/index.php/s/1P5oTA1Zow8qn5e

        • presentation / documentation / links of the gammapy DataStore structure and the content of the index files :
          https://docs.gammapy.org/dev/api/gammapy.data.DataStore.html https://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/obs_index/index.html
          https://gamma-astro-data-formats.readthedocs.io/en/latest/data_storage/hdu_index/index.html

        Google doc to fill :

        https://docs.google.com/document/d/1vVzQVfjlbSW9eCEBnRWrPP__E10v7s5lm95-Q-BzyTg/edit?usp=sharing

        HackaThonCTAKM3Net.pdf
      • 7
        Metadata schemata for resources in the IVOA, in Europe and at global level

        https://gather.town/app/PuCDQczUl8pfyXH3/ObAS

        Discussion among Marco, Markus and Dave on the path to follow for the integration of VO resources into EOSC, both in terms of the remaining part of the ESCAPE project and the future.

        Could be synthesised as follows.

        • Keep collaborating with EUDAT B2FIND on extending what we map into their metadata schema.

        Recently: Instrument, time.  Space coverage is probably unattractive (although we have quite a lot of records) because no other science uses our reference frame, and also because our geometries (MOCs) are probably too complex for them.  Spectral coverage would be cool, though.

        • Don't follow "direct onboarding" as a sustainable solution for single Registry records for now.

        The marketplace currently has hundreds of resources in "sciences" while on the VO side we have to onboard tens of thousands (2 orders of magnitude more). Single resources/providers might still want to onboard directly, suggesting all resources follow that paht looks not a suitable solution.

        If in the future EOSC Marketplace switches to a metadata schema and architecture similar to the IVOA one, possibly using an extension of the metadata schema of B2FIND, then we could feed them with mapped VOResource if necessary.

        Besides the above, discussion touched also:

        • interaction with the Task Force on Semantic Interoperability: use cases discussion might be investigated alongside metadata mappings and crosswalks;
        • a specific point was also made of reporting as a valuable solution the OAI-PMH harvesting and distributed architecture to the "EOSC Architecture and Interoperable Framework" document review;
        • considerations on metadata structures for datasets stored as detached objects or in accompanying flies were brought up and led to considerations on the RUCIO metadata solution;
        • finally some discussion on sustainability of federated resources led to something that reads "you need funding to sustain the resources that later have to be federated" not only funding for federations themselves.
  • mercredi 16 mars
    • Presentations

      Join Zoom Meeting
      https://us02web.zoom.us/j/83161947221?pwd=cVFOVkVvbER0bm5RTFFlQXluZEE1dz09

    • Lunch
    • Hackathons
      • 12
        LineTAP Proposal

        LineTAP is a proposal for a relational schema to describe spectral line transitions that can be queried using the TAP protocol. Current version can be found at https://github.com/mmpcn/slapvamdc An implementation is being prepared, and there are some open questions to be discussed.

        The hackathon will happen in gather.town
        https://app.gather.town/app/PuCDQczUl8pfyXH3/ObAS

      • 13
        Open discussion about the metadata and service interface for the ExecutionPlanner.

        The hackathon will happen in gather.town
        https://app.gather.town/app/PuCDQczUl8pfyXH3/ObAS

      • 14
        Mapping Single dish metadata to Obscore.

        Some issues with specific attributes

        Starting point : Alessandra's view on some obscore attributes issues with SD data (presented initaily in an IVOA Radio IG meeting)
        the hacka a thon has been split in two parts . Initially during the TEch Forum and then a follow up on March 21st

        Zoom link:
        https://cnrs.zoom.us/j/97900825333?pwd=MitQRkoyZXg3czBHem8vWkxYQTIyZz09

        INAF_SD-data-and-the-VO_v1.pdf
        SDHaTTechForum2022
    • Coffee Break
    • Plenary
      • 15
        Summary of Hackathons
        HackaThon-1-and4-Feedback.pdf
      • 16
        Open Discussion: What are the next technical steps for the Euro-VO aspects of ESCAPE?

        What are the next technical steps for integration with WP2, WP3, WP5 ? -- What is feasible and realistic?

        (We need some tests of the VO access in the ESAP platform - do the TAP services work correctly?)

        There are 2 more IVOA interoperability meetings during the project - what topics should we address there?

        (D4.8, the main final deliverable report for WP4 is scheduled for September 2022, so things to include there must be finished by ~end of June)

        There will be one more ESCAPE school (WP3) - should we put a tutorial there?

        WP 3/4/5

        Data Model into codemeta OSSR ESAP.

        Execution planner Note - longer term. 

        'Wow' notebooks - feasible.

        ----

        WP 3 

        put MOCWASM in OSSR

        HH interacting with FXP

        Aladin / TOPCAT / astropy container -curation effort - done in Task 4.1

        ----

        WP2 Rucio - issue can be closed. stays to be reported in D4.7

        ----

        Data Cite, Dublin Core MD, Claudia already did the mapping

        schema - not chosen yet for EOSC. We should comment on that.

        ----

        TAP services

        - moving from sync to async. TAP services check.

         

        ----

        Interops:

        MM - Registry session (like in 2019 with Claudia, MD). Feedback from Registry Group. 

        FB:

        FAIRisation of ESFRI using standards has been successful. not all groups at same. 

        Work of specific data to VO standards and proposing updates to standards.

         

        ---

        Example - CTA in ESCAPE. Real discussion on data models, coming at same time that CTA is putting its own data model group together. Provenance and VO data access. 

        DaCHS implementations - VESPA, ASTRON, JIVE, KM3NeT, EST

        ----

        Tech Forum like activity could be proposed to be included in Concluding event.

         

         

         

         

         

         

         

         

         

         

         

      • 17
        Wrap up
Fonctionne avec Indico v3.3.4