WP4 Provenance Workshop

7 sept. 2020, 14:00 → 8 sept. 2020, 18:00 Europe/Paris

Provenance information helps to explore the traceability of products, find contact information and acknowledge people. Providing provenance information allows a user to assess the quality and reliability of the products. If the stored information is sufficiently fine grained, it is possible to enable the reproducibility of an activity or sequence of activities.

Recently the IVOA released a standard to structure provenance metadata and several implementations are in development in order to capture, store, access and visualize the provenance of astronomy data products. This meeting will focus on technical solutions with presentations, demonstrations, hands-on sessions and discussions. The objective is also to collect the requirements of ESFRI projects in order to build the road map of future developments.

The IVOA Provenance data model can be found here:
http://www.ivoa.net/documents/ProvenanceDM

This workshop was accessible online through the Zoom visioconference system.

Organizing committee:
Mathieu Servillat
Catherine Boisson
François Bonnarel
Mireille Louys
Michèle Sanguillon
 

Notes during the use case presentations

Shared notes for use cases

Shared notes for VO presentations

lundi 7 septembre

Provenance model and tools

Presentations of the IVOA data model, detailed presentations of related existing implementations, including demonstrations

Shared notes for VO presentations

1 Introduction

Orateur: Mathieu Servillat (LUTH, Observatoire de Paris)

2020-09-07 introduction provenance workshop.pdf

Provenance workshop questionnaire.pdf

2 Link with IVOA and the Virtual Observatory

Orateur: Mireille Louys

provenanceConcepts-IVOAuptake-2020Sep07-v2.pdf

3 The IVOA Provenance data model

Orateur: Mathieu Servillat (LUTH, Observatoire de Paris)

2020-09-07 provenance data model.pdf

4 Capture of provenance information : example of logprov for gammapy

Orateur: Mathieu Servillat (LUTH, Observatoire de Paris)

2020-09-07 logprov in gammapy.pdf

5 Storage of provenance information: example of CTA DIRAC

Orateur: Mme Michèle Sanguillon (LUPM)

200907_Provenance_CTADIRAC_WP4ESCAPE.pdf

6 Access to provenance information: example of ProvHiPS

Orateur: François Bonnarel (CDS ObAS CNRS Université de Strasbourg)

ProvTAPProvHiPS ESCAPE.pdf

7 Visualization of provenance information: voprov and ProvSAP within OPUS

Orateur: Mathieu Servillat (LUTH, Observatoire de Paris)

2020-09-07 voprov and ProvSAP within OPUS.pdf

Example of graph using voprov and ProvSAP within OPUS (change values in the query string to get different graphs)

15:40 Coffee break

Use cases and requirements within projects

Presentations and discussion on use cases concerning the ESCAPE ESFRI and related projects

Shared notes for use cases

8 VizieR catalogs

Orateur: Gilles Landais (CDS)

Prov_VizieR_escape_2020.pdf

9 Multi-frequency polarimetry of extragalactic radio sources

The high-frequency (> 20 GHz), bright flux density (> 200 mJy) extragalactic radio sources (ERSs) population is dominated by blazars. Multi-frequency (and multi-epoch) polarimetry is invaluable to study magnetic fields and plasma in the inner and unresolved regions of their relativistic jets. Moreover, as ERSs constitute an important foreground for CMB, such studies are also crucial for Cosmology, e.g. in the search for primordial B-modes associated to inflation in the Early Universe. In this talk I will focus on radio-interferometric observations performed with the Australia Telescope Compact Array (ATCA) and the Atacama Large Millimeter/sub-millimeter Array (ALMA), and their associated data products (from calibrated visibilities and radio maps to more advanced products resulting from analysis, e.g. catalogues and source counts). I will also briefly address those ancillary products which may be relevant for quality assessment, reproducibility or data reprocessing. Then, I will summarize provenance requirements from similar projects and present how we already performed the capture, storage and distribution of provenance information.

Orateur: Vincenzo Galluzzi (INAF - Osservatorio Astronomico di Trieste)

Multi-frequency polarimetry of a complete sample of extragalactic radio sources.pdf

10 KM3NeT

Orateurs: Jutta Schnabel (Friedrich-Alexander Universität Erlangen-Nürnberg), Tamas Gal (ECAP / FAU)

KM3NeT provenance ESCAPE WP4.pdf

mardi 8 septembre

Use cases and requirements within projects

Presentations and discussion on use cases concerning the ESCAPE ESFRI and related projects

Shared notes for use cases

11 LST: Large Size Telescope prototype for CTA

Orateur: Jose Enrique Ruiz

LSTOSAProv.pdf

r0_to_dl2_01948_prov.pdf

12 BASS2000: Solar Survey Archive

Orateur: Jean Aboudarham (LESIA/PADC - Observatoire de Paris/PSL)

BASS2000.pdf

13 HEK database: solar events using VOEvent

The SPoCA-suite provide two modules, one for Active Region (AR) detection, and one for Coronal Holes(CH) detection, to the SDO Event Detection System (EDS). It runs in near-real times at Lockheed Martin Solar and Astrophysics Laboratory. Every four hours, the EDS generates and uploads the SPoCA entries into the AR and CH Catalogs of the HEK or Heliophysics Events Knowledgebase.

The VOEvent standard of 2006 is used in the HEK. In this talk, we analyze the provenance information that is encoded in the field of the VOEvent for the SPoCA-AR and SPoCA-CH event, and discuss how this information could be completed or reorganized in view the proposed provenance data model.

Orateur: Veronique Delouille (STCE/Royal Observatory of Belgium)

SPoCA_VOEvent_VDelouille.pdf

14 NenuFAR / ExPRES

Orateurs: Alan Loh, M. Baptiste Cecconi (Observatoire de Paris)

2020-09-09_Provenance_AlanLoh.pdf

ExPRES-Provenance.pdf

10:50 Coffee break

15 LOFAR / APERTIF / WSRT surveys

Orateurs: Mattia Mancini, M. Yan Grange (ASTRON, the Netherlands Institute for Radio Astronomy)

ASTRON_data.pdf

ASTRON_data.pptx

16 European VLBI Network (EVN) Archive

In this use-case presentation the EVN Archive and its provenance challenges will be presented.

The EVN is a proposal-driven open-access instrument for VLBI astronomy. There is considerable interest -- not least from funding agencies -- about the conversion of the data taken into papers published in the scientific literature (which in our case can be restricted to those registered in the Harvard-hosted register, ADS). JIVE is currently working on schemas to record this information and investigating tooling to harvest it (semi-)automatically.

Orateurs: Des Small (JIVE), Harro Verkouter (Joint Institute for VLBI ERIC), Mark Kettenis (JIVE)

EVNArchiveProvenance-Verkouter.pdf

17 Italian Radio Data Archive

Orateur: Alessandra Zanichelli

IRDA-ESCAPEWP4_8sept2020_v2.pdf

Discussion on solutions, demonstrations and conclusions

Discussion on the various developments required, with initial technical specifications for the solutions