ESCAPE European Data Provider Forum and Training Event

23 nov. 2021, 08:00 → 25 nov. 2021, 20:00 Europe/Paris

On-line

Markus Demleitner (Universität Heidelberg)

The ESCAPE project cordially invites researchers, technicians and librarians active in publishing astronomical data to its European Data Providers' Forum. As its predecessors, it will offer an opportunity to identify common challenges and problems in the dissemination of astronomical data, in particular using Virtual Observatory standards, to exchange solutions, and to share perspectives. The Forum will also be accompanied by hands-on workshops on common publication tools (DaCHS, VOLLT, MOCserver, and more).

ESCAPE is part of the European Union's Horizon 2020 program to establish a single collaborative cluster of next generation ESFRI facilities in the area of astronomy- and accelerator-based particle physics in order to implement a functional link between the projects and European Open Science Cloud (EOSC). Hence, the exchange among data providers at all scales at the EDP is of great importance for the ESCAPE project.

Location and Schedule

The EDP Forum will be held online in gather.town and zoom. The links will be disseminated to the registered participants.

Other Links

Top level info for WP4 is maintained on the Wiki pages: https://wiki.escape2020.de/index.php/WP4_-_CEVO

ESCAPE project: https://projectescape.eu

mar. 23 novembre

08:00 → 09:00 Zoom Link: https://us02web.zoom.us/j/87685216902?pwd=R2k2YlFYeGhtSTFKTldVcXhreldTQT09

09:00 → 10:05 Welcome and introduction

09:00 Welcome

Orateur: Markus Demleitner (Universität Heidelberg)

09:05 Introduction to the VO

Orateur: Hendrik Heinl (CDS/ObAS)

vo_intro.pdf

09:25 The ESO Archive experience in adopting VO Technologies

Orateur: Alberto Micol (ESO)

02_ESO_experience_adopting_VO.pdf

09:45 VO publication and EOSC publication

Orateur: Marco Molinaro (INAF)

VOpublishEOSConboard.pdf

11:00 → 12:20 Presentations

11:00 Mapping VO Protocols/Metadata on top of existing archive

Imagine you are in charge of a project on-line archive and you want to integrate it in the VO framework.
Depending of the nature of your data (images, spectra, cubes, timeseries, catalogs) and on the completeness you want to reach the choice of protocols to implement will be different.
The talk will make a review of the various possibilities according to various examples.

Orateur: François Bonnarel (CDS ObAS CNRS Université de Strasbourg)

EDP.pdf

11:20 Provenance of published data (postponed to Wed. 12:00)

In the context of Open Science, data provenance has become a key piece of information to be provided with astronomical data. It is particularly essential to ensure trust in the data, thus promoting its reuse. As such, provenance is explicitly mentioned in the FAIR principles, which aim to make research data Findable, Accessible, Interoperable and Reusable. In practice, provenance should come with the appropriate granularity and level of details. Based on the IVOA Provenance data model, I will present several proposed solutions to capture, store and access provenance metadata for published data.

Orateur: Mathieu Servillat (LUTH, Observatoire de Paris)

2021-11-24 provenance for data providers @EDPF.pdf

11:40 CAOM within the ESAC Science Data Centre

We would like to present an overview of the Common Archive Observation Model (CAOM), a general purpose data structure for use as the core data model of an astronomical data centre, designed by the Canadian Astronomy Data Centre (CADC). This common model is being used at the ESAC Science Data Centre (ESDC) in the contexts of the European Hubble and James Webb Space Telescopes Science Archives. We will demonstrate how we made possible to achieve a fruitful collaboration between Space Telescope partners by making use of a common observation data model, together with software projects built around it to enable the collection, exchange and verification of astronomical datasets.

Orateurs: Maria Arevalo Sanchez (ESA), Javier Espinosa (ESA)

20211123_CAOM_ESDC_Archives_ESCAPE_MArevalo_JEspinosa.pdf

12:00 Providing access to Solar System & heliophysics data (VESPA / Europlanet)

VESPA (Virtual European Solar and Planetary Access) has been focusing for nearly 10 years on adapting Virtual Observatory (VO) techniques to handle Planetary Science data. The objective of this activity is to build a contributive data distribution system where data services are located and maintained in research institutes, as well as in space agencies and observatories. VESPA is part of Europlanet, a series of programs funded by the European Union and dedicated to enforcing collaborative activities in Planetary Science both within and outside Europe. A new program called Europlanet-2024 has started in 2020 for a 4-year period. VESPA has defined an architecture adapted from the astronomy VO, and incorporating concepts and standards from other areas (Earth observation, Heliophysics, etc). The basic system uses the VO infrastructure: data services are installed in any location but are declared in a system of harvested registries with identifiers. Such services are interoperable via clients and tools, which also provide visualization and analysis functions. A specific access protocol (EPN-TAP) has been defined to allow the user to identify data of interest.

Orateur: Stéphane Erard (LESIA)

Erard_VESPA_ESCAPEforum_2021.pdf

14:00 → 16:30 Workshops

14:00 Taplint, STIL and samp.js

Validating TAP services with taplint
Using STIL to process VOTable in Java
Connecting Web pages and VO Tools with samp.js

Orateur: Mark Taylor (University of Bristol)

sampjs.pdf

stil.pdf

taplint.pdf

14:00 What you're always wanted to as about DaCHS and VO data publishing

Fallback topics: HTML templating in DaCHS, writing examples

Orateur: Markus Demleitner (Universität Heidelberg)

mer. 24 novembre

08:00 → 09:00 Zoom Link: https://us02web.zoom.us/j/87084462185?pwd=RlFVRGtQaU1JNlRwSjZyMWk4WTJCZz09

09:30 → 10:30 Presentations

09:30 Stored Data Retrieval through Virtual Observatory standards

A service to retrieve data stored both in a long term preservation storage and in a fast access file system has been built at the Italian Astronomical Archives (IA2), based on Virtual Observatory standards. Some IA2 users have the need to store big sized data sets for a long term. At the same time, they need to recall these data when needed, perform analysis and store the new results. The proposed hardware solution is built on a long term preservation storage (tape library) connected to a fast access storage node for direct data exploitation. The software service has a data retrieval web interface based on IVOA standards: VOSpace for the data transfer and access, Single Sign-On (SSO) for the authentication, Group Membership Service (GMS) for the authorization, Universal Worker Service (UWS) to manage transfers. The plan for the future, after putting the service into production, is to exploit the use of IVOA standards to make this service interoperable with other similar in different data centres.

Orateur: Sara Bertocco (INAF-OATs)

vospace_tape_Bertocco_final.pdf

09:50 ARTEMIX / Yaftis - Remote 3D viewer

Yafits (Yet Another FITS viewer) is a standalone web application that enables remote visualisation of 3D data cubes hosted in FITS format on the server-side. The client can display images and spectra in order to quick-look the scientific content of the observations. The prime use is for radio-astronomy data, ie the ones from ALMA, IRAM/NOEMA or SKA but any kind of 2D or 3D FITS could be remotely accessed. Dedicated methods for radio data inspection have been implemented, like for instance the comparison with expected spectral lines frequencies compiled in molecular database catalogues. Yafits also uses SAMP for interoperability with Aladin or CASSIS to enable further data analysis. Yafits is integrated into ARTEMIX (ALMA RemoTE data MIning eXperiment http://artemix.obspm.fr) a web service that offers remote FITS visualisation of the ALMA scientific Archive since 2016 and which is continuously evolving with Yafits new features.

Orateur: Philippe Salome (LERMA, Observatoire de Paris )

10:10 Low Frequency Radio Astronomy on the VO

We present the MASER (Measurement, Analysis, and Simulation of Emissions in the Radio range) toolbox, which enables science ready access to low frequency radio astronomy data. MASER focuses on time-domain and dynamic spectra data products and associated tools. Being at the interface between astronomy, heliophysics and planetary sciences, we make use of interoperable standards from each world (VOTable, TAP and UWS from the IVOA, EPN-Core from Planetary Sciences, or Das2 from heliophysics).

Orateur: Baptiste Cecconi (Observatoire de Paris)

MASER-LF-Radio-BCecconi-EDP-2021.key

MASER-LF-Radio-BCecconi-EDP-2021.pdf

11:00 → 12:20 Presentations

11:00 Publishing survey data from the Observatorio Astrofísico de Javalambre (OAJ): The CEFCA Catalogues Portal

The Centro de Estudios de Física del Cosmos de Aragón (CEFCA) is carrying out from the Observatorio Astrofísico de Javalambre (OAJ, Teruel, Spain) two large area multiband photometric sky surveys, J-PLUS and J-PAS, covering the entire optical spectrum using narrow and broad band filters. J-PAS and J-PLUS include coadded and individual frame images, dual and single catalogue data which include parameters measured from coadded images and photo-redshift computations. This contribution presents the CEFCA Catalogues portal (archive.cefca.es). A powerful web portal which has been implemented to publish all this survey data offering advanced tools, each suited to a particular need, for data search, visualization and download. This portal includes web user interface services such as sky navigator, object visualization, object list search, ADQL asynchronous queries interface, cone search and image search and download. All these web interface services support SAMP protocol to interoperate and communicate with VO-compatible applications. The CEFCA catalogues portal also offers all of this data through Virtual Observatory (VO) services like SIAP, SCS, TAP and catalogue and images HIPS. Moreover, the CEFCA Catalogues portal enhances data publication of these large surveys following FAIR principles to maximize research efficiency.

Orateur: Tamara Civera Lorenzo (CEFCA (Centro de Estudios de Física del Cosmos de Aragón))

CEFCACataloguesPortalDemo_TamaraCIvera.mp4

TamaraCivera_PublishingDataFromOAJ_2021.pdf

11:20 The EVN data archive VO service

At JIVE we are working on making interferometric visibility data from the European VLBI Network (EVN) archive available through VO protocols. For this purpose we have built a TAP service that serves ObsCore records. To create these records we wrote some special-purpose software in Python that extracts the necessary meta-data from FITS-IDI files. In this presentation I will discuss some of the choices made when writing this software as well as some further improvements we still intend to make before the service goes "live". I'll finish with some results on an attempt characterize the UV-coverage of VLBI data using a parametrization that is being considered for a Radio-specific extension to ObsCore.

Orateur: Mark Kettenis (JIVE)

EDP-2021.pdf

11:40 Publishing the e-MERLIN archive

Progress in publishing the e-MERLIN archive, and the role of this work as a precursor to SKA Regional Centre development will be described.

Orateur: Paul Harrison (Jodrell Bank, University of Manchester )

emerlinArchivePAH.pdf

12:00 Provenance of published data

In the context of Open Science, data provenance has become a key piece of information to be provided with astronomical data. It is particularly essential to ensure trust in the data, thus promoting its reuse. As such, provenance is explicitly mentioned in the FAIR principles, which aim to make research data Findable, Accessible, Interoperable and Reusable. In practice, provenance should come with the appropriate granularity and level of details. Based on the IVOA Provenance data model, I will present several proposed solutions to capture, store and access provenance metadata for published data.

Orateur: Mathieu Servillat (LUTH, Observatoire de Paris)

2021-11-24 provenance for data providers @EDPF.pdf

14:00 → 19:00 Workshops

14:00 Powering up a data access service (e.g. TAP) with VOLLT

Orateur: Grégory Mantelet (CDS, Observatoire Astronomique de Strasbourg)

14:00 Publishing HiPs on MOCServer2 & Aladin Lite

The Material we will be using is:
http://cds.unistra.fr/adass2018/

Please look into the requirements! In particular the download of the fits images will be ~200 MB

Further useful links are:
http://aladin.u-strasbg.fr/hips/HipsIn10Steps.gml
http://aladin.u-strasbg.fr/hips/HipsCat.gml

Orateur: Hendrik Heinl (CDS/ObAS)

14:00 Taplint/STIL/samp.js questions

Orateur: Mark Taylor (University of Bristol)

14:00 VOSSIA: Your Lordship Shall configure and publish simple services

Orateur: Nicola Calabria (INAF)

14:00 What you're always wanted to as about DaCHS and VO data publishing

Fallback topics: Metaprogramming in DaCHS, Regression Testing

Orateur: Markus Demleitner (Universität Heidelberg)

jeu. 25 novembre

08:00 → 09:00 Zoom Link: https://us02web.zoom.us/j/88047635414?pwd=S29CNFYra0R5ZXZEbm5PVzd0QnRLUT09

09:30 → 10:30 Presentations

09:30 ESAP: the ESCAPE ESFRI Science Analysis Platform

ESAP, the ESCAPE ESFRI Science Analysis Platform, is being developed to provide a flexible toolkit for constructing science platforms. It provides capabilities to quickly unite a range of data access and analysis services — in particular those developed by the ESCAPE project — behind a single, consistent but customizable, user interface. In this presentation, I will provide a brief overview of ESAP's aims, its architecture, and its current and expected future capabilities.

Orateur: John Swinbank (ASTRON)

2021-11-25 — ESAP.pdf

09:50 Catalog of Coronal Hole detection in TAP service

Coronal holes (CH) are seen as dark features on EUV images of the Sun. They are the source of the fast solar wind, and as such are closely monitored. The CH feature recognition module named SPoCA-CH was developped at the Royal Observatory of Belgium. Since 2010, one version of this software is running at LMSAL and provides near-real time detection of coronal holes. We would like to provide the outputs from an updated version of SPOCA-CH as a catalog for a VESPA TAP service. In this talk, we will discuss our progress, as well as questions regarding the encoding of tracking or provenance information within this TAP service.

Orateur: Veronique Delouille (STCE/Royal Observatory of Belgium)

CH_Catalog_DPF_VDelouille.pdf

10:10 ASTRON data holdings

Orateur: Yan Grange (ASTRON, the Netherlands Institute for Radio Astronomy)

Yan_Grange_ASTRON.pptx

11:00 → 12:00 Panel - Problem solving platform for data providers

11:00 Panel

Discussion on the creation of a practical problem-solving platform to facilitate implementations and prototypes of interoperable access to ESFRI and pathfinder data: This involves the identification of a pool of expertise (e.g., web pages and communication channels) for the collection of documented common solutions, as well as ways to bring those to the interested parties.

Orateurs: Kay Graf (ECAP - University of Erlangen), Marco Molinaro (INAF), Mark Allen (CDS/CNRS), Markus Demleitner (Universität Heidelberg)

14:00 → 19:00 Workshops

14:00 Manage and customise your TAP_SCHEMAta with TASMAN

Orateur: Sonia Zorba (INAF)

14:00 Taplint/STIL/samp.js questions

Orateur: Mark Taylor (University of Bristol)

14:00 What you're always wanted to as about DaCHS and VO data publishing

Patterns in Operation, PIDs in DaCHS

Orateur: Markus Demleitner (Universität Heidelberg)