iRODS workshop @ CC-IN2P3

Europe/Paris
CC-IN2P3

CC-IN2P3

12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
Jean-Yves Nief (CC-IN2P3), Reagan Moore (RENCI)
Description
This workshop aims to gather people from various communities who are interested in iRODS or who are already using it. This workshop will also include an iRODS tutorial on rules and micro-services, and presentations on applications of the iRODS technology. If you are interested in presenting, please contact Jean-Yves Nief ("support" button on the left hand side). Please note that the number of participants will be limited.
Participants
  • Adil Hasan
  • Alexis MICHON
  • Andre Schaaff
  • Andreas Landhäußer
  • Christian surace
  • Christophe Blanchet
  • Csaba Anderlik
  • Dominique de Waleffe
  • Eric Liao
  • Francesco Locunto
  • garonne vincent
  • Giuseppe Ugolotti
  • Guy Warner
  • Jean-Pierre Chanod
  • Jean-Yves Nief
  • Jerome Fuselier
  • Jerome Raciazek
  • john floan
  • Jonas Dahlblom
  • Jérôme Dupont
  • Kevin O'Neill
  • Laurent Duplouy
  • le sidaner pierre
  • Lionel Micault
  • Marco Piras
  • Mark McArdle
  • Matteo Vocale
  • Michael Wan
  • Mohammad Shahbaz Memon
  • oleg Lodygensky
  • Paolo Maggi
  • Pascal CALVAT
  • Paul Mullen
  • Pauline Mak
  • Pierre-Yves JALLUD
  • Reagan Moore
  • Roger Downing
  • Sorina CAMARASU POP
  • Thierry Cormier
  • Thierry Jacquin
  • Thomas Kachelhoffer
  • Thomas Ledoux
  • Tobias Schlauch
  • Vincenzo Forchi`
  • Wolfgang Pempe
    • 08:45 18:00
      Talks from various projects Auditorium

      Auditorium

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE

      Presentations by projects on their applications of the iRODS data grid. We would like to encourage presentations on client APIs that have been integrated with iRODS, rule sets that have been implemented, micro-services that have been developed.

      • 08:45
        Introduction 15m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: M. Jean-Yves Nief
        Slides
      • 09:00
        iRODS status. 45m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: Dr Reagan Moore
        Slides
      • 09:45
        Coffee break 30m Auditorium (CC-IN2P3)

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 10:15
        CC-IN2P3 director speech 15m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: Dr Dominique Boutigny
      • 10:30
        ARCS Data Fabric 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        The Australian Research Collaboration Service (ARCS) has implemented the Data Fabric using SRB. We are able to use Shibboleth authentication with the use of Short Lived Certificate Service (SLCS) to create GSI credentials. The SRB server is also modified such that we have automatic account creation for users from trusted Identity Providers (IdP). We have also built usage scripts that will keep a history if the federation’s usage. With the release of iRODS 2.0, the data fabric will now be migrated to iRODS. Initial evaluation of the migration process is now complete. We have been working with iRODS developers to enable automatic account creation in iRODS. As part of the migration process, we are moving our usage scripts written to iRODS. I will discuss our experience with the migration process. ARCS is also developing software for accessing the data fabric – Hermes and Davis. Hermes is a Java application designed to work with commons virtual file systems (commons-vfs), for which, new providers are added for SRB and iRODS. Davis is a Java servlet that uses the open standard WebDAV protocol. Client software is already part of many operating systems and this simplifies access to SRB and iRODS.
        Orateur: Mlle Pauline Mak
        Slides
      • 11:00
        ASPiS: integrating iRODS with Shibboleth and provenance engines 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Natively, iRODS addresses authentication by means of locally managed user lists or X509 certificates. The ASPiS project is integrating iRODS with Shibboleth, to enable authentication to be devolved onto the user’s home institution, and to suppport attribute/role-based authorisation decisions within the iRODS context. The project is also integrating iRODS with external provenance engines (PASOA and KARMA), to captute provenance data for scientific processes. Both integration approaches will make use of the rule system as far as possible.
        Orateur: M. Eric Liao
        Slides
      • 11:30
        Using iRODS with the EnginFrame grid portal into the GRIDA3 project 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        GRIDA 3 (Shared Resources Manager for Environmental Data Analysis and Applications) is an interdisciplinary project funded by the Italian Research Ministry. Primarily designed for providing a holistic description of environmental problems, the project will result in an advanced problem-solving tool for the integration, through a computing portal, of human know-how, simulation software, instrumentation and resources for data communication, storage, visualization and computation. A central part of the project is the creation of a Data Grid spanning multiple sites across federated domains, public and private: for the creation of this Data Grid we chose to use the iRODS technology. This talk will describe how we integrated iRODS with the EnginFrame Grid Portal, in order to provide applications with the ability to integrate iRODS easily and to expose to the users the access to the Data Grid resources in a intuitive and user-friendly way. In particular, we will demonstrate one of the GRIDA3 applications, showing how it can seamlessly use resources from the iRDOS data grid both as input and output, and how the user can access these resources from his web browser, upload new resources and so on.
        Orateurs: M. Francesco Locunto, M. Marco Piras
        Slides
      • 12:00
        SRB usage in BioEmergences 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        BioEmergences is an EC funded project where biologists, mathematicians, engineers and computer scientists define, implement and systematize the production of symbolic and precise reconstruction of cells evolution starting from in vivo captures of microscope images of embryos. The computational means for the project are provided by the different partners own computing facilities for initial runs of the algorithms. Afterwards the programs are moved to run under control of a workflow management application for systematic application to multiple datasets. The talk will briefly present the project, the workflow application. Then we will explain the differnt uses of SRB in the context of this project. Finally, we will conclude with a discussion on the potential migration to IRODS.
        Orateur: M. Dominique De Waleffe
        Paper
        Slides
      • 12:30
        Lunch 1h 30m Cantine (CC-IN2P3)

        Cantine

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 14:00
        Using Data Grids for Long Term Digital Preservation 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        In this talk we describe an FP7 integrated project focussed on long-term digital preservation called SHAMAN. We focus on the issues concerned with using data grids in long-term preservation.
        Orateur: Dr Adil Hasan
        Slides
      • 14:30
        The Storage Abstraction Service of the SPAR project 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        The National Library of France (BnF) is building its distributed archiving and preservation system (SPAR) in order to preserve in the long term all the digital information collected or created. This system is based on the OAIS standard but, in order to be independant of the underlying hardware infrastructure, a Storage Abstraction Service (SAS) is used. The SAS exposes its capabilities by the way of storage units, that represent some hardware designed to satisfy a given class of service, as well as records which abstracts the possible copies. In order to implement such Service, the choice of iRods has been made. In particular, a storage unit is seen as a particular resource associated with a set of irules to comply with the said class of service. In the presentation, we will show how such elements are defined and how the multiple operations needed for long-term preservation at the storage level can be achieved through the use of iRods.
        Orateur: M. Thomas Ledoux
        Slides
      • 15:00
        The Adonis research data preservation project for digital humanities in France 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        The aim of this project is to give to the french researchers on Humanities Sciences, a distributed data working space. On top of this, the project will be able to provide long term data preservation, to handle digital objects concepts, to provide data treatment facilities and some basic workflow mechanism. This activity is also connected to some European project as DARIAH. It will be currently based on two major software: iRods for the data manipulation at the file level and fedora-commons (version 3) at the digital object level. Two major french computing centers will be involved: the CINES for long term preservation and the CC-IN2P3 for data access. At this time, around twelve numerical resources centers, distributed in France, are identified to provide numerical data and high level data management for the overall Digital Humanities community.
        Orateur: M. Thomas Kachelhoffer
        Slides
      • 15:30
        iRODS as future data grid backend for TextGrid ? 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        TextGrid, which is part of the D-Grid initiative (http://www.d-grid.de), is the first project in the humanities in Germany creating a community grid for the collaborative editing, annotation, analysis, and publication of specialist text resources. The architecture of TextGrid enhances a Globus-based grid infrastructure with a specific middleware layer and an open, WebService-based service layer of specialised functionalities for textual processing. The current TextGrid data grid infrastructure is implemented by using the applicable components of Globus Toolkit 4. With regard to the coming next phase of the TextGrid project we consider the redesign of the existing storage infrastructure taking advantage of both Fedora, its Digital Object Model and the flexible, rule-based concept of iRODS.
        Orateur: M. Wolfgang Pempe
        Slides
      • 16:00
        Coffee break 30m Auditorium (CC-IN2P3)

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 16:30
        Collaborative data life-cycle management for petascale astronomy projects 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        In this brief talk, we look at the needs for Collaborative Data Life-cycle Management (CDLM), the advantages of CDLM and the concepts in iRODS that enable a large-scale CDLM infrastructure. We will highlight two astronomy projects as our usecase, namely LSST and ALMA. Both of these projects will have to manage several petabytes of data for several years from multiple independent agencies (or countries). iRODS was used by LSST in the Supercomputing 2008 HPC Storage Challenge. The objective of this talk is to introduce this problem and form a community of users who want to engineer solutions for similar large-scale data management problems.
        Orateur: M. Arun Jagatheesan
        Slides
      • 17:00
        Enabling a robust VOSpace based on iRODS 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        VOSpace is the International Virtual Observatory Alliance interface to distributed storage. It is the visible side of the storage system. To make a VOSpace usable in the real life we need an efficient storage mechanism. After a few experiences we have focused on iRODS which is a new data grid software system developed by the SDSC Storage Resource Broker team and collaborators. Our first aim was to create a storage area for Aladin but also for the new CDS Portal which is under development. In a first step we have developed an Aladin plugin giving an access to the iRODS implementation (through the Jargon Java API) and in a second step the VOSpace interface has been added over iRODS. We have developed a VOSpace Explorer in Java to access and manage the files. It is possible to do the common actions on the files. If a VO Tool supports drag and drop it is also possible to interact through this way with the explorer. iRODS is easy to implement and provides a good solution to ensure the robustness of a VOSpace. The installation is simple and can be done without much manpower. It is possible to start with a small configuration and to follow the evolution of the needs. A PLASTIC compliant tool like VOSpace Explorer is useful to provide a simple access to the stored files for VO Applications. As the main conclusion of this work we think that iRODS is a very good solution for the implementation of a robust VOSpace. And for many reasons (Open source, easy to use, flexible (definition of micro-services), follows the evolution of the architecture, etc.).
        Orateur: M. André Schaaff
        Slides
      • 17:30
        SRB service at STFC and the road to iRODS 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        STFC (Science and Technology Facilities Council, UK) is a long-time user of SRB in varied projects for internal and external customers alike. In this talk we describe how SRB has been used to meet the requirements of our main customers, and how the challenges associated with running a production service in this environment have been addressed. We also outline our plans for evaluation and migration to iRODS, noting which features we need to be present in order to migrate existing services without provoking a redesign of the infrastructure beneath.
        Orateurs: M. Kevin O'Neill, M. Roger Downing
        Slides
    • 09:00 18:00
      Tutorial Auditorium

      Auditorium

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE

      Given feedback on what participant needs, specific talks would be selected on the chosen topics for the tutorials. The talks will include live demos, exercises. The goal is to turn each talk into a discussion session on the subject.

      • 09:00
        Introduction 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Participants describe how they are using or intending to use iRODS and what they would like to know about the technology. We want feedback on which items are poorly understood, and which items need additional documentations.
        Orateur: Prof. Reagan Moore
        Paper
        Slides
      • 10:30
        Coffee break 30m Auditorium (CC-IN2P3)

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 11:00
        Hands-on demo: iRODS installation 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Installation of iRODS clients and server. Connection and exercises with the iRODS test bed build for the workshop.
        Orateur: Prof. Reagan Moore
        Slides
      • 12:30
        Lunch 2h Cantine (CC-IN2P3)

        Cantine

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 14:30
        Tutorial: introduction to rules 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: M. Michael Wan
        Slides
      • 16:00
        Coffee break 30m Auditorium (CC-IN2P3)

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 16:30
        Tutorial: assessment criteria 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: Prof. Reagan Moore
        Paper
    • 09:00 18:00
      Tutorial Auditorium

      Auditorium

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE

      Tutorial on iRODS on different topics depending upon participant needs. These topics can include: - iRODS installation, logical name spaces, iCAT metadata catalog, RDA interface to remote databases, data transfer (parallel I/O versus RBUDP), iRODS clients (Unix, web browser, Jargon), fuse client interface and performance, storage system drivers, structured information resource interface (tar files, mounted collection), rules (default rules), micro-services (default set), writing a micro-service from start to finish, HDF5 micro-services, web access micro-services, XML/XSLT micro-services, SRB to iRODS migration support, authentication (challenge-response versus GSI versus Shibboleth), planned development

      • 09:00
        Tutorial: advanced features, data transfer modes, structured file implementation, mounted collections, Fuse interface etc... 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: M. Michael Wan
        Slides
      • 10:30
        Coffee break 30m Auditorium (CC-IN2P3)

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 11:00
        Tutorial: using Remote Database Access (RDA) interface 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 12:30
        Lunch 2h Cantine (CC-IN2P3)

        Cantine

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 14:30
        Tutorial: writing and debugging a micro-service 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: M. Michael Wan
        Slides
      • 16:00
        Coffee break 30m Auditorium (CC-IN2P3)

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
      • 16:30
        Tutorial: writing and debugging a micro-service 1h 30m Auditorium

        Auditorium

        CC-IN2P3

        12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
        Orateur: M. Michael Wan
        Example
    • 09:00 10:30
      Parallel session: Astrophysics Auditorium

      Auditorium

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE

      Parallel sessions will be held so communities of interest can build collaborations.

      Minutes
    • 09:00 10:30
      Parallel session: Preservation environments Room 202

      Room 202

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE

      Parallel sessions will be held so communities of interest can build collaborations.

      Minutes
    • 10:30 11:00
      Coffee break 30m
    • 11:00 12:30
      Parallel session: Preservation environments and medical records Room 202

      Room 202

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE

      Parallel sessions will be held so communities of interest can build collaborations.

    • 12:30 14:30
      Lunch 2h Cantine

      Cantine

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE
    • 14:30 17:50
      Common session on GUIs, APIs, security etc... Room 202

      Room 202

      CC-IN2P3

      12-14 Bd Niels Bohr 69622 Villeurbanne Cedex FRANCE

      In this session, we will talk about topics of common interests where developments have been already made or are foreseen. These topics include GUIs (Hermes, Vbrowser, JUX, Windows explorer, web browsers), APIs (Java, Perl, Python, PHP), security (Shibboleth integration, GSI), management policies, specialized micro-services (web services, XML, image processing), migration from SRB to iRODS.

      • 14:30
        Davies and Hermes 15m
        Orateur: Mlle Pauline Mak
        Minutes
      • 14:45
        VBrowser 15m
        Orateur: M. Tristan Glatard
        Slides
      • 15:00
        JUX 15m
        Orateur: Dr Pascal Calvat
        Slides
      • 15:15
        Discussion on APIs, SRB to iRODS migration etc... 45m
      • 16:00
        Coffee break 30m