WP2 fortnightly meeting


ESCAPE WP2 Fortnightly 16th May 2019

Participants:  Simone, Gonzalo, Rosie, Aris, Mario, Xavier, Manuel, Tommaso, Ghita, Paul, Andrea, Simone, Frederic, Ron 

Intro :

  • WP2/WP5 joint meeting Amsterdam (1-3 July 2019) agenda is ready https://indico.in2p3.fr/event/19214/ (registrations not yet opened but timetable confirmed)
  • At the next WP2 fortnight meeting will dedicate time to discuss the topics for the face to face meeting in Amsterdam.

Datalakes ideas from FAIR (Kilian Schwarz)

  • The basic ideas is:
    • Global access point and redirect user by geo-IP to local endpoints
    • Squid proxy entry to data lake
    • Dynafed to aggregate different storages in one namespace and communicate via http to Dynafed
    • Sci-token are foreseen
    • Groningen site under discussion experience with iRODS for metadata, process of roll out the datalake idea
    • Auth server for sci-token and replication manager xrootd base framework


  • Q(Xavi): How the client based geo-ip mapping and Dynafed work together?
  • A(Kilian): Good point need to investigate how this two can work together… usage of squids might be useful.
  • Q(Paul): How this works?
  • Q(Mario): Done this in ATLAS, will circulate papers.
  • Q(Paul): How the user will authenticate in the Sci-Token server. Need to be an authentication: Open-ID connect or x509
  • A(Kilian): No idea yet. SAML is a possibility.
  • Q(Andrea):  What is the auth/authz model? by groups?
  • C(Simone):  WP2 a work package on AAI, need to have a coherent approach. Roles or capability based? etc. How flexible you are in using Sci-Tokens? and tools for data replication: FTS or equivalent third-party tools?
  • A(Kilian): Sci-Tokens is an idea as it seems to fulfil requirements but we are flexible. For the replication want to start with simple approach.
  • C(Mario): Implementing Dynafed on top means to limiting access to Webdav and S3.
  • A(Kilian): looking at xrootd forward proxy and allow xroot protocol
  • C(Paul): Usage of http in TPC transfers can be done in xroot (ie. FTS)
  • Q(Simone): Which sites are involved? 
  • A(Kilian): GSI, Groningen, SUT and KIT
  • Q(Simone): What about caching and latency hiding? which data access patterns you expect?
  • A(Kilian): We expect remote data access. System redirect the user to the closest data. Our system was the inspiration for XCache.
  • Q(Manuel): Different variation for http data access: suppose my data is somewhere we can only talk http and I want to access it via xrootd, then the cache need to talk http but serve it via xrootd.
  • A(Mario): Alistair suggested have permanent solution on cloud, go http and then Dynafed redirects and give another protocol. The other idea is do this client based: client requests a protocol translation to a service like FTS that will stream it with the correct protocols.
  • Q(Manuel): Instead of streaming we could think about pin things to the cache and when doing the pinning this is only http.
  • C(Paul): Theoretically possible and IIRC hinted in some presentations I’ve seen.
  • C(Mario): Remember meta-links to list all available protocols

Round table:

Update on the hiring process:

  • CERN: In the middle of hiring process, we will have a person on June 1st and one on July 1st if all goes well. 
  • CNAF: starting hiring senior fellow at CNAF for AAI
  • GSI: advertised the two slots they should be in the web soon, process is launched.
  • CCIN2P3): Still considering if we can work with someone that is already on site or open a temporary position.
  • SKA: Agreed two hires, signed paperwork WP2 and WP5. The post for networking is not yet advertised (12person/month)  as we want to make the networking post as a  full post, so there is a bit of delay about that.

Next meeting:  TBD as 30th of May is holiday at CERN, will distribute a doodle poll.

There are minutes attached to this event. Show them.
    • 10:00 AM 10:10 AM
      News 10m
      Speaker: Simone Campana (CERN)
    • 10:10 AM 10:40 AM
      Datalake ideas from FAIR 30m
      Speaker: Dr Kilian Schwarz
    • 10:40 AM 11:00 AM
      Round Table 20m