Bi-Weekly Datalake DepOps meeting (Rosie chairing)

Europe/Paris
Description

Weekly meeting to discuss progress on EDLK JIRA issues: https://jira.skatelescope.org/issues/?filter=15115

Zoom room: https://skatelescope.zoom.us/j/97713259777?pwd=Q2EwSWZ3NkRaazFRSy9YT3Y5UmdJZz09

    • 10:00 10:30
      Hot topics

      1) Rucio metadata / additional functionalities ahead of DAC21

      • we need to get confirmation which rucio functionalities are to be in what release, and which releases we target for each rucio instance - Rizart (TBC) to follow-up on this

      2) Infrastructure tests - automated tests continue to run on ESCAPE datalake, some via ESCAPE rucio, but others (FTS, gfal) just directly between RSEs. We need to check it's OK to keep these running for the next 12 weeks

      • PerfSONAR dashboards update (https://monit-grafana.cern.ch/d/TdKEjHCWz/escape-mesh-config-dashboard?orgId=51) - sites are hard-coded so we need to check these are correct before DAC21.

      3) Rucio-level monitoring for ESFRI rucio instances

      • ESCAPE main rucio instance (https://monit-grafana.cern.ch/d/4rmQfGYMz/rucio-events?orgId=51),
      • MAGIC+CTA rucio instance (need a link?)
      • SKA rucio (need a link, existing dashboard not working)
    • 10:30 10:40
      Datalake health

      https://monit-grafana.cern.ch/d/qByShefGk/fts-transfers?orgId=51

      FTS dash show some sites not working as destination 

    • 10:40 11:00
      AOB

      Andrea - working on fine-grained auth test suite. Using CRIC to fetch end points.

      github.com/indigo-iam/escape-auth-tests

      By-passes rucio, directly getting tokens from IAM for use at storage end points.

      HA IAM k8s cluster, now available - will test first with mock instance