Bi-Weekly Datalake DepOps meeting (Marek chairing)

Europe/Paris
Marek Szuba (GSI)
Description

Weekly meeting to discuss progress on EDLK JIRA issues: https://jira.skatelescope.org/issues/?filter=15115

Zoom room: https://skatelescope.zoom.us/j/97713259777?pwd=Q2EwSWZ3NkRaazFRSy9YT3Y5UmdJZz09

HOT TOPICS

1. Trimming the scopes

Riccardo: we now have 67 scopes in Rucio, some of whom do not follow the agreed naming convention. This complicates/unnecessarily stresses monitoring (Rizart's elaboration: we currently match scopes to experiments using '<EXPERIMENT_NAME>*' wildcards). Started a deletion campaign; users are requested to set expiration date on rules associated with to-be-deleted scopes to avoid the presence of "dark data" in the DL (Rucio does not support deletion of scopes so it must be done at the database level, which could result in inconsistencies).

ACTION ITEM (Riccardo): E-mail creators of to-be-deleted scopes.

Paul M./Rosie/Andrea: Should ask Rucio developers to add support for scope deletion, as the ATLAS "add scopes infrequently and never delete them" model does not have to apply to everyone. To begin with it Rucio could refuse deletion of scopes which still have DIDs attached, in the long run one might consider cascaded deletion.

ACTION ITEM (Paul M.): Create an issue on GitHub Done: https://github.com/rucio/rucio/issues/4974

2. MaxSpace

Rizart: RSE admins, please set MaxSpace in CRIC so that reaper can work on your sites; see RocketChat for who still needs this. Everyone should have appropriate CRIC permissions by now, if not ask for them in #cric-updates

3. OIDC support in ESCAPE Rucio:

Status report (Rizart):

 - REST API continues to work with manually acquired tokens
 - CLI issues should be fixed in the next release, ETA ~1 week
 - Web UI downgraded to an older version due to OIDC-related issues in the newer one, should be fine

DATALAKE HEALTH

1. Severe FTS failures for INFN-ROMA1, IFN-NA-DPM-FED, INFN-NA-DPM, LAPP-WEBDAV. Mostly but not exclusively auth issues. Sadly, none of the representatives present in the meeting.

ALPAMED-DPM is yellow, also auth issues. Again, representatives not present.

ACTION ITEM (Marek): create JIRA tickets for these problems.

2. Andrea has shared link to INFN Robot Framework dashboard, Rizart will add it to our list. Note that some of the test failures might be false positives for now.

3. Paul M.: DESY does not currently write data to tape (everything gets routed to dCache instead) following issues found in tape-management software. Will hopefully be addressed soon but no guarantees, i.e. tape may or may not be available during the DAC. Therefore, use cases should not rely on "written to tape" data status.

4. Marek: Observed that changing auth_type in rucio.cfg does not invalidate the current Rucio authentication token, i.e. issues with the new configuration might take up to 1 hour to appear. Paul M. suggests reporting this to Rucio developers, as it might trip up people attempting to switch from VOMS to OIDC authentication

ACTION ITEM (Marek): Report the aforementioned issue upstream Done: https://github.com/rucio/rucio/issues/4975

AOB

Maisam: could really use (better) documentation for datasets/containers etc. in CLI!
Rizart: suggest joining Rucio support channel on Slack (rucio.slack.com)
 

Il y a un compte-rendu associé à cet événement. Les afficher.
    • Hot topics
    • Datalake health
    • AOB