ESAP Technical Discussion

lundi 25 oct. 2021, 16:00 → 17:00 Europe/Paris

Zoom

Escape ZoomRoom is inviting you to a scheduled Zoom meeting.

Meeting ID: 863 5651 9787 (https://astron.zoom.us/j/86356519787)
Passcode: (see e-mail, or ask in Rocket Chat #esap-interactive-analysis)
One tap mobile
+31207940854,,86356519787# Netherlands
+31207946519,,86356519787# Netherlands

Dial by your location
        +31 20 794 0854 Netherlands
        +31 20 794 6519 Netherlands
        +31 20 794 6520 Netherlands
        +31 20 794 7345 Netherlands
        +31 707 006 526 Netherlands
        +31 20 241 0288 Netherlands
Meeting ID: 863 5651 9787
Find your local number: https://astron.zoom.us/u/kbXjhufxPF

Join by SIP
86356519787@zoomcrc.com

Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 863 5651 9787

Join by Skype for Business
https://astron.zoom.us/skype/86356519787

16:00 → 17:00 Discussion

• Gareth, James, Stelios, Aard, Nico, Klaas
• Table round
  • Gareth:
    • Trying to initialize DIRAC from within ESAP; dirac proxy init.
    • “Kind of” running within ESAP.
    • https://indigo-iam.github.io/v/v1.7.1/docs/reference/api/scim-api/
    • Need to retrieve key after it has been initialized; Gareth is experimenting with third party libraries.
    • Added his notes to an issue: #69.
  • Stelios:
    • Created two issues related to VO querying.
      • Prototyping improved TAP interfaces (#70).
      • Experimenting with async queries (#71) in TAP; blocking on query result until it's available.
  • Aard:
    • Working on JIVE JupyterHub.
    • Want to submit notebooks to archive, issue DOIs.
    • Using a “gitea” install.
    • No live connection to OSSR as of now; maybe future work.
    • Possible token-handling overlap with Klaas.
  • Klaas:
    • Looking into JupyterHub authorization with IAM.
  • James:
    • Trying science data challenge workflow on DLaaS.
  • Nico:
    • Working on querying of multiple archives.
    • Needs frontend work, but the backend is pretty solid.
    • Thinking about ESAP-async plans.
• Execution Model
  • Deferred discussion on this because some key individuals were absent.
• Asynchronous ESAP
  • Design proposal
  • Nico: worried about adding complexity to the existing shopping basket. Experimented with this a couple of years ago using pure-Python microservices for ATDB. May not be the best idea for ESAP; RabbitMQ & Celery might work better.
  • Stelios: not clear how using a worker pool really helps — ESAP isn't under load while waiting for an asynchronous response.
    • A worker waiting for an async response could still block a worker.
    • John: we could address async responses by simply using the moral equivalent of select(2); the goal of this proposal, however, is to make all external resource access async.
  • Stelios: May be better to avoid conflating job control with the shopping basket.
  • Nico: Split it up into at least two parts — job mechanism, and shopping basket changes. Goal to come up with a plan for job scheduling this week.
  • Goals for next time:
    • Nico & Klaas (& maybe Stelio): come up with a job handling mechanism.
    • John & Stelios ensure wiki is up to date with design ideas.
  • Created epic &1 to track this work.