Orateur
Description
Large EPICS installations often operate hundreds of soft IOCs distributed across multiple hosts. Managing these IOCs consistently—covering application creation, startup and shutdown, monitoring, validation, and upgrades—can become increasingly complex and error-prone as systems grow and standards evolve. To address these challenges at the Spallation Neutron Source, a new tool called IOCM has been developed.
IOCM is a command-line tool and supporting service for managing the full lifecycle of soft IOCs. It provides a unified interface for common operational tasks such as listing IOC status, starting and stopping IOCs across hosts, connecting to IOC consoles, validating applications against site standards, and locating IOCs by PV name. IOCM also supports standardized IOC application creation, including EPICS support module selection and configuration, with built-in checks to promote consistency and simplify long-term maintenance.
The tool integrates with procServ and systemd to enable reliable, reboot-safe IOC operation, while leveraging IOC statistics and filesystem conventions to provide both fleet-level visibility and detailed per-IOC diagnostics. IOCM has been deployed in production at SNS and is currently used to manage several hundred soft IOCs.
This presentation describes the motivation, architecture, and design of IOCM, demonstrates representative workflows, and discusses operational experience, including the use of IOCM to support upgrades to EPICS 7.