-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On Tue, 2020-06-30 at 15:18 -0400, Ben Cotton wrote: > https://fedoraproject.org/wiki/Changes/SID > > == Summary == > Introduce Storage Instantiation Daemon (SID) that aims to provide a > central event-driven engine to write modules for identifying specific > Linux storage devices, their dependencies, collecting information and > state tracking while > being aware of device groups forming layers and layers forming whole > stacks or simply creating custom groups of enumerated devices. SID > will provide mechanisms to retrieve and query collected information > and a possibility to bind predefined or custom triggers with actions > for each group. > > == Owner == > * Name: [[User:prajnoha | Peter Rajnoha]] > * Email: prajnoha@xxxxxxxxxx > > == Detailed Description == > Over the years, various storage subsystems have been installing hooks > within udev rules and calling out numerous external commands for them > to be able to react on events like device presence, removal or a > change in general. However, this approach ended up with very complex > rules that are hard to maintain and debug if we are considering > storage setups where we build layers consisting of several underlying > devices (horizontal scope) and where we can stack one layer on top of > another (vertical scope), building up diverse storage stacks where we > also need to track progression of states either at device level or > group level. > > SID extends udevd functionality here in a way that it incorporates a > notion of device grouping directly in its core which helps with > tracking devices in storage subsystems like LVM, multipath, MD... > Also, it provides its own database where records are separated into > per-device, per-module, global or udev namespace. The udev namespace > keeps per-device records that are imported and/or exported to/from > udev environment and this is used as compatible communication channel > with udevd. The records can be marked with restriction flags that aid > record separation and it prevents other modules to read, write or > create a record with the same key, hence making sure that only a > single module can create the records with certain keys (reserving a > key). > > Currently, SID project provides a companion command called 'usid' > which is used for communication between udev and SID itself. After > calling the usid command in a udev rule, device processing is > transferred to SID and SID strictly separates the processing into > discrete phases (device identificaton, pre-scan, device scan, > post-scan). Within these phases, it is possible to decide whether the > next phase is executed and it is possible to schedule delayed actions > or set records in the database that can fire triggers with associated > actions or records which are then exported to udev environment > (mainly > for backwards compatibility and for other udev rules to have a chance > to react). The scheduled actions and triggers are executed out of > udev > context and hence not delaying the udev processing itself and > improving issues with udev timeouts where unnecessary work is done. > > A module writer can hook into the processing phases and use SID's API > to access the database as well as set the triggers with actions or > schedule separate actions and mark devices as ready or not for use in > next layers. The database can be used within any phase to retrieve > and > store key-value records (where value could be any binary value in > general) and the records can be marked as transient (only available > during processing phases for current event) or persistent so they can > be accessed while processing subsequent events. > > == Benefit to Fedora == > The main benefit is all about centralizing the solution to solve > issues that storage subsystem maintainers have been hitting with > udev, > that is: > > * providing a central infrastructure for storage event processing, > currently targeted at udev events > > * improving the way storage events and their sequences are recognized > and for which complex udev rules were applied before > > * single notion of device readiness shared among various storage > subsystems (single API to set the state instead of setting various > variables by different subsystems) > > * providing more enhanced possibilities to store and retrieve > storage-device-related records when compared to udev database > > * direct support for generic device grouping (matching > subsystem-related groups like LVM, multipath, MD... or creating > arbitrary groups of devices) > > * centralized solution for scheduling triggers with associated > actions > defined on groups of storage devices > > * adding a centralized solution for delayed actions on storage > devices > and groups of devices (avoiding unnecessary work done within udev > context and hence avoiding frequent udev timeouts when processing > events for such devices) Is this purely about adding some package into the repositories and just to raise awarness that such tool exist? > == Scope == > * Proposal owners: > ** complete SID's infrastructure to fully support stabilized API for > other developers to start writing modules for SID; > ** document all of current SID's functionality, including the module > API and explain the difference (extension) to udev, write and > complete > man pages; > ** provide udev rules responsible for communication with SID and > possibly importing records which were marked for export to udev in > SID. > > * Other developers: > ** first version will make use of a module to handle > device-mapper-multipath devices (device-mapper-multipath package) > ** consult in more detail possibility of adding an LVM module even > for > this release (if not feasible at this moment, then postpone > development of this module to next release) > > * Release engineering: [https://pagure.io/releng/issue/9568 #9568] > * Policies and guidelines: no changes needed at this moment > * Trademark approval: N/A (not needed for this Change) > > == Upgrade/compatibility impact == > We are introducing SID in this release and it will be disabled by > default so an upgrade has no impact here - all existing udev rules > for > various storage subsystems will still be installed as previously. > Subsystems for which a module will be already available in this > release will contain a switch in their udev rules to either use the > old udev rules (if SID is not active) or skip the rules appropriately > (if SID is active and related processing is handled within the SID > module instead). > > == How To Test == > * Basic testing involves (considering we have at least multipath > and/or LVM module present as well): > ** installing new 'sid' package > ** installing device-mapper-multipath and/or lvm module (presumably > named device-mapper-multipath-sid-module and lvm2-sid-module) > ** creating a device stack including device-mapper-multipath and/or > LVM volumes > ** booting with 'sid.enabled=1' kernel command line > ** checking device-mapper-multipath and/or LVM volumes are correctly > activated > > * More thorough testing: > ** (TBD) > > == User Experience == > Regular users shouldn't notice any change. SID is providing a > system-level infrastructure for convenient handling of > storage-device-related events through modules provided by other > developers. > > == Dependencies == > * a module to handle device-mapper-multipath is in cooperative > development with this change, the module will land in > device-mapper-multipath package (or its subpackage) > * the same applies for the LVM module (but that may be postponed as > described earlier) > > == Contingency Plan == > If SID is not complete in time, there's no need to execute any > special > backup plans. The distribution still contains all the original udev > rules to handle events for storage devices. If device-mapper- > multipath > and/or LVM provides the SID modules, these won't be built and > distributed. > > SID is not enabled by default so to start using it, one needs to > enable it explicitly. If enabling SID causes problems, it can be > disabled. For this purpose, there will be a kernel command line to > enable/disable SID so we avoid possible issues even at early boot > sequence if the device handled by SID is on critical path within boot > sequence. > > == Documentation == > * documentation (will be completed): https://sid-project.github.io/* > upstream repository: https://github.com/sid-project/sid-mvp > > > -- > Ben Cotton > He / Him / His > Senior Program Manager, Fedora & CentOS Stream > Red Hat > TZ=America/Indiana/Indianapolis > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx - -- Igor Raits <ignatenkobrain@xxxxxxxxxxxxxxxxx> -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEcwgJ58gsbV5f5dMcEV1auJxcHh4FAl77lBMACgkQEV1auJxc Hh7VAQ/7B+6trAXJOJSusP7XzXfcjtyfk4rxriiyccfHsqEriHnVhbpipWJ2TrVe YnOdvDK0+iU+PNgyZueXV4OBZWaQLw76f+HC+0IbWlYuejIWKmEu/DRxnQS0Iz+6 rjjvPI5Iqggvk9djezoF6Uix9f0I4CXisnZY3Rag3DvzD9Tx420A0iBRij3UeD1t w7KBcsrl8G4g5y+0RKI3voc5yuZj1EUBAa75dAE63BGRJTSmkP8zHA2JQM9WAhRR 1hKWW48cNag3USmveu/wIRp9SIpGPVo4xwiM18bga3jZuavIlmFZF52jb3TFASL9 seqD1MMO0gCj/12yeEHUzwSAtqMH204aqeBx5uymoY9JVclgnXds8nD72oZpr4YQ mF7FpOv3gxgfrP2l47mH3GeLtrbVxt/TA9DaDM6qVi6xZRRf0Pzi0tSS5mt/kO/n jLGeLNW9TrGtMPgWjhljNBIXmbiIzZhrfoUQd+ChfUM3E49eqz7D7+fgT3XQpXoB nGTUq/HCOpDKUZNX3hVPpTELX7zxA5oOiNcNm7gGkR7scgDmqhKQV8aWMPbJLbu5 qORm0n0ERi9BBD3/OtQee2WW61g6W0iLHMs2QtkFdYUu9jrdUtA2nrPC3Fsz1J8z 4yp3aLtPksX9E+Lc/9JfqB6WxLqu9Rx6jpD3+Og8vbU+e1nddzo= =m6Nr -----END PGP SIGNATURE----- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
