On 11/29/2017 11:34 AM, Stephen John Smoogen wrote: > In security, no one is trusted. I am not trusted, Kevin is not > trusted, Patrick is not trusted. Dennis and Mohan are not trusted. > This is because we are foolable and fallible. Even worse our accounts > are not ourselves and could be used to look like us but not be us. > This gives us each a potential reliability of 0.8 and a combined > reliability of 0.1 (aka 90% of the time our root powers combined will > cause things to be worse.) > > Second our infrastructure is very much an organically grown ball of > kerosene soaked yarn. You get root on box A and magically you can > affect something on box C because someone decided that <<fill in funny > named service of the week>> needed to be used for our brand new << > fill in feature that we try in Fedora N and shoved aside in N+2 >> and > both systems use it. And we can't remove that feature or tool because > oh someone built it into koji or fas or some other critical tool to > cut down development time because it needed to be 'working' now. [Most > of our services are 0.95 reliable but combined in their myriad ways it > is probably a combined total of 0.4 -> 0.6. In another words 40-60% of > the time something is not working right for someone] > > The two systems you are wanting root access to are some of the most > interconnected in the kerosene level. Everyone who does have root on > them has accidentally mangled all of Fedora in some way for several > hours or days because we were trying to fix something and broke > something else. So let's develop a set of sudo rules so I can only run certain commands? [I think Dennis made archives completely unreadable > twice and I moved all of them to ... using a script which had been > looked at by two other sysadmins before I ran it. Those were the easy > ones to undo.. there have been others which took going through backups > to fix.] > > It would be nice if those boxes weren't the worst but every time we > redesign things to be saner, we end up with a << must have this new > service installed or Fedora will FAIL >> that can only meet the > release deadline by being put on them. We try to clean up before the > next release but oh look another << if Fedora doesn't have this > feature it will FAIL >> comes up. > > And when things fail, those of us in root are the ones who > collectively get blamed for it. Possible blame is definitely something I worry about when requesting access like this. I do propose that I work with some bumpers (i.e. only doing things after I get someone to review it, announcing things i'm going to do). Basically I would be in FBR mode all the time, except after I get reviewed I can actually do the work myself. Why did you let smooge have rights to > the system? Why did you not check his actions before he did them. How > did you not catch a ... in that line of code? It may all sound like > reasonable questions but to the people who are dealing with the > problem it gets picked up as "How did you let this moron ever do > this?" And because things fail so much.. it eventually comes across as > "You people are completely inept". > > All of this: > * High chance that someone as root is going to make a mistake and the > more people who have it more likely. > * High chance that some systems will catch fire through all the rest > of infrastructure > * Perceived blame > > makes that adding anyone else a very difficult process. We can be > prickly because of 3 and should mitigate it, but the real problem are > the first two in the list. > yes, hopefully the infra becomes less kerosene over time, but I'd like to be part of the solution there rather than just someone waiting on it to happen. I'm mostly here to offer help and also deliver/release atomic host every two weeks. Often times this means I find issues before anyone else does (because I'm releasing tomorrow and not 6 months from now). This translates mostly into: I help fix problems before other people even notice they are problems. This translates into: I think I do more good than harm. The jury is out on that, though. Dusty _______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
