I'll go ahead and ask, how long do you think such a back door would stay unnoticed, let alone survive? I'm considering the fact that we have a lot of people looking at our code, just judging from the issues and pull requests raised on github. I can't say that I have an actual answer, but it's a question worth asking as well, to see if the T.O.L.A. Act is worth a state of panic or not. Cheers, Richard ( who's on vacation and should stop reading these mails ) In message <11EDC036-87D3-4757-AD33-80C6E608F77F@xxxxxxxxxxxx> on Tue, 1 Jan 2019 01:27:53 +1100, "openssl@xxxxxxxxxxxx" <openssl@xxxxxxxxxxxx> said: > Matt et al > > 'been reviewed and approved by two current OpenSSL committers (one of whom must > be on the OpenSSL Management Committee).’ > > Due to the recent legislative changes here in Australia around the T.O.L.A. Act, can a change be > made to the OpenSSL policy so that the 2 reviewers, don’t reside in Australia, or are Australian > citizens ? > > ABI/API changes -> breaks -> back door requests…. > > -- > > Regards, > > Mark A. Lane > > Cryptopocalypse NOW 01 04 2016 > > Volumes 0.0 -> 10.0 Now available through iTunes - iBooks @ > https://itunes.apple.com/au/author/mark-a.-lane/id1100062966?mt=11 > > © Mark A. Lane 1980 - 2019, All Rights Reserved. > © FooCrypt 1980 - 2019, All Rights Reserved. > © FooCrypt, A Tale of Cynical Cyclical Encryption. 1980 - 2019, All Rights Reserved. > © Cryptopocalypse 1980 - 2019, All Rights Reserved. > > On 1 Jan 2019, at 01:11, Matt Caswell <matt@xxxxxxxxxxx> wrote: > > On 31/12/2018 11:36, C.Wehrmeyer wrote: > > On 31.12.18 10:12, Richard Levitte wrote: > > Yes, it's true, new features are going in. And it's true that it's > often more exciting to add new features than to do the janitorial > work. > > You realised what I have left unspoken thus far, which is this almost > obsession-like preference of OSS coders to add new features rather than > improving the old, boring codebase. However, there's a reason why it's still > called code*base*. It's the base of everything. And adding more and more > features to that base is going to make ripping off the band-aid more painful in > the long run. > > There has been a huge amount of effort put in over the last few years to improve > the codebase. Things that immediately spring to mind (and there's probably a > whole load more): > > - Rewrite of the state machine > - libssl record layer refactor > - Implementation of the PACKET and WPACKET abstractions in libssl > - Rewrite of the rand code > - Implementation of the new test harness > - Significant effort into developing tests > - Implementation of the coding style and reformat of the codebase to meet it > - Opaque many of the structures (which I know you don't see as an improvement, > but I'll answer that point separately) > - Implementation of continuous fuzzing > - Significant expansion of the documentation coverage > > It is simply not true to claim that we have "an obsession-like preference...to > add new features rather than improving the old, boring codebase". None of the > above things resulted in or were motivated by user visible features. They were > all about improving the codebase. > > Also, infrastructure again. I, as a user, don't care if the kernel gets a new > feature that makes some black magic happening. What I care about is that the > kernel doesn't throw away my writes (which has happened in May of 2018, see): > > https://www.postgresql.org/message-id/flat/CAMsr%2BYE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG%2BvzHun1EqMxyQg%40mail.gmail.com#CAMsr+YE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG+vzHun1EqMxyQg@xxxxxxxxxxxxxx) > > > Cryptography libs should be equally conservative, considering that cryptography > is conservative to begin with. I don't care if TLS 1.3 lets me use new exiting > ciphers and handshakes when it unreasonably bogs down my server code. > > BUT, you also have to appreciate that stuff is happening around us > that affects our focus. TLS 1.3 happened, and rather than having to > answer the question "why don't you have TLS 1.3 yet?" (there's a LOT > of interest in that version), we decided to add it. > > Sure, but didn't Matt just say that there are a lot of volunteers working on > that library? The disadvantage here is that quality assurance is barely a thing > - however, the *advantage* of this is that OpenSSL does not have to follow > commercial interests. If we look at this at face value you could just say "No, > people, it's high time we streamline some of the internal aspects of the > library, TLS 1.3 will have to wait. You can't wait that long? Well, sorry". > > However, your message is clear, we do need to do some cleanup as > well. More than that, I agree with you that it's needed (I've > screamed out in angst when stumbling upon particularly ugly or > misplaced code, so the feeling is shared, more than you might > believe). > > But what does "cleanup" entail? That's the hot-button question here. I've > already made a suggestion, that is to say, getting rid of opaque structures. If > that is deemed too insecure (for whatever reasons), export symbols that allow > programmers to query the size of structures, and provide two versions of > functions: one function expects the caller to pass an object to which the > changes are to be made, and the other one allocates the necessary memory > dynamically and then calls the first version. Or just don't allocate my object > memory dynamically anymore. > > That being said, cleanup happens, and documentation happens, in a > piecemeal fashion, 'cause that's what most people have capacity for. > > So, what you're effectively saying is that I'm the first one who ever asked for > SSL object reuse, right? Because if piecemeal work happens on the documentation, > and Viktor says that it's possible, then surely no one would have ever answered > that question on the mailing list and *not* put it piecemeal-ly in the OpenSSL > documentation, right? > > Now, here's something else that you need to consider: API/ABI > compatibility needs to be preserved. > > No it doesn't. We *know* it doesn't. When OpenSSL 1.1 was released it broke all > *sorts* of applications out there, because all sorts of applications used struct > fields rather than accessors. wget, mutt, neon, python, you name it, you broke it. > > API/ABI stability is absolutely required. Every time we make a breaking change > it is painful for our users - and more pain is felt the bigger the scale of the > break. We simply cannot go around making wholesale breaks on an ongoing basis. > If we did so then OpenSSL would be a lot less useful to our users. > > This is not to say that we can *never* make breaking changes. Only that when we > do so it must be strongly justified and only done relatively infrequently. We > made such a decision when we decided to make the structures opaque. It's not a > decision we are likely to repeat anytime soon IMO. We are still feeling the pain > of that now (and will continue to do so for at least the next year until 1.0.2 > goes out of support - and probably beyond that). > > Which brings me onto why structures were made opaque in the first place. A > significant driver for this (probably *the* most important one) was to improve > the codebase. I have witnessed first hand the harm that non-opaque structures > did to OpenSSL. We will be fixing the fallout from them for years to come. > Non-opaque structures combined with the requirements for stable API/ABI means > you cannot change anything in those structures. Renaming or deleting structure > members constitutes an API break. Even *adding* structure members constitutes an > ABI break (due to the changed size of the structure). This means the code > ossifies over time and cannot easily be refactored. Much of OpenSSL's internal > "quirkiness" results from attempting to work around this restriction. > > Things like the state machine refactor and the record layer refactor would not > have been possible without opaque structures. In my mind making the structures > opaque was one of the best things that ever happened to OpenSSL. > > https://breakpoint.cc/openssl-1.1-rebuild-2016-08-26/ > > So since when do we need to consider API/ABI compatibility? Did we grow up > recently? > > Or maybe OpenSSL should have switched the language. The point of C is that > structures are public. And if I'm going to be honest that approach saved my > sorry arse more than a couple times. When zlib choked because it couldn't go > past 4 GiBs of data since its fields were uint32_ts, I was able to easily > workaround this problem. But what do I know. > > If you really want to fiddle with OpenSSL internal structures - feel free. Just > include the OpenSSL internal header files and away you go. Just do so in the > knowledge that they could be changed at any time, and your code might break. If > this isn't a concern to you then - no problem. If it is a concern to you - then > actually you *do* care about API/ABI stability after all. > > Counting symbols is, however, nothing other than a blunt instrument. > Quite a lot of those symbols are convenience macros and functions that > have accumulated over time. > > You're taking my statement out of context. Counting the symbols wasn't supposed > to suggest that there are too *many* of them. I'm in no position to say that, > seeing as the original context in which my statement was put is that *I'm not > familiar enough with the library*. > > What I said was that reading the code is easy. Learning what the library > provides is hard, and that you won't learn much just by looking at the symbols > because there's so many of them. > > But nevertheless, I do hear you call for a remake of the SSL API as > well as cleaner internals. The latter is easier, and I'm sure it will > happen piecemeal as per usual so as to not break something / > inadvertently change a behavior (i.e. break ABI). The former is a > fairly massive project, and is more of creating a new API and library > rather than a mere cleanup job. That will be a massive effort, and > you do have to keep in mind how much time all involved can put into > it. > > I'm not saying it needs to be done right now. I'm merely suggesting that it > might be a good goal post for OpenSSL 2.0. > > Turning structures opaque doesn't prevent people from still messing > with their internal fields. > > True. But it makes for a clear delineation where people are forced to > be aware that they are playing with internal stuff, and that it may > not be a safe thing to do. > > Then why not provide small helper functions for covering the "playing with > internal stuff" part? That way it's still controlled, and documented, and > unified. You guys must've had some examples to show off in order to justify the > process, so surely you know what it is that people do when they use internal > stuff. Make functions for those. Don't give them any reason to continue playing > with internal stuff. > > I don't like code that tries to protect programmers from themselves. I like code > that lets good programmers do smart things. And if bad programmers use that > freedom to do bad stuff, then doesn't that mean your API simply didn't support > this, and they had to make it work somehow else? Again, helper functions. > > Uhmmmm.... this is factually incorrect. OpenSSL doesn't use its own > memory pooling. We have thin wrappers around the usual malloc() / > realloc() / free(), which allows any application to do its own memory > pooling. > > https://web.archive.org/web/20150207180717/http://article.gmane.org/gmane.os.openbsd.misc/211963 > > > The BUF_FREELISTS code that this post references was ripped out years ago. This > no longer represents the current state of OpenSSL in any supported version. > > To conclude, I have a question for you: are you only willing to rant > (*), or are you willing to help out in another way? > > This is not the question I feel you should ask because we haven't even > established if I *could* make contributions to the project, as my mindset > appears to be so much more different. Especially the idea of not wanting to > break APIs/ABIs is a huge limitation - just looking at SSL_new() made me give up > hope here. > > Yes - not breaking APIs/ABIs is a huge limitation. BoringSSL is not suitable for > general purpose use precisely because of this. The only users BoringSSL cares > about whether they break or not are Google users. As soon as you have a library > that wants to cater for large numbers of users (which we do) then you have to > accept that limitation. > > As to whether or not we have established whether you *could* make such > contributions - I think you are missing the point. We cannot know whether you > are capable or not until you try. It is on the basis of your code that we would > make such a judgement. In order for your code to get into OpenSSL it must have > been reviewed and approved by two current OpenSSL committers (one of whom must > be on the OpenSSL Management Committee). We invite anyone to contribute. In > order for this to be a healthy open-source community we *need* those > contributions. Only those that make the grade will make it in. > > Note - this review process wasn't always the case. Things used to be much more > informal in pre-heartbleed days. This is no longer the case. > > I'm no cryptography expert, I've made that clear from mail one, and my cleanup > jobs would be more widespread than what seems to be deemed acceptable right now. > I can read and write scalable C code, otherwise I wouldn't even have tried to > reuse that SSL object from the beginning. > > So, I ask a question in return: what do you think I *could* be helping with? > > Well, you have vocally complained about the state of the documentation. You have > the benefit of being a new OpenSSL user. You know what things were confusing or > unclear in the documentation. More experienced OpenSSL coders often don't have > the perspective - because some things are just "obvious" to them. So help with > pull requests to improve the documentation. > > Matt > -- > openssl-users mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users > -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
