3 - 6 months Happy New Year..;) > On 1 Jan 2019, at 02:43, Richard Levitte <levitte@xxxxxxxxxxx> wrote: > > I'll go ahead and ask, how long do you think such a back door would > stay unnoticed, let alone survive? I'm considering the fact that we > have a lot of people looking at our code, just judging from the issues > and pull requests raised on github. > > I can't say that I have an actual answer, but it's a question worth > asking as well, to see if the T.O.L.A. Act is worth a state of panic > or not. > > Cheers, > Richard ( who's on vacation and should stop reading these mails ) > > In message <11EDC036-87D3-4757-AD33-80C6E608F77F@xxxxxxxxxxxx> on Tue, 1 Jan 2019 01:27:53 +1100, "openssl@xxxxxxxxxxxx" <openssl@xxxxxxxxxxxx> said: > >> Matt et al >> >> 'been reviewed and approved by two current OpenSSL committers (one of whom must >> be on the OpenSSL Management Committee).’ >> >> Due to the recent legislative changes here in Australia around the T.O.L.A. Act, can a change be >> made to the OpenSSL policy so that the 2 reviewers, don’t reside in Australia, or are Australian >> citizens ? >> >> ABI/API changes -> breaks -> back door requests…. >> >> -- >> >> Regards, >> >> Mark A. Lane >> >> Cryptopocalypse NOW 01 04 2016 >> >> Volumes 0.0 -> 10.0 Now available through iTunes - iBooks @ >> https://itunes.apple.com/au/author/mark-a.-lane/id1100062966?mt=11 >> >> © Mark A. Lane 1980 - 2019, All Rights Reserved. >> © FooCrypt 1980 - 2019, All Rights Reserved. >> © FooCrypt, A Tale of Cynical Cyclical Encryption. 1980 - 2019, All Rights Reserved. >> © Cryptopocalypse 1980 - 2019, All Rights Reserved. >> >> On 1 Jan 2019, at 01:11, Matt Caswell <matt@xxxxxxxxxxx> wrote: >> >> On 31/12/2018 11:36, C.Wehrmeyer wrote: >> >> On 31.12.18 10:12, Richard Levitte wrote: >> >> Yes, it's true, new features are going in. And it's true that it's >> often more exciting to add new features than to do the janitorial >> work. >> >> You realised what I have left unspoken thus far, which is this almost >> obsession-like preference of OSS coders to add new features rather than >> improving the old, boring codebase. However, there's a reason why it's still >> called code*base*. It's the base of everything. And adding more and more >> features to that base is going to make ripping off the band-aid more painful in >> the long run. >> >> There has been a huge amount of effort put in over the last few years to improve >> the codebase. Things that immediately spring to mind (and there's probably a >> whole load more): >> >> - Rewrite of the state machine >> - libssl record layer refactor >> - Implementation of the PACKET and WPACKET abstractions in libssl >> - Rewrite of the rand code >> - Implementation of the new test harness >> - Significant effort into developing tests >> - Implementation of the coding style and reformat of the codebase to meet it >> - Opaque many of the structures (which I know you don't see as an improvement, >> but I'll answer that point separately) >> - Implementation of continuous fuzzing >> - Significant expansion of the documentation coverage >> >> It is simply not true to claim that we have "an obsession-like preference...to >> add new features rather than improving the old, boring codebase". None of the >> above things resulted in or were motivated by user visible features. They were >> all about improving the codebase. >> >> Also, infrastructure again. I, as a user, don't care if the kernel gets a new >> feature that makes some black magic happening. What I care about is that the >> kernel doesn't throw away my writes (which has happened in May of 2018, see): >> >> https://www.postgresql.org/message-id/flat/CAMsr%2BYE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG%2BvzHun1EqMxyQg%40mail.gmail.com#CAMsr+YE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG+vzHun1EqMxyQg@xxxxxxxxxxxxxx) >> >> >> Cryptography libs should be equally conservative, considering that cryptography >> is conservative to begin with. I don't care if TLS 1.3 lets me use new exiting >> ciphers and handshakes when it unreasonably bogs down my server code. >> >> BUT, you also have to appreciate that stuff is happening around us >> that affects our focus. TLS 1.3 happened, and rather than having to >> answer the question "why don't you have TLS 1.3 yet?" (there's a LOT >> of interest in that version), we decided to add it. >> >> Sure, but didn't Matt just say that there are a lot of volunteers working on >> that library? The disadvantage here is that quality assurance is barely a thing >> - however, the *advantage* of this is that OpenSSL does not have to follow >> commercial interests. If we look at this at face value you could just say "No, >> people, it's high time we streamline some of the internal aspects of the >> library, TLS 1.3 will have to wait. You can't wait that long? Well, sorry". >> >> However, your message is clear, we do need to do some cleanup as >> well. More than that, I agree with you that it's needed (I've >> screamed out in angst when stumbling upon particularly ugly or >> misplaced code, so the feeling is shared, more than you might >> believe). >> >> But what does "cleanup" entail? That's the hot-button question here. I've >> already made a suggestion, that is to say, getting rid of opaque structures. If >> that is deemed too insecure (for whatever reasons), export symbols that allow >> programmers to query the size of structures, and provide two versions of >> functions: one function expects the caller to pass an object to which the >> changes are to be made, and the other one allocates the necessary memory >> dynamically and then calls the first version. Or just don't allocate my object >> memory dynamically anymore. >> >> That being said, cleanup happens, and documentation happens, in a >> piecemeal fashion, 'cause that's what most people have capacity for. >> >> So, what you're effectively saying is that I'm the first one who ever asked for >> SSL object reuse, right? Because if piecemeal work happens on the documentation, >> and Viktor says that it's possible, then surely no one would have ever answered >> that question on the mailing list and *not* put it piecemeal-ly in the OpenSSL >> documentation, right? >> >> Now, here's something else that you need to consider: API/ABI >> compatibility needs to be preserved. >> >> No it doesn't. We *know* it doesn't. When OpenSSL 1.1 was released it broke all >> *sorts* of applications out there, because all sorts of applications used struct >> fields rather than accessors. wget, mutt, neon, python, you name it, you broke it. >> >> API/ABI stability is absolutely required. Every time we make a breaking change >> it is painful for our users - and more pain is felt the bigger the scale of the >> break. We simply cannot go around making wholesale breaks on an ongoing basis. >> If we did so then OpenSSL would be a lot less useful to our users. >> >> This is not to say that we can *never* make breaking changes. Only that when we >> do so it must be strongly justified and only done relatively infrequently. We >> made such a decision when we decided to make the structures opaque. It's not a >> decision we are likely to repeat anytime soon IMO. We are still feeling the pain >> of that now (and will continue to do so for at least the next year until 1.0.2 >> goes out of support - and probably beyond that). >> >> Which brings me onto why structures were made opaque in the first place. A >> significant driver for this (probably *the* most important one) was to improve >> the codebase. I have witnessed first hand the harm that non-opaque structures >> did to OpenSSL. We will be fixing the fallout from them for years to come. >> Non-opaque structures combined with the requirements for stable API/ABI means >> you cannot change anything in those structures. Renaming or deleting structure >> members constitutes an API break. Even *adding* structure members constitutes an >> ABI break (due to the changed size of the structure). This means the code >> ossifies over time and cannot easily be refactored. Much of OpenSSL's internal >> "quirkiness" results from attempting to work around this restriction. >> >> Things like the state machine refactor and the record layer refactor would not >> have been possible without opaque structures. In my mind making the structures >> opaque was one of the best things that ever happened to OpenSSL. >> >> https://breakpoint.cc/openssl-1.1-rebuild-2016-08-26/ >> >> So since when do we need to consider API/ABI compatibility? Did we grow up >> recently? >> >> Or maybe OpenSSL should have switched the language. The point of C is that >> structures are public. And if I'm going to be honest that approach saved my >> sorry arse more than a couple times. When zlib choked because it couldn't go >> past 4 GiBs of data since its fields were uint32_ts, I was able to easily >> workaround this problem. But what do I know. >> >> If you really want to fiddle with OpenSSL internal structures - feel free. Just >> include the OpenSSL internal header files and away you go. Just do so in the >> knowledge that they could be changed at any time, and your code might break. If >> this isn't a concern to you then - no problem. If it is a concern to you - then >> actually you *do* care about API/ABI stability after all. >> >> Counting symbols is, however, nothing other than a blunt instrument. >> Quite a lot of those symbols are convenience macros and functions that >> have accumulated over time. >> >> You're taking my statement out of context. Counting the symbols wasn't supposed >> to suggest that there are too *many* of them. I'm in no position to say that, >> seeing as the original context in which my statement was put is that *I'm not >> familiar enough with the library*. >> >> What I said was that reading the code is easy. Learning what the library >> provides is hard, and that you won't learn much just by looking at the symbols >> because there's so many of them. >> >> But nevertheless, I do hear you call for a remake of the SSL API as >> well as cleaner internals. The latter is easier, and I'm sure it will >> happen piecemeal as per usual so as to not break something / >> inadvertently change a behavior (i.e. break ABI). The former is a >> fairly massive project, and is more of creating a new API and library >> rather than a mere cleanup job. That will be a massive effort, and >> you do have to keep in mind how much time all involved can put into >> it. >> >> I'm not saying it needs to be done right now. I'm merely suggesting that it >> might be a good goal post for OpenSSL 2.0. >> >> Turning structures opaque doesn't prevent people from still messing >> with their internal fields. >> >> True. But it makes for a clear delineation where people are forced to >> be aware that they are playing with internal stuff, and that it may >> not be a safe thing to do. >> >> Then why not provide small helper functions for covering the "playing with >> internal stuff" part? That way it's still controlled, and documented, and >> unified. You guys must've had some examples to show off in order to justify the >> process, so surely you know what it is that people do when they use internal >> stuff. Make functions for those. Don't give them any reason to continue playing >> with internal stuff. >> >> I don't like code that tries to protect programmers from themselves. I like code >> that lets good programmers do smart things. And if bad programmers use that >> freedom to do bad stuff, then doesn't that mean your API simply didn't support >> this, and they had to make it work somehow else? Again, helper functions. >> >> Uhmmmm.... this is factually incorrect. OpenSSL doesn't use its own >> memory pooling. We have thin wrappers around the usual malloc() / >> realloc() / free(), which allows any application to do its own memory >> pooling. >> >> https://web.archive.org/web/20150207180717/http://article.gmane.org/gmane.os.openbsd.misc/211963 >> >> >> The BUF_FREELISTS code that this post references was ripped out years ago. This >> no longer represents the current state of OpenSSL in any supported version. >> >> To conclude, I have a question for you: are you only willing to rant >> (*), or are you willing to help out in another way? >> >> This is not the question I feel you should ask because we haven't even >> established if I *could* make contributions to the project, as my mindset >> appears to be so much more different. Especially the idea of not wanting to >> break APIs/ABIs is a huge limitation - just looking at SSL_new() made me give up >> hope here. >> >> Yes - not breaking APIs/ABIs is a huge limitation. BoringSSL is not suitable for >> general purpose use precisely because of this. The only users BoringSSL cares >> about whether they break or not are Google users. As soon as you have a library >> that wants to cater for large numbers of users (which we do) then you have to >> accept that limitation. >> >> As to whether or not we have established whether you *could* make such >> contributions - I think you are missing the point. We cannot know whether you >> are capable or not until you try. It is on the basis of your code that we would >> make such a judgement. In order for your code to get into OpenSSL it must have >> been reviewed and approved by two current OpenSSL committers (one of whom must >> be on the OpenSSL Management Committee). We invite anyone to contribute. In >> order for this to be a healthy open-source community we *need* those >> contributions. Only those that make the grade will make it in. >> >> Note - this review process wasn't always the case. Things used to be much more >> informal in pre-heartbleed days. This is no longer the case. >> >> I'm no cryptography expert, I've made that clear from mail one, and my cleanup >> jobs would be more widespread than what seems to be deemed acceptable right now. >> I can read and write scalable C code, otherwise I wouldn't even have tried to >> reuse that SSL object from the beginning. >> >> So, I ask a question in return: what do you think I *could* be helping with? >> >> Well, you have vocally complained about the state of the documentation. You have >> the benefit of being a new OpenSSL user. You know what things were confusing or >> unclear in the documentation. More experienced OpenSSL coders often don't have >> the perspective - because some things are just "obvious" to them. So help with >> pull requests to improve the documentation. >> >> Matt >> -- >> openssl-users mailing list >> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users >> > -- > openssl-users mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
