| Matt et al
'been reviewed and approved by two current OpenSSL committers (one of whom must
be on the OpenSSL Management Committee).’
Due to the recent legislative changes here in Australia around the T.O.L.A. Act, can a change be made to the OpenSSL policy so that the 2 reviewers, don’t reside in Australia, or are Australian citizens ?
ABI/API changes -> breaks -> back door requests….
-- Regards, Mark A. Lane Cryptopocalypse NOW 01 04 2016 Volumes 0.0 -> 10.0 Now available through iTunes - iBooks @ https://itunes.apple.com/au/author/mark-a.-lane/id1100062966?mt=11© Mark A. Lane 1980 - 2019, All Rights Reserved. © FooCrypt 1980 - 2019, All Rights Reserved. © FooCrypt, A Tale of Cynical Cyclical Encryption. 1980 - 2019, All Rights Reserved. © Cryptopocalypse 1980 - 2019, All Rights Reserved.
On 31/12/2018 11:36, C.Wehrmeyer wrote:On 31.12.18 10:12, Richard Levitte wrote:
Yes, it's true, new features are going in. And it's true that it's
often more exciting to add new features than to do the janitorial
work.
You realised what I have left unspoken thus far, which is this almost
obsession-like preference of OSS coders to add new features rather than
improving the old, boring codebase. However, there's a reason why it's still
called code*base*. It's the base of everything. And adding more and more
features to that base is going to make ripping off the band-aid more painful in
the long run.
There has been a huge amount of effort put in over the last few years to improvethe codebase. Things that immediately spring to mind (and there's probably awhole load more):- Rewrite of the state machine- libssl record layer refactor- Implementation of the PACKET and WPACKET abstractions in libssl- Rewrite of the rand code- Implementation of the new test harness- Significant effort into developing tests- Implementation of the coding style and reformat of the codebase to meet it- Opaque many of the structures (which I know you don't see as an improvement,but I'll answer that point separately)- Implementation of continuous fuzzing- Significant expansion of the documentation coverageIt is simply not true to claim that we have "an obsession-like preference...toadd new features rather than improving the old, boring codebase". None of theabove things resulted in or were motivated by user visible features. They wereall about improving the codebase.
Also, infrastructure again. I, as a user, don't care if the kernel gets a new
feature that makes some black magic happening. What I care about is that the
kernel doesn't throw away my writes (which has happened in May of 2018, see):
https://www.postgresql.org/message-id/flat/CAMsr%2BYE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG%2BvzHun1EqMxyQg%40mail.gmail.com#CAMsr+YE5Gs9iPqw2mQ6OHt1aC5Qk5EuBFCyG+vzHun1EqMxyQg@xxxxxxxxxxxxxx)
Cryptography libs should be equally conservative, considering that cryptography
is conservative to begin with. I don't care if TLS 1.3 lets me use new exiting
ciphers and handshakes when it unreasonably bogs down my server code.
BUT, you also have to appreciate that stuff is happening around us
that affects our focus. TLS 1.3 happened, and rather than having to
answer the question "why don't you have TLS 1.3 yet?" (there's a LOT
of interest in that version), we decided to add it.
Sure, but didn't Matt just say that there are a lot of volunteers working on
that library? The disadvantage here is that quality assurance is barely a thing
- however, the *advantage* of this is that OpenSSL does not have to follow
commercial interests. If we look at this at face value you could just say "No,
people, it's high time we streamline some of the internal aspects of the
library, TLS 1.3 will have to wait. You can't wait that long? Well, sorry".
However, your message is clear, we do need to do some cleanup as
well. More than that, I agree with you that it's needed (I've
screamed out in angst when stumbling upon particularly ugly or
misplaced code, so the feeling is shared, more than you might
believe).
But what does "cleanup" entail? That's the hot-button question here. I've
already made a suggestion, that is to say, getting rid of opaque structures. If
that is deemed too insecure (for whatever reasons), export symbols that allow
programmers to query the size of structures, and provide two versions of
functions: one function expects the caller to pass an object to which the
changes are to be made, and the other one allocates the necessary memory
dynamically and then calls the first version. Or just don't allocate my object
memory dynamically anymore.
That being said, cleanup happens, and documentation happens, in a
piecemeal fashion, 'cause that's what most people have capacity for.
So, what you're effectively saying is that I'm the first one who ever asked for
SSL object reuse, right? Because if piecemeal work happens on the documentation,
and Viktor says that it's possible, then surely no one would have ever answered
that question on the mailing list and *not* put it piecemeal-ly in the OpenSSL
documentation, right?
Now, here's something else that you need to consider: API/ABI
compatibility needs to be preserved.
No it doesn't. We *know* it doesn't. When OpenSSL 1.1 was released it broke all
*sorts* of applications out there, because all sorts of applications used struct
fields rather than accessors. wget, mutt, neon, python, you name it, you broke it.
API/ABI stability is absolutely required. Every time we make a breaking changeit is painful for our users - and more pain is felt the bigger the scale of thebreak. We simply cannot go around making wholesale breaks on an ongoing basis.If we did so then OpenSSL would be a lot less useful to our users.This is not to say that we can *never* make breaking changes. Only that when wedo so it must be strongly justified and only done relatively infrequently. Wemade such a decision when we decided to make the structures opaque. It's not adecision we are likely to repeat anytime soon IMO. We are still feeling the painof that now (and will continue to do so for at least the next year until 1.0.2goes out of support - and probably beyond that).Which brings me onto why structures were made opaque in the first place. Asignificant driver for this (probably *the* most important one) was to improvethe codebase. I have witnessed first hand the harm that non-opaque structuresdid to OpenSSL. We will be fixing the fallout from them for years to come.Non-opaque structures combined with the requirements for stable API/ABI meansyou cannot change anything in those structures. Renaming or deleting structuremembers constitutes an API break. Even *adding* structure members constitutes anABI break (due to the changed size of the structure). This means the codeossifies over time and cannot easily be refactored. Much of OpenSSL's internal"quirkiness" results from attempting to work around this restriction.Things like the state machine refactor and the record layer refactor would nothave been possible without opaque structures. In my mind making the structuresopaque was one of the best things that ever happened to OpenSSL.
https://breakpoint.cc/openssl-1.1-rebuild-2016-08-26/
So since when do we need to consider API/ABI compatibility? Did we grow up
recently?
Or maybe OpenSSL should have switched the language. The point of C is that
structures are public. And if I'm going to be honest that approach saved my
sorry arse more than a couple times. When zlib choked because it couldn't go
past 4 GiBs of data since its fields were uint32_ts, I was able to easily
workaround this problem. But what do I know.
If you really want to fiddle with OpenSSL internal structures - feel free. Justinclude the OpenSSL internal header files and away you go. Just do so in theknowledge that they could be changed at any time, and your code might break. Ifthis isn't a concern to you then - no problem. If it is a concern to you - thenactually you *do* care about API/ABI stability after all.
Counting symbols is, however, nothing other than a blunt instrument.
Quite a lot of those symbols are convenience macros and functions that
have accumulated over time.
You're taking my statement out of context. Counting the symbols wasn't supposed
to suggest that there are too *many* of them. I'm in no position to say that,
seeing as the original context in which my statement was put is that *I'm not
familiar enough with the library*.
What I said was that reading the code is easy. Learning what the library
provides is hard, and that you won't learn much just by looking at the symbols
because there's so many of them.
But nevertheless, I do hear you call for a remake of the SSL API as
well as cleaner internals. The latter is easier, and I'm sure it will
happen piecemeal as per usual so as to not break something /
inadvertently change a behavior (i.e. break ABI). The former is a
fairly massive project, and is more of creating a new API and library
rather than a mere cleanup job. That will be a massive effort, and
you do have to keep in mind how much time all involved can put into
it.
I'm not saying it needs to be done right now. I'm merely suggesting that it
might be a good goal post for OpenSSL 2.0.
Turning structures opaque doesn't prevent people from still messing
with their internal fields.
True. But it makes for a clear delineation where people are forced to
be aware that they are playing with internal stuff, and that it may
not be a safe thing to do.
Then why not provide small helper functions for covering the "playing with
internal stuff" part? That way it's still controlled, and documented, and
unified. You guys must've had some examples to show off in order to justify the
process, so surely you know what it is that people do when they use internal
stuff. Make functions for those. Don't give them any reason to continue playing
with internal stuff.
I don't like code that tries to protect programmers from themselves. I like code
that lets good programmers do smart things. And if bad programmers use that
freedom to do bad stuff, then doesn't that mean your API simply didn't support
this, and they had to make it work somehow else? Again, helper functions.
Uhmmmm.... this is factually incorrect. OpenSSL doesn't use its own
memory pooling. We have thin wrappers around the usual malloc() /
realloc() / free(), which allows any application to do its own memory
pooling.
https://web.archive.org/web/20150207180717/http://article.gmane.org/gmane.os.openbsd.misc/211963
The BUF_FREELISTS code that this post references was ripped out years ago. Thisno longer represents the current state of OpenSSL in any supported version.To conclude, I have a question for you: are you only willing to rant
(*), or are you willing to help out in another way?
This is not the question I feel you should ask because we haven't even
established if I *could* make contributions to the project, as my mindset
appears to be so much more different. Especially the idea of not wanting to
break APIs/ABIs is a huge limitation - just looking at SSL_new() made me give up
hope here.
Yes - not breaking APIs/ABIs is a huge limitation. BoringSSL is not suitable forgeneral purpose use precisely because of this. The only users BoringSSL caresabout whether they break or not are Google users. As soon as you have a librarythat wants to cater for large numbers of users (which we do) then you have toaccept that limitation.As to whether or not we have established whether you *could* make suchcontributions - I think you are missing the point. We cannot know whether youare capable or not until you try. It is on the basis of your code that we wouldmake such a judgement. In order for your code to get into OpenSSL it must havebeen reviewed and approved by two current OpenSSL committers (one of whom mustbe on the OpenSSL Management Committee). We invite anyone to contribute. Inorder for this to be a healthy open-source community we *need* thosecontributions. Only those that make the grade will make it in.Note - this review process wasn't always the case. Things used to be much moreinformal in pre-heartbleed days. This is no longer the case.
I'm no cryptography expert, I've made that clear from mail one, and my cleanup
jobs would be more widespread than what seems to be deemed acceptable right now.
I can read and write scalable C code, otherwise I wouldn't even have tried to
reuse that SSL object from the beginning.
So, I ask a question in return: what do you think I *could* be helping with?
Well, you have vocally complained about the state of the documentation. You havethe benefit of being a new OpenSSL user. You know what things were confusing orunclear in the documentation. More experienced OpenSSL coders often don't havethe perspective - because some things are just "obvious" to them. So help withpull requests to improve the documentation.Matt-- openssl-users mailing listTo unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
|
