On 13/02/2019 20:28, Michael Richardson wrote: > > Matt Caswell <matt@xxxxxxxxxxx> wrote: > > Please see my blog post for an OpenSSL 3.0 and FIPS Update: > > > https://www.openssl.org/blog/blog/2019/02/13/FIPS-update/ > > Thank you, it is very useful to have these plans made up front. > I think your posts should probably explain what happened to 2.x, and if this > represents a move towards semantic versioning. (I think it does...?) This is all explained in one of my previous blog posts. See: https://www.openssl.org/blog/blog/2018/11/28/version/ > > In the various things linked, in particular: > https://www.openssl.org/docs/OpenSSL300Design.html > > I think that there is a missing box. Specifically, the PERL API wrappers > that are used in the test bench. I believe that the "applications" are > a serious problem as there are (in 1.1.1) still many things that are very > difficult (sometimes, it seems, impossible) to do programmatically, and which > the test cases actually simply shell out to the application to do. > An example of this is adding certain extensions to a certificate when > generating it, which is only really possible by loading pieces of > configuration file in. > > So what I'd like to see is to remove many of the applications from the core > of OpenSSL, put them into a seperate package using better-documented API > calls. Let them evolve according their own time-scale, probably taking > patches faster without disrupting the underlying libraries. > > My observation is that the Perl testing system is used to drive the tests, > but the tests do not actually use the Perl API wrapper for OpenSSL, but > rather rely on the vast number of .c files in test/*. > In other (more purely agile) projects, the test cases often serve as > documentation as to how to use the API. In OpenSSL, the test cases > rely too much on the openssl "applications", and the API is hidden. > > This would involve adopting some or all of Net::SSLeay. > While there would be some initial duplication of effort, I think that over > time it would sort itself out. Perl is no longer as cool as it used to be (I > still like it) and maybe someone would argue for Python3 or something, and > frankly I don't care which. > > What I care about is that the test cases actually test the API, rather than > depend upon 20 years of twisty code in the "applications". > And that the applications are permitted to grow/change/adapt to people's > needs, rather than living in a hard spot between developer needs and end > user needs, pissing off both groups. I don't think it is accurate to characterise the tests as not directly testing the API but instead depending on the applications to do that. That *is* probably the case in many older tests but I don't recall many (any?) such tests being written in recent years. Instead there has been much effort put into directly testing the API (as an example see sslapitest.c which did not exist a few years ago (it doesn't appear in 1.0.2), but is currently over 6000 lines long). There are also the TLSProxy tests which do use s_server/s_client. But in those cases s_server/s_client are just used to drive a handshake. The tests themselves are actually written in perl. These are not API tests (so they don't depend on adding lots of obscure options to s_server/s_client) but are instead protocol tests. These tests modify the handshake in-flight to confirm that we can handle unusual or invalid protocol messages. Actually I would love to see the removal of the s_server/s_client dependency to something custom written. IMO the applications are no longer driven by developer needs. That may have once been the case, but I don't think it is true today. That said, of course, there is plenty of room for improvement in our testing. I would love to see more complete direct testing of the API. I do think we are moving in the right direction, but it is definitely a long term project. Matt -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
