> nothing SQL-compatible is going to do the job.
> bincode, msgpack, protobuf, and of course, IETF's CBOR RFC7049. Also JSON.
i've had experience with protobuf, json, (plus json-schema and graphql).
as far as serializer/reviver ergonomics:
1.
json is idiot-proof for non-relational data. json-schema is soso for relational-data.
2.
protobuf and graphql are likely idiot-proof *if* you work at google/facebook (and access to internal-tooling). but my [frontend] experience with their opensource serializers/revivers are disappointing and error-prone.
3.
wasm-sqlite's serializers/revivers are idiot-proof for relational-data:
```_javascript_
// webassembly serializer:
FS.readFile("/sqlite-database.db") // return <serialized Uint8Array>
// webassembly reviver:
var dbRevived;
var ptr = stackAlloc(4);
FS.createDataFile("/", "sqlite-database.db", <serialized Uint8Array>, true, true);
sqlite3_open("sqlite-database.db", ptr);
dbRevived = getValue(ptr, "i32"); // revived sqlite-database (in virtual fs)
FS.readFile("/sqlite-database.db") // return <serialized Uint8Array>
// webassembly reviver:
var dbRevived;
var ptr = stackAlloc(4);
FS.createDataFile("/", "sqlite-database.db", <serialized Uint8Array>, true, true);
sqlite3_open("sqlite-database.db", ptr);
dbRevived = getValue(ptr, "i32"); // revived sqlite-database (in virtual fs)
...
sqlite3_exec(dbRevived, "SELECT * FROM my_table;", <callback>, 0, ptr);
sqlite3_exec(dbRevived, "SELECT * FROM my_table;", <callback>, 0, ptr);
...
```
On Sat, Nov 28, 2020 at 4:34 AM John C Klensin <john-ietf@xxxxxxx> wrote:
--On Saturday, November 28, 2020 01:14 -0500 Michael Richardson
<mcr+ietf@xxxxxxxxxxxx> wrote:
>
> It's an interesting idea.
>
> There are quite a number of other serialization formats out
> there including: bincode, msgpack, protobuf, and of course,
> IETF's CBOR RFC7049. Also JSON. For "opendata" we've would up
> with CSV, which I really dislike. {So funny that the UK's data
> problem with COVID rates was due to some data flow that went
> CSV->XLS->database, fixed when they went CSV->XLSX->database,
> when there was never a reason to use XLS* at all. So perhaps
> this argues your case}
>
> The question you likely need to answer are:
>
> 1) do the sqlite people wish to turn change control over to
> the IETF?
>
> 2) Given the Library of Congress statement, is there actually
> a net benefit to the world? Maybe it's already moving to
> defacto standard anyway. Would the IETF process help?
> Given the stability, would there be any benefit for IANA?
>
> 2B) There may be NAFTA Article 10 ("Performance
> Specification") reasons for having an RFP'able
> specification.
>
> 3) Are there enough implementers who are not sqlite.org who
> would benefit? Is the wasm-sqlite from sqlite.org (I don't
> know).
>
> I looked through the specification, and turning it into an RFC
> wouldn't be hard. You may wish to look at the work in the
> CELLAR WG, which is doing something similar for archival
> formats for AV.
I have not studied or used sqlite, but, stepping back from the
details...
Let me add something from a very different perspective -- maybe
useful, maybe not. If you are looking for a data serialization
and interchange standpoint _and_ expect it to be able to
accommodate the full range of statistical and scientific
databases rather than only nicely rectangular, row-oriented,
commercial databases and things that behave like them, nothing
SQL-compatible is going to do the job.
To save typing in a longer list, several of the papers in
Rafanelli, M., Klensin, J.C., and Svensson, P., eds.,
_Statistical and Scientific Database Management: Fourth
International Working Conference on Statistical and Scientific
Database Management, Rome, Italy, June 1988, Proceedings_,
Springer-Verlag Lecture Notes in Computer Science #339, 1989.
(And, yes, for those who don't know, I had a life or three
before getting sucked back, almost fully, into Internet work.)
best,
john
On Sat, Nov 28, 2020 at 4:34 AM John C Klensin <john-ietf@xxxxxxx> wrote:
--On Saturday, November 28, 2020 01:14 -0500 Michael Richardson
<mcr+ietf@xxxxxxxxxxxx> wrote:
>
> It's an interesting idea.
>
> There are quite a number of other serialization formats out
> there including: bincode, msgpack, protobuf, and of course,
> IETF's CBOR RFC7049. Also JSON. For "opendata" we've would up
> with CSV, which I really dislike. {So funny that the UK's data
> problem with COVID rates was due to some data flow that went
> CSV->XLS->database, fixed when they went CSV->XLSX->database,
> when there was never a reason to use XLS* at all. So perhaps
> this argues your case}
>
> The question you likely need to answer are:
>
> 1) do the sqlite people wish to turn change control over to
> the IETF?
>
> 2) Given the Library of Congress statement, is there actually
> a net benefit to the world? Maybe it's already moving to
> defacto standard anyway. Would the IETF process help?
> Given the stability, would there be any benefit for IANA?
>
> 2B) There may be NAFTA Article 10 ("Performance
> Specification") reasons for having an RFP'able
> specification.
>
> 3) Are there enough implementers who are not sqlite.org who
> would benefit? Is the wasm-sqlite from sqlite.org (I don't
> know).
>
> I looked through the specification, and turning it into an RFC
> wouldn't be hard. You may wish to look at the work in the
> CELLAR WG, which is doing something similar for archival
> formats for AV.
I have not studied or used sqlite, but, stepping back from the
details...
Let me add something from a very different perspective -- maybe
useful, maybe not. If you are looking for a data serialization
and interchange standpoint _and_ expect it to be able to
accommodate the full range of statistical and scientific
databases rather than only nicely rectangular, row-oriented,
commercial databases and things that behave like them, nothing
SQL-compatible is going to do the job.
To save typing in a longer list, several of the papers in
Rafanelli, M., Klensin, J.C., and Svensson, P., eds.,
_Statistical and Scientific Database Management: Fourth
International Working Conference on Statistical and Scientific
Database Management, Rome, Italy, June 1988, Proceedings_,
Springer-Verlag Lecture Notes in Computer Science #339, 1989.
(And, yes, for those who don't know, I had a life or three
before getting sucked back, almost fully, into Internet work.)
best,
john
