Re: FolderDB - advice wanted

[Sam Hobbs <Sam@xxxxxxxxxxxxxxxxxx>]

Start with a requirements definition. Write it as non-technical as possible; I say that because things like:

• pure PHP
• not even using the object oriented functionality
• file format

Should not be part of a requirements definition unless there is a true requirement. You will need a clear requirements definition for many reasons, especially if you want others to help.

You should evaluate database systems to anticipate as much as possible what your requirements are. A database system is the type of thing that can waste substantial amounts of time if the requirements are not clear and accurate from the beginning. If you do develop something and then realize (a year from now or two years) that you should have done it differently then perhaps that will at least have taught you the value of spending time on the requirements definition in the beginning. Databases have many features that you might not have thought about.

Rene Veerman

Sunday, July 21, 2019 8:54 AM

Hi.

I like JSON a lot better than the SQL data model,
but now that i've been using it for a few years,
i find myself combining file system storage of lots of JSON files in many sub-folders into a web app,
like news gathering with commandline PHP per N minutes from 280+ RSS online sources,
into a PHP web-app that loads the latest news, via a web-powered PHP filesystem search of the latest new-items JSON files, stored on a filesystem thats got folders like ./YYYY/MM/DD/hh/mm/ss/CATEGORY_SUBMENU_PATH leading to JSON files.

so now i want something more flexible. something that can give me a web-based treeview of data spanning many JSON-like files (in more complicated ways than my news app) and even into other file formats stored somewhere on the filesystem of some server.

i plan to license this right from the start under a LGPL typed license,
because i believe input from other developers is key to making a DB system that is not only easy to use, but also very efficient to run.

i'd like the code to stay pure PHP, not even using the object oriented functionality of PHP, just plain properly named functions and their return values.

i'll name this DB architecture FolderDB, and to prevent web-apps from forcing the server to reload (and decode!) many .jsone files upon a simple request to change something in some referenced .jsone file, i plan to run a single PHP server daemon that will respond to a yet to be written ASCII/UNICODE protocol.

the file format itself, has to differ from JSON somewhat i think, and i also believe we can lose some of the overhead.
i post this all here to hear your opinion, especially when it points to insurmountable problems related to my designs.

i'm also interested to learn about how to add cloud redundancy to this DB architecture, preferably with tiered backup servers, or at least be prepared for such expansions of the DB architecture later on.

here's the string based format that i came up with so far :

JSON-Extended file format (ASCII / UNICODE)

------------------
{
    key1 : s|string value,
    key2 : 2,
    key3 : 3.4,
    key4a : t,
    key4b : f,
    key5 : [
        numbered array
    ],
    key6 : {
        named array
    },
    key7 : url|/siteData/bla/bla/file.jsone,
    key8 : key|key7/someKeyInsideFileBehindURLofKey7_key1/key1a,
    key9 : url|/siteMedia/backgrounds/someFile.png||thumb|/siteMedia/backgrounds/thumbs/someFile.png,
    key10 : url|/siteData/bla2/bla2a/file.odt||thumb|/siteData/bla2/bla2a/file.odt.thumb.png||thumbGenerator|URL
}
------------------

key4a = true
key4b = false

key1 would need it's value escaped, using \| and \,