On Thu, 15 Aug 2013, Nulik Nol wrote: > Thanks, id didn't know about omap, but it is a good idea. I also found > that Eleanor Cawthon made a tree balancing project over OSDs. After > analyzing a bit more, I found that some librados and omap functions > aren't asynchronous. This is a considerable disadvantage when writing > a service where you expect high load. With synchronous calls I would > keep in wait queue a lot of client connections So, I think the best > solution in my case would be to write key/values in my own format over > OSDs storing the data in chunks (ceph objects) of say 64KB , like > 'pages' in a traditional DB engines. This way, it will be faster than > omap implementation and it also will work with asynchronous calls. Don't worry, everything can be done async. Any sync call you see in librados is just a wrapper around an async call. There are probably gaps in the aio_*() methods because those are also just wrappers aio_operate(), which submits an arbitrary transaction (of 1 or more operations). Look at librados.hpp for the most complete reference of what is supported. sage > > On Tue, Aug 13, 2013 at 6:09 PM, Samuel Just <sam.just@xxxxxxxxxxx> wrote: > > 2 is certainly an intriguing option. RADOS isn't really a database > > engine (even a nosql one), but should be able to serve your needs > > here. Have you seen the omap api available in librados? It allows > > you to efficiently store key/value pairs attached to a librados object > > (uses leveldb on the OSDs to actually handle the key/value mapping). > > > > One caveat is that the C api is somewhat less complete than the C++ > > api. That would be pretty easily remedied if there were demand > > though. > > -Sam > > > > On Tue, Aug 13, 2013 at 2:01 PM, Nulik Nol <nuliknol@xxxxxxxxx> wrote: > >> Hi, > >> I am planning to use Ceph as a database storage for a webmail > >> client/server application, and I am thinking to store the data as > >> key/value pair instead of using any RDBMSs, for speed. The webmail > >> will manage companies, and each company will have many users, users > >> will end/receive emails and store them in their inboxes, kind of like > >> Gmail, but per company. The server will be developed in C, client code > >> in HTML/Javascript and binary client (standalone app) in C++ > >> So, my question is, how would you recommend me to design the backend ? > >> > >> I have thought of these choices: > >> > >> 1. Use Ceph as filesystem and BerkeleyDB as the database engine. > >> Berekley DB uses 2 files per table, so I will have 1 directory per > >> company and a 2 files per each table, I think there will be no more > >> than 20 tables in my whole app. Ceph will be used here as a remote > >> filesystem where BerkeleyDB will do all the data organization. The > >> RADOS interface of Ceph (to store key/pair values) will be not used, > >> since Berkeley DB will write and read to the OSDs directly and > >> Berkeley DB is a key/value pair database. But I have never used a DB > >> one a remote filesystem not sure if it will work well. Advantages of > >> this architecture: quick & easy. > >> Disadvantages: lower performance (overhead in CephFS and BerkeleyDB), > >> also I will not be able to write plugins for RADOS in C++ to combine > >> many data modifications in a single call to the server. > >> > >> 2. Use librados C api and write all the 'queries' hardcoded in C > >> specifically for the > >> application. Since the application is pretty standard and is not > >> supposed to change > >> much, I can do this. I would create a RADOS object for each > >> application object (like for example 'user' record, 'email' record, > >> 'chat message' record, etc...). > >> Advantages: high performance. Disadvantages: a bit more to code , > >> specially the data search functions. > >> > >> I am interested in performance, so I am thinking to go for the option > >> 2, what do you think? Can RADOS fully replace a database engine ? (I > >> mean, NoSQL engine, like Berkeley for example) > >> > >> Will appreciate very much your comments. > >> TIA > >> Nulik > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users@xxxxxxxxxxxxxx > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |