Re: Postgres as In-Memory Database?

[Stefan Keller <sfkeller@xxxxxxxxx>]

Hi,

I wrote

> Finally the paper is mostly about column stores - nothing about persistence. 

Regarding column store, Hadi wrote 2014-04-03 18:43 GMT+02:00 about the release of a PostgreSQL Columnar Store called "cstore_fdw" [1]!

@Hadi: Can you say something about usage of cstore FDW in-memory?

Regards, S.

[1] http://citusdata.com/blog/76-postgresql-columnar-store-for-analytics

2014-04-02 0:32 GMT+02:00 Stefan Keller <sfkeller@xxxxxxxxx>:

Hi Yeb

Thanks for the pointers.

Of course disk access is not obsolete: As I said, I suppose changes are streamed to disk. 

When I mentioned "no disk access" I meant the indices of RDBMS which designed to handle disk access - which seems to me different in in-memory dabases.

The paper referred by you is coming from SAP's chief scientist and it confirms actually my claim, that there's no need for a primary index since the primary attribute (i.e. all attributes) is already kept sorted in-memory.

It also mentions an insert-only technique: "This approach has been adopted before in POSTGRES [21] in 1987 and was called "time-travel". 

I would be interested what "time-travel" is and if this is still used by Postgres. 

Finally the paper is mostly about column stores - nothing about persistence. In mentions Disaster recovery" in the last section about future work, though. 

-S.

2014-04-01 21:57 GMT+02:00 Yeb Havinga <yebhavinga@xxxxxxxxx>:

On 2014-04-01 04:20, Jeff Janes wrote:

On Sunday, March 30, 2014, Stefan Keller <sfkeller@xxxxxxxxx> wrote:

Hi Jeff

2013/11/20 Jeff Janes <jeff.janes@xxxxxxxxx>

I don't know what you mean about enhancements in the buffer pool.  For an in-memory database, there shouldn't be a buffer pool in the first place, as it is *all* in memory.  

You are right: In-memory DBs are making buffer-pooling obsolete - except for making data persistent (see below).

I would be very reluctant to use any database engine which considered disk access obsolete.

The disk is not obsolete but something called 'anti-caching' is used:
http://www.vldb.org/pvldb/vol6/p1942-debrabant.pdf

 

Are there any show cases out there?

What did the HANA users have to say?  Seems like they would be in the best position to provide the test cases.

This paper provides some insights into the research behind HANA http://www.sigmod09.org/images/sigmod1ktp-plattner.pdf

regards
Yeb