On Thu, Aug 20, 2009 at 2:45 AM, Merlin Moncure<mmoncure@xxxxxxxxx> wrote: > > I think your first step is to stand back, think about the problem as a > whole and see if you can't define your requirements better: > > *) how much data comes in each day? how much rolls out? Need to get some data...will have this tomorrow. > *) what types of analysis of the data would you like to do? Basically, what I need to do is feed the tick data to the CEP engine as a data stream. The CEP engine creates various data bars streams say 2 minute OHLC (open/high/low/close), 3 min, 5 min, 15 min, 30 min or 20 tick, 2000 tick, 5000 tick etc. & so on. These are to be fed to strategies which will trade. At least this is how it is done if data arrives from the online data provider. I am sure that this could be done on database. I had originally thought of populated views of prebuilt data bars so building the bars should not be required. They should be built using triggers as soon as the data arrives in postgres, but then everyone advised that it was exactly due to slow results in this type of data that CEP engines were built. I was told that if I used this type of thing, so many triggers would be running that I would have my processors and RAM maxing out, not to say the exponential increase in disk space requirements. And this is for one symbol. On an average 2000 plus scrips need to be analysed. Again the analysis is backtesting trade simulation results according to various strategies, so that means at least three years plus data, if one wants to be a stickler 5-6 years but one year is the bare minimum. > *) what are your performance expectations? > Well, needfully the simulations should finish overnight with three years worth of data. > For example, try and describe what you would like to do in detail. > The data you would import, how you would like it structured, and a > typical query. While the amount of price data a stock market produces > is immense, these types of problems tend to scale very well > horizontally. So let's define the basic amount of work you'd like to > do, then build off that. > I'll get back on this by tomorrow. Thanks. With best regards. Sanjay. -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
