On Tue, Sep 13, 2016 at 2:06 AM, Vinicius Segalin <vinisegalin@xxxxxxxxx> wrote: > 2016-09-12 18:22 GMT-03:00 Istvan Soos <istvan.soos@xxxxxxxxx>: >> At Heap we have non-trivial complexity in our analytical queries, and >> some of them can take a long time to complete. We did analyze features >> like the query planner's output, our query properties (type, >> parameters, complexity) and tried to automatically identify factors >> that contribute the most into the total query time. It turns out that >> you don't need to use machine learning for the basics, but at this >> point we were not aiming for predictions yet. > > And how did you do that? Manually analyzing some queries? In this case, it was automatic analysis and feature discovery. We were generating features out of our query parameters, out of the SQL string, and also out of the explain analyze output. For each of these features, we have examined the P(query is slow | feature is present), and measured its statistical properties (precision, recall, correlations...). With these we have built a decision tree-based partitioning, where our feature-predicates divided the queries into subsets. Such a tree could be used for predictions, or if we would like to be fancy, we could use the feature vectors to train a neural network. Hope this helps for now, Istvan -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
