(Let's keep this on this list) On Thu, 13 Apr 2023 at 12:08, Alexander Saydakov <saydakov@xxxxxxxxxxxx> wrote: > > Yes, I am talking about a custom aggregate in C++ with internal state type. > You did not really answer my first question. Does the state always go through serialfinc-deserialfunc before reaching a combiner? Well, I think maybe you asked the question wrongly. The answer I gave was "No" because the serial and deserial functions are only used for internal typed aggregates. But if the question is "are serial and deserial functions always used for internal typed aggregates between the Partial and Finalize phase", the answer is "Yes", they are *currently*. I wouldn't want to rely on that staying true forever, however. I could think of a couple of reasons that this could change in the future: 1) Partition-wise aggregates don't really require it. Someone might submit a patch that allows the Partial Aggregate phase just to return a pointer to memory and have the Finalize Aggregate state just work on that pointer directly rather than having its own copy. 2) If we ever changed away from the process model into a threading model then we *may* consider not performing serialisation as an optimisation. Even if we ever did those 2, we might still need serial/deserial functions for sending the states to other servers. That's something we don't currently do as there's no SQL-level way to express "just give me the raw states and don't call the final functions". > The type "internal" in C++ is just a pointer. So it can be an arbitrary data structure. For some aggregations it would be better to have different state types in the first phase of the aggregation (processing raw data using state transition function) and the second phase (combining partial results). So I wonder if there is a clean separation between the phases: once partial aggregation is done the results go through serial-deserial barrier and only combining is done after that (sfunc never receives results of combining). If so, the question remains how to make finalfunc understand both states: directly from sfunc if there is no partial aggregation, and from the combiner. > Can a combiner receive results of another combiner? Can a combiner output also go through serial-deserial? You have to remember that the final func can be called without the state ever having gone through the combine func. This is what happens in non-parallel aggregation. Also, think of the case with > 2 parallel workers. There might be more than 2 states to combine for any given group. So the combine function must be able to operate on aggregate states that have already been combined from other states. You could just do something similar to how we handle NodeTag's in PostgreSQL. Effectively all Node typed structs have a NodeTag field at the start of the struct. This is just an enum that code can look at to determine the node type of whichever pointer it is looking at. Perhaps you can get away with coding your aggregate function's component functions in a way that can handle both types, you'd just need to look at the first 4 bytes of the pointer so you know what to do. In Postgres, we have an IsA macro to help us with that. Have a look at nodes.h. David
