On Fri, May 06, 2011 at 09:15:37AM -0400, Vick Khera wrote: > > Oh... I remember erserver. It served us well for about 2 years for a > simple, not very high-velocity database that was 99.44% read-only. I > did have to monitor it closely and restart regularly. At least with > slony I don't worry about it crashing out from under me... :) <oldtimer style="doddering"> To be fair to it, erserver managed to live through the launch of the .info registry and its expansion, as well as the technical takeover of .org by Afilias. These were both fairly high-volume systems. It was plenty challenging to use for this arrangement, but it did manage to do the job. The big flaw in it was the decision on the part of the "architect" at the time to use Java. Java was picked because, well, because it said Enterprise in the name, as far as I could tell. This was made considerably worse by the decision to employ a programmer (who shall remain nameless) on the project who didn't know anything about Postgres. (He was gone from the project and from anything to do with the Afilias systems by the end of 2001, IIRC.) This was all to re-implement POC code that Vadim Mikheev had written, which was an extenion of the rserv code. That used named pipes, and the guy who got to make these decisions thought named pipes were for weenies. All the deep and clever tricks in erserver were ones Vadim wrote. (I suspect if you ask Jan, he'll tell you that some of the Slony code, especially in the way the triggers work, was also inspired by that code.) The fellow who reimplemented all this in Java desperately wanted to use multiple threads to get all the data, so he could run in parallel. (He had a mania for threads. He had an error reporting thread that was supposed to handle all exceptions. He stubbed it out and then never wrote the exception handler, which meant that every single exception you ever got from the code came from the same line number and said "Exception" but nothing more. This was extremely fun for debugging. To this day, whenever someone starts telling me that threads are the obvious solution to some problem, I want to quiz them really hard. I am pretty sure I can count in single digits the number of people I've ever met who actually know what they're doing with threads.) His plan, in fact, was to open one connection to the database for every table in the replication set. When I told him that this was a non-starter (we'd have used 50 or 60 connections just for this -- in the days when the JDBC driver would issue BEGIN and sit there like a dummy, blocking all your vacuums), he didn't revise his plan, so we ended up with a system that could in principle pull inconsistent views of the data into the replication engine. The way to fix this, as it turned out, was to force the connection pool only to use one connection. Anyway, because of this multithreading plan, he couldn't sort the data to be applied in a reasonable way in the database, so he'd pull all the pending data into the replication engine, then sort it, and then apply it on the slave. Well, no, actually. He'd sort and copy it, because if he had more than one slave the sort might have to be different. Or something -- I never actually understood the reason for this, and I stopped trying after all my teeth had to be replaced from the grinding. But it meant that if you got far enough behind, the replication engine would blow up because it ran out of memory it could allocate. IIRC, the VM was limited to 4G total. We had a large number of mitigation strategies we used to catch the possibility that the replication engine would get behind. I'm pretty sure it was the analysis of on-call billing hours that ultimately convinced management we needed to start over. I know that some people (I am among them) find Slony complicated and hard to manage. I can tell you, however, that compared to the stone tools we had for replicating in 2001, it was a dream. </oldtimer> A -- Andrew Sullivan ajs@xxxxxxxxxxxxxxx -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
