Search Postgresql Archives

Re: TEXT column > 1Gb

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 4/12/23 08:59, Joe Carlson wrote:
I’m curious what you learned. I’ve been tripping over the buffer allocation issue when either splitting input text into chunks or aggregating chunks in selects. I’ve decided that I need to move this to client side.

The use case is genomics. Extracting substrings is common. So going to chunked storage makes sense.

And we have a menagerie of code to deal with. Legacy Perl for loading. Clients in Perl, Java, node, python. A whole zoo.

Thanks

My first stab was a java implementation based Selector class which worked ok using a fixed buffer size but finding the end of last transmission was hokey.  I move to an embedded Tomcat implementation and tomcat does the heavy lifting PERFECTLY.  Easily slipped in to both client and backend.

rjs






[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Postgresql Jobs]     [Postgresql Admin]     [Postgresql Performance]     [Linux Clusters]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Databases]     [Postgresql & PHP]     [Yosemite]

  Powered by Linux