|
I work with Postgres and wonder whether for my purposes there is a good-enough reason to update one of these days. I’m an editor working with some 60,000 Early Modern texts, many of them in need of some editorial attention. The texts are XM encoded documents. Each word is wrapped in a <w> element with attributes for various
linguistic metadata. Typically a type of error occurs several or many times, and at the margins they need individual attention. I use Python scripts to extract stuff from the main corpus—sometimes dozens, sometimes thousands or millions—turn them into keyword
in contexts and import them into Postgres. I basically use Postgres as a giant spreadsheet. Its excellent string-handling routines make it relatively easy to to perform search and sort operations that identify tokens in need of correction. Once they corrections
are made in Postgres—typically as batch updates-- I move them as a data frame into Python, and from Python I move them back into the texts. I do this on a recent Mac with 64 GB of memory and a 6 cor i& processor. I use Data Studio as an editing interface. Unless a more recent version of Postgress has additional string handling routines, or indexing routines that speed up working with tables with rows in the low millions, or other features that are likely to
speed up operations, I don’t see any reasons to update. I could imagine a table that has up to 40 million rows. That would be pretty sluggish on my current equipment, which handles up to 10 million rows quite comfortably. A I right in thinking that given my tasks and equipment it would be a waste of time to update? Or is there something I’m missing? Martin Mueller Professor emeritus of English and Classiccs Northwestern University |
