Re: Increasing parallelism of queries while using file fdw and partitions

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Justin,

Thanks for the idea. I pulled down the source for multicorn and added that to it. I do not see parallel queries in the analyze output (unless I force it and then it only gets one worker), but it does look like it is reading more than one file at once if I go with a non-partitioned table that looks at all the files. Not any better if I have the table split up into partitions. 

So it’s better, but still curious if this would work with partitions. 

Thanks again. 

Pat

> On Dec 18, 2018, at 22:51, Justin Pryzby <pryzby@xxxxxxxxxxxxx> wrote:
> 
>> On Tue, Dec 18, 2018 at 08:39:36PM -0800, Patrick Mulrooney wrote:
>> Wondering if anyone had any thoughts on how to tweak my setup to get it to
>> read many files at once instead of one at a time when using file fdw and
>> partitions.
> 
> I found this:
> 
> https://www.postgresql.org/docs/current/parallel-safety.html
> |The following operations are always parallel restricted.
> |Scans of foreign tables, unless the foreign data wrapper has an IsForeignScanParallelSafe API which indicates otherwise.
> 
> https://github.com/ergo70/vcf_fdw/blob/master/vcf_fdw/__init__.py
> => has no such API marker, since it's couple years old, same as multicorn.
> 
> Justin





[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Yosemite]

  Powered by Linux