Re: 7 hrs for a pg_restore?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Feb 19, 2008 11:53 PM, Jeff Davis <pgsql@xxxxxxxxxxx> wrote:
>
>
> Keep in mind, if you have several GB worth of indexes, they take up
> basically no space in the logical dump (just the "CREATE INDEX" command,
> and that's it). But they can take a lot of processor time to build up
> again, especially with localized text.
>
>

I think it would be interesting if we can build these indexes in parallel.
Each index build requires a seq scan on the table. If the table does
not fit in shared buffers, each index build would most likely result
in lots of IO.

One option would be to add this facility to the backend so that multiple
indexes can be built with a single seq scan of the table. In theory, it
should be possible, but might be tricky given the way index build works
(it calls respective ambuild method to build the index which internally
does the seq scan).

Other option is to make pg_restore multi-threaded/processed. The
synchronized_scans facility would then synchronize the multiple heap
scans. ISTM that if we can make pg_restore mult-processed, then
we can possibly add more parallelism to the restore process.

My two cents.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB     http://www.enterprisedb.com

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

               http://archives.postgresql.org

[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Yosemite]

  Powered by Linux