Re: [HACKERS] Improve lseek scalability v3

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Sep 19, 2011 at 8:31 AM, Stephen Frost <sfrost@xxxxxxxxxxx> wrote:
> * Benjamin LaHaise (bcrl@xxxxxxxxx) wrote:
>> For such tables, can't Postgres track the size of the file internally?  I'm
>> assuming it's keeping file descriptors open on the tables it manages, in
>> which case when it writes to a file to extend it, the internally stored size
>> could be updated.  Not making a syscall at all would scale far better than
>> even a modified lseek() will perform.
>
> We'd have to have it in shared memory and have a lock around it, it
> wouldn't be cheap at all.

In theory, we could implement a lock-free cache.  But I still think it
would be better to see this fixed on the kernel side.  If we had some
evidence that all of those lseek() calls were a performance problem
even when the i_mutex is not seriously contended, then that would be a
good argument for doing this in user-space, but I haven't seen any
such evidence.  On the other hand, the numbers I posted show that when
i_mutex IS contended, it can cause a throughput regression of up to
90%.  That seems worth fixing.  If it turns out that lseek() is too
expensive even in the uncontended case or with the i_mutex contention
removed (or if the Linux community is unwilling to accept the proposed
fix), then we can (and should) look at further optimizing it within
PostgreSQL.  My guess, though, is that an unlocked lseek will be fast
enough that we won't need to worry about installing our own caching
infrastructure (or at least, there will be plenty of more significant
performance problems to hunt down first).

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]
  Powered by Linux