On Tue, 2017-11-07 at 14:16 +0200, Amir Goldstein wrote:
> On Tue, Nov 7, 2017 at 1:54 PM, Jeff Layton <jlayton@xxxxxxxxxx> wrote:
> > On Tue, 2017-11-07 at 13:01 +0200, Amir Goldstein wrote:
> > > On Tue, Nov 7, 2017 at 12:11 PM, Raphael Hertzog <raphael@xxxxxxxxx> wrote:
> > > > Hello Amir,
> > > >
> > > > Le samedi 04 novembre 2017, Amir Goldstein a écrit :
> > > > > I tries mounting squashfs+overlayfs to /var/lib/postgresql and create
> > > > > db on Ubuntu and it seemed ok.
> > > >
> > > > FWIW, in my failing case, it uses PostgreSQL 10.0 as in Debian
> > > > Testing/Unstable. In Ubuntu, it's only available in Bionic Beaver (development
> > > > release).
> > >
> > > And is this the same PostgreSQL version that worked with kernel v4.12.6?
> > >
> > > [...]
> > >
> > > > As for strace output, postgresql is split over multiple processes. The one that
> > > > generates the error in the log is 31599 (checkpointer process). I also attach
> > > > some file listing of the directories that it fails to fsync. strace looks like
> > > > this (in loop):
> > > >
> > > > # strace -f -p 31599
> > > > select(0, NULL, NULL, NULL, {tv_sec=1, tv_usec=0}) = 0 (Timeout)
> > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> > > > open("pg_xact", O_RDONLY) = 3
> > > > fsync(3) = 0
> > > > close(3) = 0
> > > > open("pg_commit_ts", O_RDONLY) = 3
> > > > fsync(3) = -1 EINVAL (Invalid argument)
> > >
> > > The reason for the error is quite straight forward.
> > > open O_RDONLY gets an open file on lower read-only squashfs
> > > that doesn't have an fsync operation, so fsync returns EINVAL as per
> > > the man page documentation:
> > >
> > > EROFS, EINVAL
> > > fd is bound to a special file which does not support
> > > synchronization.
> > >
> >
> > If that's the case, then why didn't the fsync(3) call not return
> > EINVAL? Was it because it was copied up first?
>
> Allegedly yes.
> We see in ls -l at the end of report that file 0000 inside pg_xact
> mtime (Nov 7) is newer than squashfs mtime (Oct 30).
>
> >
> > If so, then maybe something changed in v4.13 to cause the pg_commit_ts
> > file
>
> Wait, I misread the information in the report and I wrongly assumed that
> pg_commit_ts is a file. It is not. it's a directory in which case, the
> inode is an
> overlay inode and it does have fsync f_op.
> But in the case of a lower directory that has no been copied up (which seems
> to be the case with pg_commit_ts) overlayfs will simple vfs_fsync_range the
> lower dir, so we are back to EINVAL.
>
> > to not have been be copied up here, when it would have before?
> >
>
> That is possible, but I would need more information about all the previous
> access to directory pg_commit_ts by postgresql to figure it out.
>
> Are there any aspects of fsync error reporting for directory fsync that
> we need to consider as leads to investigate?
>
> Amir.
At the VFS layer, we don't really make a big distinction between file
and dir inodes with fsync. If it has dirty data, it'll get synced out
either way.
If you think that the -EINVAL is getting stored and reported via the
inode's errseq_t, you can try enabling the file_check_and_advance_wb_err
and filemap_set_wb_err tracepoints to catch it. Those only fire when an
error is reported or recorded via that subsystem.
--
Jeff Layton <jlayton@xxxxxxxxxx>
--
To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
