On Fri, Oct 14, 2022 at 03:15:33PM +1100, Donald Douwsma wrote: > > > On 14/10/2022 14:37, Donald Douwsma wrote: > > > > > > On 14/10/2022 06:05, Darrick J. Wong wrote: > > > On Thu, Oct 13, 2022 at 02:15:16PM +1100, Donald Douwsma wrote: > > > > xfsdump can create multiple streams, when restoring the online inventory > > > > with multiple streams we fail to process these and assert when the > > > > inventory buffer is not fully decoded. > > > > > > > > [root@rhel8 ~]# xfsdump -L "Test1" -f /dev/nst0 -M tape1 -f > > > > /dev/nst1 -M tape2 /boot > > > > xfsdump: using scsi tape (drive_scsitape) strategy > > > > xfsdump: using scsi tape (drive_scsitape) strategy > > > > xfsdump: version 3.1.8 (dump format 3.0) - type ^C for status > > > > and control > > > > xfsdump: level 0 dump of rhel8:/boot > > > > xfsdump: dump date: Thu Oct 6 13:50:45 2022 > > > > xfsdump: session id: aa25fa48-4493-45c7-9027-61e53e486445 > > > > xfsdump: session label: "Test1" > > > > xfsdump: ino map phase 1: constructing initial dump list > > > > xfsdump: ino map phase 2: skipping (no pruning necessary) > > > > xfsdump: ino map phase 3: identifying stream starting points > > > > xfsdump: stream 0: ino 133 offset 0 to ino 28839 offset 0 > > > > xfsdump: stream 1: ino 28839 offset 0 to end > > > > xfsdump: ino map construction complete > > > > xfsdump: estimated dump size: 328720704 bytes > > > > xfsdump: estimated dump size per stream: 164375728 bytes > > > > xfsdump: /var/lib/xfsdump/inventory created > > > > xfsdump: drive 0: preparing drive > > > > xfsdump: drive 1: preparing drive > > > > xfsdump: drive 1: creating dump session media file 0 (media 0, file 0) > > > > xfsdump: drive 1: dumping ino map > > > > xfsdump: drive 1: dumping non-directory files > > > > xfsdump: drive 0: creating dump session media file 0 (media 0, file 0) > > > > xfsdump: drive 0: dumping ino map > > > > xfsdump: drive 0: dumping directories > > > > xfsdump: drive 0: dumping non-directory files > > > > xfsdump: drive 1: ending media file > > > > xfsdump: drive 1: media file size 166723584 bytes > > > > xfsdump: drive 1: waiting for synchronized session inventory dump > > > > xfsdump: drive 0: ending media file > > > > xfsdump: drive 0: media file size 165675008 bytes > > > > xfsdump: drive 0: waiting for synchronized session inventory dump > > > > xfsdump: drive 0: dumping session inventory > > > > xfsdump: drive 0: beginning inventory media file > > > > xfsdump: drive 0: media file 1 (media 0, file 1) > > > > xfsdump: drive 0: ending inventory media file > > > > xfsdump: drive 0: inventory media file size 2097152 bytes > > > > xfsdump: drive 0: writing stream terminator > > > > xfsdump: drive 0: beginning media stream terminator > > > > xfsdump: drive 0: media file 2 (media 0, file 2) > > > > xfsdump: drive 0: ending media stream terminator > > > > xfsdump: drive 0: media stream terminator size 1048576 bytes > > > > xfsdump: drive 1: dumping session inventory > > > > xfsdump: drive 1: beginning inventory media file > > > > xfsdump: drive 1: media file 1 (media 0, file 1) > > > > xfsdump: drive 1: ending inventory media file > > > > xfsdump: drive 1: inventory media file size 2097152 bytes > > > > xfsdump: drive 1: writing stream terminator > > > > xfsdump: drive 1: beginning media stream terminator > > > > xfsdump: drive 1: media file 2 (media 0, file 2) > > > > xfsdump: drive 1: ending media stream terminator > > > > xfsdump: drive 1: media stream terminator size 1048576 bytes > > > > xfsdump: dump size (non-dir files) : 328189016 bytes > > > > xfsdump: dump complete: 4 seconds elapsed > > > > xfsdump: Dump Summary: > > > > xfsdump: stream 0 /dev/nst0 OK (success) > > > > xfsdump: stream 1 /dev/nst1 OK (success) > > > > xfsdump: Dump Status: SUCCESS > > > > [root@rhel8 ~]# xfsdump -I > > > > file system 0: > > > > fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 > > > > session 0: > > > > mount point: rhel8:/boot > > > > device: rhel8:/dev/sda1 > > > > time: Thu Oct 6 13:50:45 2022 > > > > session label: "Test1" > > > > session id: aa25fa48-4493-45c7-9027-61e53e486445 > > > > level: 0 > > > > resumed: NO > > > > subtree: NO > > > > streams: 2 > > > > stream 0: > > > > pathname: /dev/nst0 > > > > start: ino 133 offset 0 > > > > end: ino 28839 offset 0 > > > > interrupted: NO > > > > media files: 2 > > > > media file 0: > > > > mfile index: 0 > > > > mfile type: data > > > > mfile size: 165675008 > > > > mfile start: ino 133 offset 0 > > > > mfile end: ino 28839 offset 0 > > > > media label: "tape1" > > > > media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > > > > media file 1: > > > > mfile index: 1 > > > > mfile type: inventory > > > > mfile size: 2097152 > > > > media label: "tape1" > > > > media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > > > > stream 1: > > > > pathname: /dev/nst1 > > > > start: ino 28839 offset 0 > > > > end: ino 1572997 offset 0 > > > > interrupted: NO > > > > media files: 2 > > > > media file 0: > > > > mfile index: 0 > > > > mfile type: data > > > > mfile size: 166723584 > > > > mfile start: ino 28839 offset 0 > > > > mfile end: ino 1572997 offset 0 > > > > media label: "tape2" > > > > media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 > > > > media file 1: > > > > mfile index: 1 > > > > mfile type: inventory > > > > mfile size: 2097152 > > > > media label: "tape2" > > > > media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 > > > > xfsdump: Dump Status: SUCCESS > > > > [root@rhel8 ~]# mv /var/lib/xfsdump/inventory > > > > /var/lib/xfsdump/inventory_two_sessions > > > > [root@rhel8 ~]# xfsdump -I > > > > xfsdump: Dump Status: SUCCESS > > > > > > > > [root@rhel8 ~]# xfsrestore -L Test1 -f /dev/nst0 /tmp/test1/ > > > > xfsrestore: using scsi tape (drive_scsitape) strategy > > > > xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status > > > > and control > > > > xfsrestore: searching media for dump > > > > xfsrestore: preparing drive > > > > xfsrestore: examining media file 2 > > > > xfsrestore: found dump matching specified label: > > > > xfsrestore: hostname: rhel8 > > > > xfsrestore: mount point: /boot > > > > xfsrestore: volume: /dev/sda1 > > > > xfsrestore: session time: Thu Oct 6 13:50:45 2022 > > > > xfsrestore: level: 0 > > > > xfsrestore: session label: "Test1" > > > > xfsrestore: media label: "tape1" > > > > xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 > > > > xfsrestore: session id: aa25fa48-4493-45c7-9027-61e53e486445 > > > > xfsrestore: media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > > > > xfsrestore: searching media for directory dump > > > > xfsrestore: rewinding > > > > xfsrestore: examining media file 0 > > > > xfsrestore: reading directories > > > > xfsrestore: 9 directories and 320 entries processed > > > > xfsrestore: directory post-processing > > > > xfsrestore: restoring non-directory files > > > > xfsrestore: examining media file 1 > > > > xfsrestore: inv_stobj.c:1119: stobj_unpack_sessinfo: Assertion > > > > `(size_t) ( p - (char *) bufp ) == bufsz' failed. > > > > Aborted (core dumped) > > > > > > > > Make sure we unpack multiple streams when restoring the online > > > > inventory from media. > > > > > > > > Signed-off-by: Donald Douwsma <ddouwsma@xxxxxxxxxx> > > > > > > Much better now, though I hope there's an fstest coming to make sure > > > that multistream restore/dump work properly. > > > > That could be a little tricky. > > > > xfstests currently only supports one TAPE_DEV (and a remote > > RMT_TAPE_DEV). So we'd need to add support for multiple tapes, > > something like > > Thinking about this some more RMT_TAPE_DEV may be ok. If streaming to > multiple targets is meant to enable an offsite dump it could be done > via remote i.e. > > # xfsdump -L "Test3" -f /dev/nst0 -M "tape1" -f backup.example.com:/dev/nst0 > -M "tape2" / > > The existing remote tests xfs/037 and xfs/038 just use the device for a > simple backup. > > Or I could be inventing a never useful use case. I wonder, does the "remote" tape device have to be remote, or is xfsdump flexible enough to let you set up a dummy file to dump into? I bet any fstest that actually requires a real tape device hasn't been run in years, except maybe by the RH QA department. :/ --D > Don > > > > > TAPE_DEV1=/dev/nst0 > > TAPE_DEV2=/dev/nst1 > > > > Then it can be hard to get access to a tape drive, let alone two. > > > > Probably the easiest way to address that is to provide instructions on > > how to setup mhvtl, https://github.com/markh794/mhvtl and load the > > tapes before a test run with > > > > [root@rhel8 ~]# systemctl start mhvtl.target > > [root@rhel8 ~]# mtx -f /dev/sch0 load 1 0; mtx -f /dev/sch load 2 1 > > Loading media from Storage Element 1 into drive 0...done > > Loading media from Storage Element 2 into drive 1...done > > [root@rhel8 ~]# ./check -g xfs/tape > > > > Not sure there's many people who test the tape group, so this may be > > generally useful. > > > > mhvtl requires a kernel module to be built, not sure if that would > > bother QE folk when doing general tests, but for xfsdump userspace > > testing it should be fine. > > > > Don > > > > > > > Reviewed-by: Darrick J. Wong <djwong@xxxxxxxxxx> > > > > > > --D > > > > > > > --- > > > > inventory/inv_stobj.c | 13 +++++++------ > > > > 1 file changed, 7 insertions(+), 6 deletions(-) > > > > > > > > diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c > > > > index b461666..025d431 100644 > > > > --- a/inventory/inv_stobj.c > > > > +++ b/inventory/inv_stobj.c > > > > @@ -1065,25 +1065,26 @@ stobj_unpack_sessinfo( > > > > return BOOL_FALSE; > > > > } > > > > + /* get the seshdr and then, the remainder of the session */ > > > > xlate_invt_seshdr((invt_seshdr_t *)p, (invt_seshdr_t *)tmpbuf, 1); > > > > bcopy(tmpbuf, p, sizeof(invt_seshdr_t)); > > > > - > > > > - /* get the seshdr and then, the remainder of the session */ > > > > s->seshdr = (invt_seshdr_t *)p; > > > > s->seshdr->sh_sess_off = -1; > > > > p += sizeof(invt_seshdr_t); > > > > - > > > > xlate_invt_session((invt_session_t *)p, (invt_session_t > > > > *)tmpbuf, 1); > > > > bcopy (tmpbuf, p, sizeof(invt_session_t)); > > > > s->ses = (invt_session_t *)p; > > > > p += sizeof(invt_session_t); > > > > /* the array of all the streams belonging to this session */ > > > > - xlate_invt_stream((invt_stream_t *)p, (invt_stream_t *)tmpbuf, 1); > > > > - bcopy(tmpbuf, p, sizeof(invt_stream_t)); > > > > s->strms = (invt_stream_t *)p; > > > > - p += s->ses->s_cur_nstreams * sizeof(invt_stream_t); > > > > + for (i = 0; i < s->ses->s_cur_nstreams; i++) { > > > > + xlate_invt_stream((invt_stream_t *)p, > > > > + (invt_stream_t *)tmpbuf, 1); > > > > + bcopy(tmpbuf, p, sizeof(invt_stream_t)); > > > > + p += sizeof(invt_stream_t); > > > > + } > > > > /* all the media files */ > > > > s->mfiles = (invt_mediafile_t *)p; > > > > -- > > > > 2.31.1 > > > > > > > >
