Re: [Gluster-Maintainers] Master branch lock down: RCA for tests (bugs/distribute/bug-1122443.t)

[Raghavendra Gowdappa <rgowdapp@xxxxxxxxxx>]

Initial RCA to point out commit 7131de81f72dda0ef685ed60d0887c6e14289b8c caused the issue was done by Nithya. Following was the conversation:

<snip>

With the latest master, I created a single brick volume and some files
    inside it.
   
    [root@rhgs313-6 ~]# umount -f /mnt/fuse1; mount -t glusterfs -s
    192.168.122.6:/thunder /mnt/fuse1; ls -l /mnt/fuse1/; echo "Trying
    again"; ls -l /mnt/fuse1
    umount: /mnt/fuse1: not mounted
    total 0
    ----------. 0 root root 0 Jan  1  1970 file-1
    ----------. 0 root root 0 Jan  1  1970 file-2
    ----------. 0 root root 0 Jan  1  1970 file-3
    ----------. 0 root root 0 Jan  1  1970 file-4
    ----------. 0 root root 0 Jan  1  1970 file-5
    d---------. 0 root root 0 Jan  1  1970 subdir
    Trying again
    total 3
    -rw-r--r--. 1 root root 33 Aug  3 14:06 file-1
    -rw-r--r--. 1 root root 33 Aug  3 14:06 file-2
    -rw-r--r--. 1 root root 33 Aug  3 14:06 file-3
    -rw-r--r--. 1 root root 33 Aug  3 14:06 file-4
    -rw-r--r--. 1 root root 33 Aug  3 14:06 file-5
    d---------. 0 root root  0 Jan  1  1970 subdir
    [root@rhgs313-6 ~]#
   
    Conversation can be followed on gluster-devel on thread with subj:
    tests/bugs/distribute/bug-1122443.t - spurious failure. git-bisected
    pointed this patch as culprit.

</snip>

commit 7131de81f72dda0ef685ed60d0887c6e14289b8c zeroed out all members of iatt except for ia_gfid and ia_type in certain scenarios (one case that led to this bug was when a fresh inode - not linked - was picked up by readdirplus). This led to fuse_readdirp_cbk to wrongly think it has a valid stat (due to valid ia_gfid and ia_type) and give to kernel zeroed out attributes causing failures. Fix is included in https://review.gluster.org/20639 to make sure to let kernel know attributes are not valid in this scenario (and not zero out stats even if inode picked up by readdirplus is not linked yet).

regards,

Raghavendra

On Mon, Aug 13, 2018 at 6:12 AM, Shyam Ranganathan <srangana@xxxxxxxxxx> wrote:

As a means of keeping the focus going and squashing the remaining tests
that were failing sporadically, request each test/component owner to,

- respond to this mail changing the subject (testname.t) to the test
name that they are responding to (adding more than one in case they have
the same RCA)
- with the current RCA and status of the same

List of tests and current owners as per the spreadsheet that we were
tracking are:

./tests/basic/distribute/rebal-all-nodes-migrate.t              TBD
./tests/basic/tier/tier-heald.t         TBD
./tests/basic/afr/sparse-file-self-heal.t               TBD
./tests/bugs/shard/bug-1251824.t                TBD
./tests/bugs/shard/configure-lru-limit.t                TBD
./tests/bugs/replicate/bug-1408712.t    Ravi
./tests/basic/afr/replace-brick-self-heal.t             TBD
./tests/00-geo-rep/00-georep-verify-setup.t     Kotresh
./tests/basic/afr/gfid-mismatch-resolution-with-fav-child-policy.t Karthik
./tests/basic/stats-dump.t              TBD
./tests/bugs/bug-1110262.t              TBD
./tests/basic/ec/ec-data-heal.t         Mohit
./tests/bugs/replicate/bug-1448804-check-quorum-type-values.t           Pranith
./tests/bugs/snapshot/bug-1482023-snpashot-issue-with-other-processes-accessing-mounted-path.t
TBD
./tests/basic/ec/ec-5-2.t               Sunil
./tests/bugs/shard/bug-shard-discard.t          TBD
./tests/bugs/glusterd/remove-brick-testcases.t          TBD
./tests/bugs/protocol/bug-808400-repl.t         TBD
./tests/bugs/quick-read/bug-846240.t            Du
./tests/bugs/replicate/bug-1290965-detect-bitrotten-objects.t           Mohit
./tests/00-geo-rep/georep-basic-dr-tarssh.t     Kotresh
./tests/bugs/ec/bug-1236065.t           Pranith
./tests/00-geo-rep/georep-basic-dr-rsync.t      Kotresh
./tests/basic/ec/ec-1468261.t           Ashish
./tests/basic/afr/add-brick-self-heal.t         Ravi
./tests/basic/afr/granular-esh/replace-brick.t          Pranith
./tests/bugs/core/multiplex-limit-issue-151.t           Sanju
./tests/bugs/glusterd/validating-server-quorum.t                Atin
./tests/bugs/replicate/bug-1363721.t            Ravi
./tests/bugs/index/bug-1559004-EMLINK-handling.t                Pranith
./tests/bugs/replicate/bug-1433571-undo-pending-only-on-up-bricks.t             Karthik
./tests/bugs/glusterd/add-brick-and-validate-replicated-volume-options.t
        Atin
./tests/bugs/glusterd/rebalance-operations-in-single-node.t             TBD
./tests/bugs/replicate/bug-1386188-sbrain-fav-child.t           TBD
./tests/bitrot/bug-1373520.t    Kotresh
./tests/bugs/distribute/bug-1117851.t   Shyam/Nigel
./tests/bugs/glusterd/quorum-validation.t       Atin
./tests/bugs/distribute/bug-1042725.t           Shyam
./tests/bugs/replicate/bug-1586020-mark-dirty-for-entry-txn-on-quorum-failure.t
        Karthik
./tests/bugs/quota/bug-1293601.t                TBD
./tests/bugs/bug-1368312.t      Du
./tests/bugs/distribute/bug-1122443.t           Du
./tests/bugs/core/bug-1432542-mpx-restart-crash.t       1608568 Nithya/Shyam

Thanks,
Shyam
_______________________________________________
maintainers mailing list
maintainers@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/maintainers

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-devel