Re: bug-1421590-brick-mux-reuse-ports.t failure

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On Wed, Apr 12, 2017 at 12:07 PM, Atin Mukherjee <amukherj@xxxxxxxxxx> wrote:
As per http://fstat.rht.gluster.org/weeks/1 the test in $Subject has failed multiple times and is now blocking most of the patches to pass the regression. I have a patch https://review.gluster.org/#/c/17033/ to remove this test entirely and I have the reason in the commit message.

Can this patch be reviewed and merged asap?

+1. This is stopping us from many patches which could have passed otherwise.

Regards,
Amar
 

---------- Forwarded message ----------
From: Atin Mukherjee <amukherj@xxxxxxxxxx>
Date: Mon, Apr 10, 2017 at 2:57 PM
Subject: Re: [Gluster-Maintainers] Build failed in Jenkins: regression-test-burn-in #2932
To: GlusterFS Maintainers <maintainers@xxxxxxxxxxx>, Jeff Darcy <jeff@xxxxxxxxxx>


bug-1421590-brick-mux-reuse-ports.t seems to be a bad test to me and here is my reasoning:

This test tries to check if the ports are reused or not. When a volume is restarted, by the time glusterd tries to allocate a new port to the one of the brick processes of the volume there is no guarantee that the older port will be allocated given the kernel might take some extra time to free up the port between this time frame. From https://build.gluster.org/job/regression-test-burn-in/2932/console we can clearly see that post restart of the volume, glusterd allocated port 49153 & 49155 for brick1 & brick2 respectively but the test was expecting the ports to be matched with 49155 & 49156 which were allocated before the volume was restarted.

@Jeff - Is there any specific reason we want to keep this test running?


On Sat, Apr 8, 2017 at 8:12 AM, Atin Mukherjee <amukherj@xxxxxxxxxx> wrote:

On Sat, 8 Apr 2017 at 08:06, <jenkins@xxxxxxxxxxxxxxxxx> wrote:
See <http://build.gluster.org/job/regression-test-burn-in/2932/display/redirect>

------------------------------------------
[...truncated 12020 lines...]
ok 5, LINENUM:32
ok 6, LINENUM:33
ok 7, LINENUM:35
not ok 8 , LINENUM:37
FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
ok 9, LINENUM:38
not ok 10 , LINENUM:40
FAILED COMMAND: gluster --mode=script --wignore volume start patchy
not ok 11 Got "" instead of "49152", LINENUM:42
FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
not ok 12 , LINENUM:47
FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
not ok 13 , LINENUM:48
FAILED COMMAND: gluster --mode=script --wignore volume start patchy
not ok 14 Got "" instead of "49152", LINENUM:50
FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
not ok 15 Got "" instead of "get_nth_brick_port_for_volume", LINENUM:51
FAILED COMMAND: get_nth_brick_port_for_volume patchy 2
not ok 16 , LINENUM:53
FAILED COMMAND: gluster --mode=script --wignore volume stop patchy
ok 17, LINENUM:55
not ok 18 , LINENUM:57
FAILED COMMAND: gluster --mode=script --wignore volume start patchy
not ok 19 Got "" instead of "49152", LINENUM:59
FAILED COMMAND: 49152 get_nth_brick_port_for_volume patchy 1
volume set: success
Failed 10/19 subtests

Test Summary Report
-------------------
./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t (Wstat: 0 Tests: 19 Failed: 10)
  Failed tests:  8, 10-16, 18-19
Files=1, Tests=19, 249 wallclock secs ( 0.03 usr  0.01 sys + 13.40 cusr  3.36 csys = 16.80 CPU)
Result: FAIL
End of test ./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t

Something is wrong with this test, have seen it failing in many regression test burns. I'll take a look at it.


================================================================================


Run complete
================================================================================
Number of tests found:                             199
Number of tests selected for run based on pattern: 199
Number of tests skipped as they were marked bad:   7
Number of tests skipped because of known_issues:   4
Number of tests that were run:                     188

1 test(s) failed
./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t

0 test(s) generated core


Tests ordered by time taken, slowest to fastest:
================================================================================
./tests/basic/ec/ec-12-4.t  -  336 second
./tests/basic/ec/ec-7-3.t  -  199 second
./tests/basic/ec/ec-6-2.t  -  178 second
./tests/basic/ec/self-heal.t  -  158 second
./tests/basic/afr/split-brain-favorite-child-policy.t  -  151 second
./tests/basic/ec/ec-5-2.t  -  150 second
./tests/basic/ec/ec-5-1.t  -  150 second
./tests/basic/afr/entry-self-heal.t  -  150 second
./tests/basic/afr/self-heal.t  -  137 second
./tests/bugs/core/bug-1421590-brick-mux-reuse-ports.t  -  131 second
./tests/basic/tier/legacy-many.t  -  127 second
./tests/basic/tier/tier.t  -  126 second
./tests/basic/ec/ec-4-1.t  -  123 second
./tests/basic/ec/ec-optimistic-changelog.t  -  111 second
./tests/basic/afr/self-heald.t  -  109 second
./tests/basic/ec/ec-3-1.t  -  95 second
./tests/basic/volume-snapshot-clone.t  -  88 second
./tests/bugs/core/bug-1110917.t  -  85 second
./tests/basic/tier/new-tier-cmds.t  -  83 second
./tests/basic/afr/split-brain-heal-info.t  -  82 second
./tests/bugs/cli/bug-1320388.t  -  81 second
./tests/basic/ec/heal-info.t  -  80 second
./tests/basic/ec/ec-new-entry.t  -  79 second
./tests/basic/ec/ec-background-heals.t  -  77 second
./tests/basic/afr/split-brain-healing.t  -  76 second
./tests/basic/afr/metadata-self-heal.t  -  74 second
./tests/basic/ec/ec-notify.t  -  73 second
./tests/basic/afr/sparse-file-self-heal.t  -  73 second
./tests/basic/afr/granular-esh/cli.t  -  69 second
./tests/basic/quota.t  -  67 second
./tests/bugs/bug-1368312.t  -  63 second
./tests/basic/uss.t  -  52 second
./tests/basic/tier/frequency-counters.t  -  52 second
./tests/bugs/cli/bug-770655.t  -  50 second
./tests/basic/volume-snapshot.t  -  49 second
./tests/basic/afr/quorum.t  -  48 second
./tests/basic/tier/fops-during-migration-pause.t  -  47 second
./tests/basic/mount-nfs-auth.t  -  46 second
./tests/basic/ec/ec.t  -  46 second
./tests/basic/ec/ec-readdir.t  -  43 second
./tests/basic/afr/inodelk.t  -  40 second
./tests/basic/mpx-compat.t  -  39 second
./tests/basic/afr/arbiter.t  -  38 second
./tests/basic/ec/ec-cpu-extensions.t  -  37 second
./tests/basic/tier/locked_file_migration.t  -  36 second
./tests/bitrot/br-state-check.t  -  35 second
./tests/basic/tier/tier-heald.t  -  35 second
./tests/bitrot/bug-1294786.t  -  34 second
./tests/basic/volume-snapshot-xml.t  -  33 second
./tests/basic/mgmt_v3-locks.t  -  33 second
./tests/basic/tier/unlink-during-migration.t  -  32 second
./tests/basic/afr/granular-esh/conservative-merge.t  -  32 second
./tests/basic/afr/gfid-self-heal.t  -  31 second
./tests/basic/geo-replication/marker-xattrs.t  -  29 second
./tests/basic/quota-ancestry-building.t  -  27 second
./tests/bugs/cli/bug-1353156-get-state-cli-validations.t  -  26 second
./tests/basic/afr/arbiter-add-brick.t  -  26 second
./tests/bugs/core/bug-1402841.t-mt-dir-scan-race.t  -  25 second
./tests/basic/afr/heal-quota.t  -  25 second
./tests/basic/afr/data-self-heal.t  -  25 second
./tests/basic/afr/arbiter-mount.t  -  24 second
./tests/bugs/bitrot/bug-1227996.t  -  23 second
./tests/bitrot/bug-1373520.t  -  23 second
./tests/basic/nufa.t  -  23 second
./tests/basic/afr/durability-off.t  -  23 second
./tests/basic/mount.t  -  22 second
./tests/basic/gfapi/gfapi-ssl-test.t  -  22 second
./tests/basic/ec/quota.t  -  22 second
./tests/basic/afr/replace-brick-self-heal.t  -  22 second
./tests/basic/afr/granular-esh/replace-brick.t  -  22 second
./tests/basic/op_errnos.t  -  21 second
./tests/basic/glusterd/volfile_server_switch.t  -  21 second
./tests/bugs/bitrot/bug-1245981.t  -  20 second
./tests/basic/afr/granular-esh/granular-indices-but-non-granular-heal.t  -  20 second
./tests/basic/afr/granular-esh/granular-esh.t  -  20 second
./tests/basic/0symbol-check.t  -  20 second
./tests/bugs/cli/bug-1113476.t  -  19 second
./tests/bugs/changelog/bug-1225542.t  -  19 second
./tests/bugs/bitrot/bug-1288490.t  -  19 second
./tests/basic/tier/file_with_spaces.t  -  19 second
./tests/basic/glusterd/heald.t  -  19 second
./tests/basic/afr/client-side-heal.t  -  19 second
./tests/basic/afr/add-brick-self-heal.t  -  19 second
./tests/bugs/cli/bug-1077682.t  -  18 second
./tests/bugs/bitrot/1209751-bitrot-scrub-tunable-reset.t  -  18 second
./tests/basic/ec/statedump.t  -  18 second
./tests/basic/afr/split-brain-resolution.t  -  18 second
./tests/basic/afr/granular-esh/add-brick.t  -  18 second
./tests/bugs/cli/bug-1169302.t  -  17 second
./tests/bugs/bug-1110262.t  -  17 second
./tests/bugs/bitrot/bug-1228680.t  -  17 second
./tests/basic/tier/readdir-during-migration.t  -  17 second
./tests/basic/volume.t  -  16 second
./tests/basic/afr/resolve.t  -  16 second
./tests/basic/afr/read-subvol-data.t  -  16 second
./tests/bugs/cli/bug-1047416.t  -  15 second
./tests/bugs/changelog/bug-1321955.t  -  15 second
./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t  -  15 second
./tests/basic/quota-anon-fd-nfs.t  -  15 second
./tests/basic/pump.t  -  15 second
./tests/basic/glusterd/arbiter-volume-probe.t  -  15 second
./tests/basic/gfid-access.t  -  15 second
./tests/bugs/changelog/bug-1211327.t  -  14 second
./tests/bitrot/bug-1207627-bitrot-scrub-status.t  -  14 second
./tests/basic/rpc-coverage.t  -  14 second
./tests/basic/multiplex.t  -  14 second
./tests/basic/glusterd/disperse-create.t  -  14 second
./tests/basic/cdc.t  -  14 second
./tests/basic/bd.t  -  14 second
./tests/basic/afr/stale-file-lookup.t  -  14 second
./tests/basic/afr/root-squash-self-heal.t  -  14 second
./tests/basic/afr/read-subvol-entry.t  -  14 second
./tests/bugs/cli/bug-822830.t  -  13 second
./tests/bugs/cli/bug-1030580.t  -  13 second
./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t  -  13 second
./tests/basic/volume-status.t  -  13 second
./tests/basic/quota-nfs.t  -  13 second
./tests/basic/inode-quota-enforcing.t  -  13 second
./tests/basic/ec/ec-root-heal.t  -  13 second
./tests/basic/stats-dump.t  -  12 second
./tests/basic/glusterd/arbiter-volume.t  -  12 second
./tests/basic/ec/ec-read-policy.t  -  12 second
./tests/bugs/cli/bug-1087487.t  -  11 second
./tests/bugs/changelog/bug-1208470.t  -  11 second
./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t  -  11 second
./tests/bugs/access-control/bug-887098-gmount-crash.t  -  11 second
./tests/bitrot/bug-1244613.t  -  11 second
./tests/basic/fop-sampling.t  -  11 second
./tests/basic/ec/ec-anonymous-fd.t  -  11 second
./tests/bugs/core/bug-1119582.t  -  10 second
./tests/bugs/bug-1258069.t  -  10 second
./tests/bugs/bitrot/bug-1229134-bitd-not-support-vol-set.t  -  10 second
./tests/bugs/access-control/bug-958691.t  -  10 second
./tests/basic/tier/ctr-rename-overwrite.t  -  10 second
./tests/basic/meta.t  -  10 second
./tests/basic/afr/arbiter-remove-brick.t  -  10 second
./tests/bugs/cli/bug-1022905.t  -  9 second
./tests/bugs/bitrot/bug-1210684-scrub-pause-resume-error-handling.t  -  9 second
./tests/basic/ec/nfs.t  -  9 second
./tests/basic/distribute/bug-1265677-use-readdirp.t  -  9 second
./tests/basic/afr/gfid-mismatch.t  -  9 second
./tests/basic/afr/gfid-heal.t  -  9 second
./tests/basic/afr/compounded-write-txns.t  -  9 second
./tests/basic/afr/arbiter-statfs.t  -  9 second
./tests/bugs/access-control/bug-1051896.t  -  8 second
./tests/bitrot/br-stub.t  -  8 second
./tests/basic/pgfid-feat.t  -  8 second
./tests/basic/md-cache/bug-1317785.t  -  8 second
./tests/basic/gfapi/upcall-cache-invalidate.t  -  8 second
./tests/basic/gfapi/gfapi-dup.t  -  8 second
./tests/basic/gfapi/anonymous_fd.t  -  8 second
./tests/basic/ec/ec-internal-xattrs.t  -  8 second
./tests/basic/ec/dht-rename.t  -  8 second
./tests/basic/afr/heal-info.t  -  8 second
./tests/bugs/core/bug-1168803-snapd-option-validation-fix.t  -  7 second
./tests/bugs/core/949327.t  -  7 second
./tests/bugs/cli/bug-983317-volume-get.t  -  7 second
./tests/bugs/cli/bug-982174.t  -  7 second
./tests/bugs/cli/bug-961307.t  -  7 second
./tests/bitrot/bug-1221914.t  -  7 second
./tests/basic/quota-rename.t  -  7 second
./tests/basic/md-cache/bug-1418249.t  -  7 second
./tests/basic/gfapi/glfs_sysrq.t  -  7 second
./tests/basic/gfapi/gfapi-trunc.t  -  7 second
./tests/basic/gfapi/gfapi-async-calls-test.t  -  7 second
./tests/basic/gfapi/bug-1241104.t  -  7 second
./tests/basic/fops-sanity.t  -  7 second
./tests/basic/distribute/throttle-rebal.t  -  7 second
./tests/bugs/core/bug-1135514-allow-setxattr-with-null-value.t  -  6 second
./tests/bugs/core/bug-1111557.t  -  6 second
./tests/bugs/cli/bug-977246.t  -  6 second
./tests/bugs/cli/bug-921215.t  -  6 second
./tests/bugs/cli/bug-867252.t  -  6 second
./tests/bugs/cli/bug-1378842-volume-get-all.t  -  6 second
./tests/bugs/cli/bug-1004218.t  -  6 second
./tests/bugs/access-control/bug-1387241.t  -  6 second
./tests/bitrot/bug-internal-xattrs-check-1243391.t  -  6 second
./tests/basic/gfapi/libgfapi-fini-hang.t  -  6 second
./tests/basic/afr/arbiter-cli.t  -  6 second
./tests/bugs/core/bug-1117951.t  -  5 second
./tests/bugs/cli/bug-969193.t  -  5 second
./tests/bugs/cli/bug-949298.t  -  5 second
./tests/bugs/cli/bug-764638.t  -  5 second
./tests/bugs/cli/bug-1047378.t  -  5 second
./tests/basic/posixonly.t  -  1 second
./tests/basic/netgroup_parsing.t  -  1 second
./tests/basic/exports_parsing.t  -  1 second
./tests/basic/first-test.t  -  0 second

Result is 1

tar: Removing leading `/' from member names
Logs archived in http://slave28.cloud.gluster.org/logs/glusterfs-logs-regression-test-burn-in-2932.tgz
kernel.core_pattern = /%e-%p.core
Build step 'Execute shell' marked build as failure
_______________________________________________
maintainers mailing list
maintainers@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/maintainers
--
- Atin (atinm)



--

ATin Mukherjee

Associate Manager, RHGS Development

Red Hat

amukherj@xxxxxxxxxx    M: +919739491377     IM: IRC: atinm, twitter: @mukherjee_atin




--

ATin Mukherjee

Associate Manager, RHGS Development

Red Hat

amukherj@xxxxxxxxxx    M: +919739491377     IM: IRC: atinm, twitter: @mukherjee_atin


_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel



--
Amar Tumballi (amarts)
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux