> Would it be possible to turn this into a full reproducer script? > Something that someone that knows nothing about mysqld/mariadb can just > run and have it reproduce. If I install the 10.6 packages from above, > then it doesn't seem to use io_uring or be linked against liburing. Sorry Jens. Hope containers are ok. mkdir ~/mdbtest/ $ podman run -d -e MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 -e MARIADB_USER=sbtest -e MARIADB_PASSWORD=sbtest -e MARIADB_DATABASE=sbtest --name mdb10.6-uring_test -v $HOME/mdbtest:/var/lib/mysql:Z --security-opt seccomp=unconfined quay.io/danielgblack/mariadb-test:10.6-impish-sysbench --innodb_log_file_size=1G --innodb_buffer_pool_size=50G --innodb_io_capacity=5000 --innodb_io_capacity_max=9000 --innodb_flush_log_at_trx_commit=0 --innodb_adaptive_flushing_lwm=0 --innodb-adaptive-flushing=1 --innodb_flush_neighbors=1 --innodb-use-native-aio=1 --innodb_file-per-table=1 --innodb-fast-shutdown=0 --innodb-flush-method=O_DIRECT --innodb_lru_scan_depth=1024 --innodb_lru_flush_size=256 # drop 50G pool size down if you don't have it. Not critical to reproduction. IO capacity here should be about what the hardware is. Otherwise gaps of 0 tps will appear without it being the cause of the bug. $ podman logs mdb10.6-uring_test ... 2021-11-11 6:06:49 0 [Warning] innodb_use_native_aio may cause hangs with this kernel 5.15.0-0.rc7.20211028git1fc596a56b33.56.fc36.x86_64; see https://jira.mariadb.org/browse/MDEV-26674 2021-11-11 6:06:49 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2021-11-11 6:06:49 0 [Note] InnoDB: Number of pools: 1 2021-11-11 6:06:49 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2021-11-11 6:06:49 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts) 2021-11-11 6:06:49 0 [Note] InnoDB: Using liburing Should contain first and last line here: $ podman exec mdb10.6-uring_test sysbench /usr/share/sysbench/oltp_update_index.lua --mysql-password=sbtest --percentile=99 --tables=8 --table_size=2000000 prepare Creating table 'sbtest1'... Inserting 2000000 records into 'sbtest1' Creating a secondary index on 'sbtest1'... Creating table 'sbtest2'... Inserting 2000000 records into 'sbtest2' Creating a secondary index on 'sbtest2'... Creating table 'sbtest3'... Inserting 2000000 records into 'sbtest3' Creating a secondary index on 'sbtest3'... Creating table 'sbtest4'... Inserting 2000000 records into 'sbtest4' Creating a secondary index on 'sbtest4'... Creating table 'sbtest5'... Inserting 2000000 records into 'sbtest5' Creating a secondary index on 'sbtest5'... Creating table 'sbtest6'... Inserting 2000000 records into 'sbtest6' Creating a secondary index on 'sbtest6'... Creating table 'sbtest7'... Inserting 2000000 records into 'sbtest7' Creating a secondary index on 'sbtest7'... Creating table 'sbtest8'... Inserting 2000000 records into 'sbtest8' Creating a secondary index on 'sbtest8'... # Adjust threads there to the amount of hardware threads available. time is the length of the test. $ podman exec mdb10.6-uring_test sysbench /usr/share/sysbench/oltp_update_index.lua --mysql-password=sbtest --percentile=99 --tables=8 --table_size=2000000 --rand-seed=42 --rand-type=uniform --max-requests=0 --time=600 --report-interval=5 --threads=64 run Eventually after https://mariadb.com/kb/en/innodb-system-variables/#innodb_fatal_semaphore_wait_threshold of 600 seconds the podman logs mdb10.6-uring_test will contains an error like: 2021-10-07 17:06:43 0 [ERROR] [FATAL] InnoDB: innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch. Please refer to https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ 211007 17:06:43 [ERROR] mysqld got signal 6 ; Restarting the container on the same populated ~/mdbtest volume could be slow due to recovery time. Remove contents and repeat prepare step. cleanup: podman kill mdb10.6-uring_test podman rm mdb10.6-uring_test sudo rm -rf ~/mdbtest
