Hi all,
I'm testing PostgreSQL(RDBMS) backup using LVM2 snapshot on RHEL4.
After creating a snapshot, some processes (kjournald,
one PostgreSQL backend and others) go to iowait status,
and they never come back. (see pid 8105 and 2973 in below)
In this situation, one PostgreSQL backend process is waiting
in COMMIT processing(it generates fsync() on logical volume),
and also kjournald is waiting something.
There is no kernel oops, and the processors are not used.
When I set an PostgreSQL option not to generate fsync
on COMMIT, it looks work well. No process hangs.
I guess some race conditions is occured around kjournald.
Any comments and suggestions?
Thanks.
------------------------------------
# uname -a
Linux st17 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 2005 i686 i686 i386 GNU/Linux
# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda5 20161172 6717648 12419384 36% /
/dev/sda1 202219 11423 180356 6% /boot
none 1037448 0 1037448 0% /dev/shm
/dev/mapper/vg0-pgdata
1032088 515532 464128 53% /pgdata
# ps ax
PID TTY STAT TIME COMMAND
1 ? S 0:00 init [3]
2 ? S 0:00 [migration/0]
3 ? SN 0:00 [ksoftirqd/0]
4 ? S 0:00 [migration/1]
5 ? SN 0:00 [ksoftirqd/1]
6 ? S< 0:00 [events/0]
7 ? S< 0:00 [events/1]
8 ? S< 0:00 [khelper]
9 ? S< 0:00 [kacpid]
30 ? S< 0:00 [kblockd/0]
31 ? S< 0:00 [kblockd/1]
44 ? S< 0:00 [aio/0]
45 ? S< 0:00 [aio/1]
32 ? S 0:00 [khubd]
43 ? S 0:11 [kswapd0]
118 ? S 0:00 [kseriod]
186 ? S 0:00 [scsi_eh_0]
201 ? S 0:17 [kjournald]
1157 ? S<s 0:00 udevd
1342 ? S 0:00 [kjournald]
1761 ? Ss 0:00 syslogd -m 0
1765 ? Ss 0:00 klogd -x
1776 ? Ss 0:00 irqbalance
1794 ? Ss 0:00 portmap
1814 ? Ss 0:00 rpc.statd
1913 ? Ss 0:00 rpc.idmapd
1990 ? Ss 0:00 /usr/sbin/acpid
2003 ? Ss 0:00 cupsd
2042 ? Ss 0:00 /usr/sbin/sshd
2116 ? Ss 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
2126 ? Ss 0:00 gpm -m /dev/input/mice -t imps2
2169 ? Ss 0:00 /usr/sbin/htt -retryonerror 0
2170 ? S 0:00 htt_server -nodaemon
2182 ? Ss 0:00 /usr/sbin/cannaserver -syslog -u canna
2233 ? Ss 0:00 crond
2274 ? Ss 0:00 xfs -droppriv -daemon
2293 ? Ss 0:00 /usr/sbin/atd
2303 ? Ssl 0:00 dbus-daemon-1 --system
2317 ? Ss 0:00 cups-config-daemon
2328 ? Ss 0:01 hald
2338 tty1 Ss+ 0:00 /sbin/mingetty tty1
2339 tty2 Ss+ 0:00 /sbin/mingetty tty2
2340 tty3 Ss+ 0:00 /sbin/mingetty tty3
2341 tty4 Ss+ 0:00 /sbin/mingetty tty4
2342 tty5 Ss+ 0:00 /sbin/mingetty tty5
2343 tty6 Ss+ 0:00 /sbin/mingetty tty6
2885 ? Ss 0:00 sshd: snaga [priv]
2887 ? S 0:00 sshd: snaga@pts/0
2888 pts/0 Ss 0:00 -bash
2908 ? Ss 0:00 sshd: snaga [priv]
2910 ? S 0:00 sshd: snaga@pts/1
2911 pts/1 Ss 0:00 -bash
2931 pts/1 S 0:00 su
2932 pts/1 S 0:00 bash
2973 ? D 0:31 [kjournald]
2980 ? S 0:54 [rpciod]
2981 ? S 0:00 [lockd]
3016 pts/0 S 0:00 su
3017 pts/0 S 0:00 bash
3035 pts/0 S 0:00 su postgres
3036 pts/0 S 0:00 bash
7866 pts/0 S+ 0:00 /bin/sh ./pgbench_lvm.sh
7906 pts/0 S+ 0:00 /bin/sh ./pgbench_lvm.sh
7907 pts/0 S+ 0:00 sed -e s/^/lvm:/
7915 pts/0 S+ 0:00 /usr/local/pgsql81b3/bin/postmaster -D /pgdata/data
7917 pts/0 S+ 0:03 postgres: writer process
7918 pts/0 S+ 0:00 postgres: archiver process
7919 pts/0 S+ 0:00 postgres: stats buffer process
7920 pts/0 S+ 0:00 postgres: stats collector process
8015 ? S 0:01 [pdflush]
8094 pts/0 S+ 0:01 pgbench -s 10 -t 1000 -c 16 pgbench
8096 pts/0 S+ 0:00 postgres: postgres pgbench [local] COMMIT
8097 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8098 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8099 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8100 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8101 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8102 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8103 pts/0 S+ 0:00 postgres: postgres pgbench [local] COMMIT
8104 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8105 pts/0 D+ 0:00 postgres: postgres pgbench [local] COMMIT
8106 pts/0 S+ 0:00 postgres: postgres pgbench [local] COMMIT
8107 pts/0 S+ 0:00 postgres: postgres pgbench [local] COMMIT
8108 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8109 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8110 pts/0 S+ 0:00 postgres: postgres pgbench [local] UPDATE waiting
8111 pts/0 S+ 0:00 postgres: postgres pgbench [local] COMMIT
8119 ? S 0:00 [pdflush]
8147 ? S< 0:00 [kcopyd]
8179 ? S 0:00 [kjournald]
8188 pts/1 R+ 0:00 ps ax
# cat /etc/issue
Red Hat Enterprise Linux ES release 4 (Nahant)
Kernel \r on an \m
#
------------------------------------
--
NAGAYASU Satoshi <nagayasus@nttdata.co.jp>
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
