Troy Knabe wrote:
We have several Oracle instances 10.2 (some RAC, some not, but all are
having the same issue) all running on Red Hat 4.5. The binaries and data
files are all on the NetApp and mounted via NFS.
Sporadically when we shut the database down, we are unable to restart it.
/u1/app/oracle/admin/prodrac/bdump/prodrac1_dbw0_10023.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit
Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining
options
ORACLE_HOME = /u1/app/oracle/product/10.2.0
System name: Linux
Node name: orarac1
Release: 2.6.9-55.0.2.ELsmp
Version: #1 SMP Tue Jun 12 17:58:20 EDT 2007
Machine: x86_64
Instance name: prodrac1
Redo thread mounted by this instance: 1
Oracle process number: 8
Unix process pid: 10023, image: oracle@orarac1 (DBW0)
*** SERVICE NAME:() 2007-08-18 02:05:27.878
*** SESSION ID:(1194.1) 2007-08-18 02:05:27.878
ORA-01157: cannot identify/lock data file 1 - see DBWR trace file
ORA-01110: data file 1: '/u1/oradata/prod/system/system01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 2 - see DBWR trace file
ORA-01110: data file 2: '/u1/oradata/prod/temp/undotbs01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 3 - see DBWR trace file
ORA-01110: data file 3: '/u1/oradata/prod/system/sysaux01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 4 - see DBWR trace file
ORA-01110: data file 4: '/u1/oradata/prod/table/users01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 5 - see DBWR trace file
ORA-01110: data file 5: '/u1/oradata/prod/temp/undotbs02.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 6 - see DBWR trace file
ORA-01110: data file 6: '/u1/oradata/prod/temp/undotbs03.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 7 - see DBWR trace file
ORA-01110: data file 7: '/u1/oradata/prod/table/esis_codes_data01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 8 - see DBWR trace file
ORA-01110: data file 8: '/u1/oradata/prod/index/esis_codes_indx01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 9 - see DBWR trace file
ORA-01110: data file 9: '/u1/oradata/prod/index/esis_large_indx.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 10 - see DBWR trace file
ORA-01110: data file 10: '/u1/oradata/prod/index/esis_temp_indx.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 11 - see DBWR trace file
ORA-01110: data file 11: '/u1/oradata/prod/index/esis_indx.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 12 - see DBWR trace file
ORA-01110: data file 12: '/u1/oradata/prod/table/school_connects.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 13 - see DBWR trace file
ORA-01110: data file 13: '/u1/oradata/prod/table/esis_temp_data01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 14 - see DBWR trace file
ORA-01110: data file 14: '/u1/oradata/prod/table/esis_large_data01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 15 - see DBWR trace file
ORA-01110: data file 15: '/u1/oradata/prod/table/esis_data01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
ORA-01157: cannot identify/lock data file 16 - see DBWR trace file
ORA-01110: data file 16: '/u1/oradata/prod/table/tools01.dbf'
ORA-27086: unable to lock file - already in use
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 8
The work around that we have found is to copy all of the files off of that
mount point and then move them back. We even tried stopping everything
and umounting/remounting the file system, but it still has the same
errors.
Oracle and Netapp are both pointing to other as the root of the problem.
But to me it sounds like an NFS locking issue. Has anyone else
experienced this, or have any suggestions on a better work around that
having to copy the data files off and move them back?
(sorry if this is a duplicate)
Im assuming you have checked to be certain no firewall is blocking any
NFS ports. If not, do that first. At the very least, on the client run
rpcinfo -p <server>.
Check your NFS mount parameters. Oracle can be very picky and the error
messages are usually not very helpful. Here is what I've found works
(YMMV):
server:/etc/exports (rw,insecure,root_squash,no_subtree_check,sync)
client:/etc/fstab user,tcp,rsize=32768,wsize=32768,hard,intr,nfsvers=3
If the system is RAC'ed add noac to the list
The rsize and wsize can be critical under heavy I/O.
Thanks
-Troy
--
Stephen Carville <stephen@xxxxxxxxxxxxxx>
Systems Engineer
Land America
1.626.667.1450 X1326
--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list
