Re: pNFS Block layout issues / questions

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Forward to the correct mailing list.

jim is the most qualified for any pnfs-blocks issues

cheer
Boaz

On 03/28/2011 03:35 PM, Phil C wrote:
> 
> I was able to get the pNFS file-layout code functioning with 1 DS, and a MDS and
> 1 Client.  I need to get block layout working for a research project I am
> working on, however have so far been unable to get this working.
> 
> I would very much appreciate help or suggestions (or a kick in the head if I'm
> doing something stupid).
> 
> I referenced the wiki pages on client and DS setup and referenced the two
> threads I saw on this topic, however cannot get block-layout working.  
> 
> Below I have shown the steps I have taken to try and get this working (sorry for
> the long list).  
> 
> I have also shown the testing and troubleshooting I have undertaken, and the
> results there-of.
> 
> Data Server: 192.168.1.2 MDS: 192.168.1.1 Client: 192.168.1.3
> 
> On Data-server(s) create folder with mkdir /pnfs 
> 
> On Data-server(s) Edit /etc/exports.  Add the string: 
>    /pnfs  *(rw, sync,fsid=0,insecure,no_subtree_check,pnfs,no_root_squash)
> 
> On Data-server(s) type parted /dev/sdb 
>    (parted) mklabel gpt
>    (parted) mklabel 1 <Provide ext4 file-system, and partition start and end>
>    (parted> print
>    Model: ATA SSDSA2SH06â (scsi)
>    Disk /dev/sdb: 64.0GB
>    Sector size (logical/physical): 512B/512B
>    Partition Table: gpt
>    Number Start	   End		size	File system	Name	Flags
>        1  1049kB  64.0GB       64.0GB	      ext4         1
> 	
> On Data-server(s) type mkfs.ext4 âb 4096 /dev/sdb1
> 
> On Data-server(s) type /etc/init.d/iptables stop
> 
> On Data-server(s) type /etc/init.d/tgtd start
> 
> On Data-server(s) type:
>      tgtadm âlld iscsi âop new âmode target âtid 1 âT
> iqn.2011-03.com.intel:openblock
> 
> On Data-server(s) type 
>   tgtadm âlld iscsi âmode logicalunit âop new âtid 1 âlun 1 âbacking-store /dev/sdb1
> 
> On Data-server(s) type 
>    tgtadm âlld iscsi âmode target âop bind âtid 1 âinitiator-address ALL
> 
> On Data-server(s) type tgtadm âlld iscsi âop show âmode target
> 
> On Data-server(s) type mount /dev/sdb1 /pnfs
> 
> On Data-server(s) type service nfs restart
> 
> On Data-server(s) cd to the ctl directory and type ./ctl âu & 
>    See: âProcessing: MSG_VERS	: Version: 8
> 
> On MDS copy the spnfsd.conf file from pNFS_Packages/pnfs-nfs-utils/utils to
> /etc/spnfsd.conf and edit it for 1 DS and the proper DS-Mount-Directory, DS1_IP,
> and DS1_Root.
>    [General]
>    Verbosity = 1
>    Stripe-size = 8192
>    Dense-striping = 0
>    Pipefs-Directory = /var/lib/nfs/rpc_pipefs
>    DS-Mount-Directory = /spnfs
>    [DataServers]
>    NumDS = 1
>    DS1_IP=	192.168.1.2
>    DS1_PORT=	2049
>    DS1_ROOT = /pnfs
>    DS1_ID = 1
> 
> On MDS create the /spnfs/192.168.1.2 folder
> 
> On MDS edit /etc/exports and add below string:
>    /	*(rw, sync,fsid=0,insecure,no_subtree_check,pnfs,no_root_squash)
>    
> On MDS type mount 192.168.1.2:/pnfs /spnfs/192.168.1.2
>    See on the DS â the ctl application prints out:
>    Processing: MSG_VERS	:	Version: 	8
> 
> On MDS disable iptables with /etc/init.d/iptables stop
> 
> On MDS type service nfs restart â See:
>    Starting NFS services:	[	OK	]
>    Starting NFS quotas:	[	OK	]
>    Starting NFS daemon:	[	OK	]
>    Starting NFS mountd:	[	OK	]
>   
> On MDS type modprobe nfsd to start the NFS daemon
> 
> On MDS type spnfsd to start service on MDS server.  
>    See recapitulation of /etc/spnfsd.confâ; 
>    
> On the Client disable iptables with /etc/init.d/iptables stop
> 
> On the Client log onto the Data-Server iSCSI target by typing:
>    iscsi âm discovery ât st âp 192.168.1.2 âl
> 
> On the Client start the blkmapd service by typing service blkmapd start
> 
> On the Client run /usr/sbin/blkmapd âf 
> 
> On the Client mount MDS server by typing:
>    mount ât nfs4 âo minorversion =1 192.168.1.1:/spnfs /mnt
> 
> TESTING:
> On the Client generate some activity in /mnt (copy file from local client disk
> to /mnt)
> 
> On the Client cat /proc/self/mountstats and look for LAYOUT and see:
>    nfsv4: bm0=0xfdffbfff,bm1=0x40f9be3e,acl=0x3,sessions,pnfs=not configured
>    LAYOUTGET:	0 0 0 0 0 0 0 0
>    LAYTOUTCOMMIT: 0 0 0 0 0 0 0 0
>    LAYOUTRETURN: 0 0 0 0 0 0 0 0
> 
>    On the Client I see in /mnt the file I copied
>    On the MDS I see in /spnfs (note: not /spnfs/192.168.1.2) the file I copied
> from the Client
>    On the MDS I see in /spnfs/192.168.1.2 a file called 2359299.1868952557
>    On the DS I see in /pnfs a file called 2359299.1868952557
> 
> TROUBLESHOOTING:
> Read the new-ish trouble shooting information on the wiki-page.
> 
> 1 â Ran more traffic and still see the same information from the clientâs 
> /proc/self/mountstats
> 
> 2 â Ran /usr/sbin/blkmapd âf on the Client and saw no output
> 
> 3 â Looked at /var/log/messages on all 3 machines.  
>    Saw nothing on either the MDS or the DS.  
>    On the Client I see the innocuous: 
>       âFS-Cache: Loadedâ
>       Registering the id_resolver key type
>       FS-Cache: Netfs ânfsâ registered for caching
> 
> 4 â On the Client I ran âmountâ and see I have:
>    192.168.1.1:/spnf on /mnt type nfs4  
> (rw,minorversion=1,addr192.168.1.1,clientaddr=192.168.1.3)
> 
> <Note: no output from the running â/usr/sbin/blkmapd âfâ when I ran the mount
> command>
> 
> 5 â On the Client I verified I have an iSCSI target attached.
> 
> 6 â On the Client I did an lsmod|grep bllock to see I had blocklayoutdriver
> loaded and used by nfs and sunrpc
> 
> QUESTIONS:
> 1 â do you see anything missing or wrong about my procedure?
> 
> 2 â The MDS mounts the DS using âmount 192.168.1.2:/pnfs /spnfs/192.168.1.2â. 
> Does the MDS also/instead need to act as an iSCSI initiator and discover/logon 
> to the DS iSCSI target?
> 
> 3 â from /proc/self/mountstats on the client I see I do NOT in fact have
> pnfs=LAYOUT_BLOCK_VOLUEM so I assume we can suspect we do not have a pNFS mount.
>  What does this mean and how can I address the issue?
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [SCSI Target Devel]     [Linux SCSI Target Infrastructure]     [Kernel Newbies]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Linux IIO]     [Samba]     [Device Mapper]
  Powered by Linux