Could you correlate these with the SMB counters (stats) on the client in /proc/fs/cifs/ which also should include bytes written/read? Based on your description it looks like reads totally dominate this workload. The obvious thing to check is the read size - a few quick things does "forcedirectio" (or "cache=none" for newer kernels) help - in that case the network read size which usually match what the client application requests (which is usually a bad thing because we want large i/o over the network and apps tend to read too small). Second thing to try is a larger rsize - IIRC Windows should generally support up to rsize of about 127K (for Samba the rsize can be larger due to Unix extensions, we set the rsize smaller than 127K to Windows to work around bugs in other non-Samba, non-Windows servers). On Fri, Dec 14, 2012 at 5:11 PM, Senén de Diego <senen@xxxxxxxxxxxxx> wrote: > El 14/12/2012 3:39, Jeff Layton escribió: > >> On Thu, 13 Dec 2012 23:53:00 +0100 >> Senén de Diego <senen@xxxxxxxxxxxxx> wrote: >>> >>> Hello, >>> I'm executing, in a linux server, a program that reads data from dbf >>> files in a windows share. I've tried to execute the same program in an >>> upgraded server and I've found that it runs much much much slower. >>> These are the versions of the first (older) server: >>> kernel: 2.6.37.6-24 >>> cifs.ko: 1.68 >>> and of the second (newer): >>> kernel: 3.4.11-2.16 >>> cifs.ko: 1.78 >>> The windows machine runs windows 2000 server and has EnableOplocks >>> disabled in the LanManServer service. >>> The windows share is mounted as a cifs file system with some differences >>> in the parameters assigned by default: >>> first server: rsize=16384,wsize=57344 >>> second server: sec=ntlm,nounix,rsize=61440,wsize=65536 >>> I've captured the network traffic in both servers, and these are the >>> service response time statistics: >>> First server: >>> ================================================================= >>> SMB SRT Statistics: >>> Filter: >>> Commands Calls Min SRT Max SRT Avg SRT >>> Close 37 0.000140 0.001635 0.000404 >>> Read AndX 11554 0.000196 0.017179 0.002099 >>> Write AndX 12 0.000255 0.000657 0.000510 >>> NT Create AndX 43 0.000190 0.000406 0.000237 >>> Transaction2 Commands Calls Min SRT Max SRT Avg SRT >>> FIND_FIRST2 83 0.000747 0.001953 0.001735 >>> FIND_NEXT2 880 0.000486 0.002885 0.001711 >>> QUERY_PATH_INFO 25678 0.000170 0.005451 0.000250 >>> QUERY_FILE_INFO 67 0.000169 0.000206 0.000182 >>> NT Transaction Commands Calls Min SRT Max SRT Avg SRT >>> ================================================================= >>> Second server: >>> ================================================================= >>> SMB SRT Statistics: >>> Filter: >>> Commands Calls Min SRT Max SRT Avg SRT >>> Close 118 0.000128 0.006194 0.000517 >>> Trans 1 0.000872 0.000872 0.000872 >>> Read AndX 3490 0.000260 0.019713 0.009729 >>> Write AndX 12 0.000508 0.000522 0.000515 >>> NT Create AndX 118 0.000253 0.001496 0.000342 >>> Transaction2 Commands Calls Min SRT Max SRT Avg SRT >>> FIND_FIRST2 171 0.000761 0.002862 0.002404 >>> FIND_NEXT2 326 0.002342 0.003058 0.002429 >>> QUERY_PATH_INFO 318093 0.000168 0.041616 0.000320 >>> QUERY_FILE_INFO 137 0.000235 0.002855 0.000370 >>> NT Transaction Commands Calls Min SRT Max SRT Avg SRT >>> ================================================================= >>> where there is a great difference in the number of calls, specially in >>> the number of QUERY_PATH_INFO commands executed. And the average SRT are >>> also slower in the newer server. >>> I would like to know what is the cause and what can I do to correct this? >> >> The difference between those two kernel represents more than a year of >> development time. The best first step you can take is to try a more >> recent kernel (like something 3.7-ish) to see if that might help. > > I've tried with a 3.6.10 kernel (the last available in my distro) and it's > still slow: > > > ================================================================= > SMB SRT Statistics: > Filter: > Commands Calls Min SRT Max SRT Avg SRT > Close 116 0.000137 1.101347 0.009745 > Trans 3 0.000517 0.000722 0.000642 > Echo 1 0.000203 0.000203 0.000203 > Read AndX 122893 0.000206 0.154745 0.000599 > Write AndX 12 0.000253 0.005756 0.000789 > NT Create AndX 118 0.000266 0.000865 0.000458 > > > Transaction2 Commands Calls Min SRT Max SRT Avg SRT > FIND_FIRST2 170 0.000843 0.003413 0.002590 > FIND_NEXT2 325 0.002373 0.006695 0.002908 > QUERY_PATH_INFO 320423 0.000181 2.066630 0.000485 > QUERY_FILE_INFO 135 0.000252 0.000768 0.000395 > > > NT Transaction Commands Calls Min SRT Max SRT Avg SRT > ================================================================= > > > >> The next problem is that it's difficult for us to infer anything about >> your workload. You'll probably need to do some investigative work to >> figure out what's actually slow. In particular, if you can phrase the >> problem in terms of particular syscalls being slower, then we might be >> able to help you more. >> Even better is to come up with a small reproducer program that >> demonstrates the slowdown. > > The file access is done by a proprietary jdbc driver inside a jvm. I really > don't know how to do what you suggest. The SMB service response time > statitics are all I've got. > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-cifs" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Thanks, Steve -- To unsubscribe from this list: send the line "unsubscribe linux-cifs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
