On Sunday 18 May 2003 19:09, Michael Harris wrote: htrees should speed up your directories i think, but they are not stable yet, as far as i know. I think putting up the files via scp or ftp and not nfs or smb should also speed up things, because no directory-listing is made. Perhaps there is a method to prevent samba from showing the directory-content to the clients, so the directory-listing would not happen. > > On Sunday 18 May 2003 18:01, Michael Harris wrote: > > > Just to add, I can attest that moving the files from the old dir to the > > > new as described improves performance on my machines dramatically. In > > > our service we end up with directories of 150k+ files which are > > > generally touched only as they are added, though every file will be > > > touched several times over a month. The files are each around 50kB. > > > When the directory entry gets to be about 4MB it begins to take a long > > > time for remote machines to copy files into the directory, maybe 4 > > > seconds for a 50kB file on a switched 100 base network. The performance > > > hit is worst with remote machines using SMB. Compressing the directory > > > entry with mkdir new cp old/* new/ > > > rm -rf old > > > mv old new > > > definitely improves things, but generally when there gets to be more > > > than 200k files we have to roll over to a new directory to keep things > > > moving. I suspect the remote machines are effectively downloading the > > > directory entry with each copy to the server, but I also see the smbd > > > tasks pegging on the server as well, but never really investigated it. > > > We see this with ext2 and ext3. Not really looking for a solution here > > > but just offering the info, but if anyone has a quick fix please share > > > it. I may try resiserfs someday but for now we just use thousands of > > > directories for the files. Mike > > > > which way do you normaly use to push the files when you don't use smb? > > NFS, though virtually all of the uploads to the directory are via SMB from > NT machines. The local machine will write files as well, maybe 5% of those > in the directory overall. Most of the file reads are from the local machine > which takes a little hit with big directories, though reads from the NT > machines over SMB get very slow but are infrequent. Splitting the files > into many directories has been an acceptable fix. -- e-admin internet gmbh Andreas Gietl tel +49 941 3810884 Ludwig-Thoma-Strasse 35 fax +49 89 244329104 93051 Regensburg mobil +49 171 6070008 PGP/GPG-Key unter http://www.e-admin.de/gpg.html _______________________________________________ Ext3-users@redhat.com https://www.redhat.com/mailman/listinfo/ext3-users
