> On Sunday 18 May 2003 18:01, Michael Harris wrote: > > Just to add, I can attest that moving the files from the old dir to the new > > as described improves performance on my machines dramatically. In our > > service we end up with directories of 150k+ files which are generally > > touched only as they are added, though every file will be touched several > > times over a month. The files are each around 50kB. When the directory > > entry gets to be about 4MB it begins to take a long time for remote > > machines to copy files into the directory, maybe 4 seconds for a 50kB file > > on a switched 100 base network. The performance hit is worst with remote > > machines using SMB. Compressing the directory entry with mkdir new > > cp old/* new/ > > rm -rf old > > mv old new > > definitely improves things, but generally when there gets to be more than > > 200k files we have to roll over to a new directory to keep things moving. I > > suspect the remote machines are effectively downloading the directory entry > > with each copy to the server, but I also see the smbd tasks pegging on the > > server as well, but never really investigated it. We see this with ext2 and > > ext3. Not really looking for a solution here but just offering the info, > > but if anyone has a quick fix please share it. I may try resiserfs someday > > but for now we just use thousands of directories for the files. > > Mike > > which way do you normaly use to push the files when you don't use smb? NFS, though virtually all of the uploads to the directory are via SMB from NT machines. The local machine will write files as well, maybe 5% of those in the directory overall. Most of the file reads are from the local machine which takes a little hit with big directories, though reads from the NT machines over SMB get very slow but are infrequent. Splitting the files into many directories has been an acceptable fix. _______________________________________________ Ext3-users@redhat.com https://www.redhat.com/mailman/listinfo/ext3-users
