Theodore Tso wrote: > > More thoughts: > > 1) It's not just about storage efficiency, but also about transfer > efficiency. Disk drives generally like to transfer hunks of data in > 16k to 64k at a time. So getting related pieces of small hunks of > data read at the same time, we can win big on performance. BUT, it's > extremely hard to do this at the filesystem level, since the > application is much more likely to know which micro-file of 16 bytes > is likely to be needed at the same time as some other micro-file which > is only 16 bytes long. > > 2) If you have millions of separate files, each 16 bytes long, and you > need to read a huge number of them, you can end up getting killed on > system call overhead. > > I remember having this argument with Hans Reiser at one point. His > argument was that parsing was evil; and should never have to be done. > (And if anyone has ever seen the vast quanties of garbage which is > generated when you implement an XML parser in Java, and the resulting > GC overhead I can't blame them for thinking this...) So his argument > was that instead of parsing a file like /etc/inetd.conf, there should > be an /etc/inetd.conf.d directory, and in that directory there might > be directory called telnet, and another one called ssh, and yet > another called smtp, and then you might have files such as: > > FILENAME CONTENTS > =============================================================== > > /etc/inetd.conf.d/telnet/port 23 > /etc/inetd.conf.d/telnet/protocol tcp > /etc/inetd.conf.d/telnet/flags nowait > /etc/inetd.conf.d/telnet/user root > /etc/inetd.conf.d/telnet/daemon /sbin/telnetd > > /etc/inetd.conf.d/ssh/port 22 > /etc/inetd.conf.d/ssh/protocol tcp > /etc/inetd.conf.d/ssh/flags nowait > /etc/inetd.conf.d/ssh/user root > /etc/inetd.conf.d/ssh/daemon /sbin/sshd > > etc. When I pointed out the system call overhead that would result > since instead of an open, read, close to read /etc/inetd.conf, you > would now need perhaps a hundred or more system calls do the > opendir/readir loop, and then individually opening, reading, and > closing each file, Hans had a solution ---- I have a different solution. Plugins into the VFS that handle special situations. I could do something like this manually in userland via loop/FUSE, but a more integrated solution could prove more useful. Thanks! -- Al - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
