My primary pain points in decreasing order of severity: 1. For years I've been battling alignment issues in virtual machines with no success. I can't take the performance hit or invest the time in twiddling alignment anymore. TCP overhead is nothing compared to extra writes due to disk alignment. iSCSI straight out of the box doesn't have this issue. 2. Zero distribution (CentOS/RedHat) integration. Having to maintain my own init scripts and making the system unload the aoe module before the bonding module on shutdown, start/stop the vblades, and various other bits of trickery has become a real hassle. 3. Despite claims that AoE use is accelerating I've never run into another person who uses it or even really knows what it is. I've never seen/heard about it in any trade magazine aside from a Linux Journal article years ago. Nobody stops in the AoE IRC channel except for disappointed "Age of Empires" players. Traffic on this list is nearly dead. The blog you set up is quiet. I've had an AoE google alert setup for years. There is nearly no chatter about it. I have had an RSS search feed for AoE on ServerFault for quite some time. It very rarely comes up. Currently, it hasn't been mentioned since July: http://serverfault.com/questions/412957/how-to-access-aoe-block-device-from-redhat-hypervisor And look at the comments. Ouch. There is a lot of support and training material for iSCSI and it is part of the RHCE exam. I have had to do all of my own in-house training for AoE and it scares customers that it is such a niche technology with so few people on earth knowing how to use it, despite its on-the-wire simplicity. It actually hurts my business for people to know I use AoE. At one point I thought I could sell it as a competitive advantage. Not only is it an operational hassle but it has turned into a competitive disadvantage. 4. While I have always advocated AoE on the basis of its simplicity, the actual number of steps needed to get iSCSI working on RedHat/CentOS are far fewer than AoE which in practice makes AoE not simple. I have now scripted/cookbooked the steps to getting iSCSI up and running and my junior admin can do it. Mounting an iSCSI volume is a minor task on the RHCE exam taken by relatively newbie Linux admins. The issues involved in getting AoE working reliably and especially the troubleshooting are beyond many administrators. I have puppet scripts to distribute kernel modules (have to distribute it and compile it per kernel if you aren't running the exact same version everywhere, and deal with a recompile when you yum update the kernel), manage overly complicated init scripts, tweak sysctls, manage vblade configs, etc. Using the builtin distro iSCSI none of this is necessary. I could perhaps resolve all of this myself by getting involved in Fedora and contributing patches to make AoE work as smoothly as iSCSI in RHEL. But so far I've been completely technically incapable of solving the alignment issue and as for the rest: Why should I invest so much time and effort duplicating the excellent work of the iSCSI people? Anne Hathaway of Coraid actually contacted me last year about working for Coraid. I was very tempted but simply unable to move away from San Diego. And the first thing I would have wanted to do would be to make AoE work well in RHEL as target and initiator which I really doubt Coraid would be willing to pay me to do... I have often had the feeling that AoE's lack of distro integration has been because Coraid would prefer we all spend tens of thousands on Coraid hardware and if the free version worked well enough there would be less incentive for people to do so. I understand that every business needs to make money. But I'm not sure I can get behind that as a business model. Last I looked (it's been a while) a Coraid box was all Supermicro hardware which I could assemble myself from Newegg. I seem to recall it was Plan 9 for the OS, a strange choice. I suspect it's something to do with licensing, which makes me somewhat uncomfortable. Again, they need to make money but I'm not interested in helping them to work against my own interest. I've got a business to run too, can't afford to purchase Coraid, and just wanted a decent ethernet SAN which, given today's level of commodity technology, is quite possible for a very reasonable price. I'm afraid I've hurt myself spending so much time on making AoE work. I wish AoE well and maybe some day I'll come back to it but for six years it has cost me way too much time (and therefore money) and now I'm a bit embarrassed by the whole thing. On Fri, Oct 19, 2012 at 06:19:13AM PDT, Ed Cashin spake thusly: > AoE use is accelerating rapidly, and iSCSI persists because even though it's not the best fit for same-LAN data storage, folks can still get it to work if they are willing and able to deal with the complexity, sacrifice the performance, and don't need the kind of scaling that virtualization and cloud deployments require. > > Although a lot of the expansion of AoE use lately has been by users of the Coraid HBA, there are still a lot of first-time AoE users using the coraid.com-distributed Linux initiator and also the kernel.org-distributed initiator. Recently there have been a lot of patches from Coraid to the Linux Kernel Mailing List for bringing the kernel.org-distributed driver up to date. For example, > > http://thread.gmane.org/gmane.linux.kernel/1377362 > > We also fixed a regression in the Linux kernel's network layer that affected AoE performance: > > http://thread.gmane.org/gmane.linux.network/243626 > > I don't have much time to participate on the aoetools-discuss mailing list right now, partly so that I can keep generating those patches, but I think pessimism is especially inappropriate today, when the use of AoE is accelerating along with the development of its associated technologies, including open source technologies. > > -- > Ed Cashin > ecashin@xxxxxxxxxx > > -- Tracy Reed ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct _______________________________________________ Aoetools-discuss mailing list Aoetools-discuss@xxxxxxxxxxxxxxxxxxxxx https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
