I have few questions regarding how we plan to use our new Fedora Cloud: 1) Do we want to use Swift there? How much space we should allocate for Swift storage? 2) How should I configure /var/lib/nova/instances? This is directory where RootFS, swap and ephemeral storages of instances are stored. Right now fed-cloud09 have this path on root mount point where is 15 GB available. But it can be extended to whole size of "vg_server" volume group, which is 383GB. fed-cloud1X have now: /dev/mapper/vg_guests-nova 3.2T 32G 3.0T 2% /var/lib/nova This gives us accumulated (over controller and 6 compute nodes) storage of 18.4 TB storage. However this setup does not allows live migration, because /var/lib/nova is not on shared FS. So I propose that /var/lib/nova/instances is NFS exported from fed-cloud09 to all compute nodes. This will allow live migrations of VMs. However this means that /var/lib/nova/instances on fed-cloud09 should be big enough to host all rootfs, swaps and ephemeral devices for all VMs in Fedora Cloud. This is 3-20 GB per instance. so 383 GB is not enough. Other disadvantage is that ephemeral devices will be on network storage and not so fast as local device. However I suppose we do not use ephemeral storage too much (maybe not at all). I would use /dev/md127 from fed-cloud09 (2.89 TB) for /var/lib/nova. This should be enough for approximately 287 instances. Which is cca accumulated capacity of our compute nodes. However it may slightly limit us in future if we ever add more compute nodes. I would suggest to use /dev/md127 (2.89TB raid6 storage) on fed-cloud1X(s) as storage for Swift. And configure Glance to store images in Swift. This have advantage that Swift can be set up to store more replicas. On the other hand this is little bit overkill. As we would have 3TB+ (it depend how much replicas we will use) storage for Swift. However biggest consumer would be Glance. And Glance in fed-cloud02 only consume 100GB currently. What you like most, hove more space for /var/lib/nova as in current setup but without live migrations? Or smaller storage, which on the other hand allows live migration? Or you have better idea how to split disks and machines to different services? P.S. I am putting aside normal volumes for data, which will reside in DellEqualogic which is 20TB. And it is perfectly fine. -- Miroslav Suchy, RHCE, RHCDS Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys _______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
