On Thu, Feb 18, 2021 at 01:27:09PM -0800, Erik Jensen wrote: > On 2/18/21 4:15 AM, Matthew Wilcox wrote: > > > On Thu, Feb 18, 2021 at 04:54:46PM +0800, Qu Wenruo wrote: > > > Recently we got a strange bug report that, one 32bit systems like armv6 > > > or non-64bit x86, certain large btrfs can't be mounted. > > > > > > It turns out that, since page->index is just unsigned long, and on 32bit > > > systemts, that can just be 32bit. > > > > > > And when filesystems is utilizing any page offset over 4T, page->index > > > get truncated, causing various problems. > > 4TB? I think you mean 16TB (4kB * 4GB) > > > > Yes, this is a known limitation. Some vendors have gone to the trouble > > of introducing a new page_index_t. I'm not convinced this is a problem > > worth solving. There are very few 32-bit systems with this much storage > > on a single partition (everything should work fine if you take a 20TB > > drive and partition it into two 10TB partitions). > For what it's worth, I'm the reporter of the original bug. My use case is a > custom NAS system. It runs on a 32-bit ARM processor, and has 5 8TB drives, > which I'd like to use as a single, unified storage array. I chose btrfs for > this project due to the filesystem-integrated snapshots and checksums. > Currently, I'm working around this issue by exporting the raw drives using > nbd and mounting them on a 64-bit system to access the filesystem, but this > is very inconvenient, only allows one machine to access the filesystem at a > time, and prevents running any tools that need access to the filesystem > (such as backup and file sync utilities) on the NAS itself. > > It sounds like this limitation would also prevent me from trying to use a > different filesystem on top of software RAID, since in that case the logical > filesystem would still be over 16TB. > > > As usual, the best solution is for people to stop buying 32-bit systems. > I purchased this device in 2018, so it's not exactly ancient. At the time, > it was the only SBC I could find that was low power, used ECC RAM, had a > crypto accelerator, and had multiple sata ports with port-multiplier > support. I'm sorry you bought unsupported hardware. This limitation has been known since at least 2009: https://lore.kernel.org/lkml/19041.4714.686158.130252@notabene.brown/ In the last decade, nobody's tried to fix it in mainline that I know of. As I said, some vendors have tried to fix it in their NAS products, but I don't know where to find that patch any more. https://bootlin.com/blog/large-page-support-for-nas-systems-on-32-bit-arm/ might help you, but btrfs might still contain assumptions that will trip you up.
