Re: USB drive on server?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




----- Original Message ----- From: "John Austin" <ja@xxxxxxxxxx>
Newsgroups: gmane.linux.redhat.fedora.general
To: "For users of Fedora" <fedora-list@xxxxxxxxxx>
Sent: Wednesday, October 24, 2007 6:15 AM
Subject: Re: USB drive on server?


On Tue, 2007-10-23 at 16:32 -0700, Ted Marshall wrote:
From: "John Austin" <ja@xxxxxxxxxx>
Newsgroups: gmane.linux.redhat.fedora.general
To: <tim@xxxxxxxxxxxxxxxxxxxxxx>; "For users of Fedora"
<fedora-list@xxxxxxxxxx>
Sent: Tuesday, October 23, 2007 9:40 AM
Subject: Re: USB drive on server?


> Dup from other thread
>
>> Over 10 year life time
>
> http://www.corsairmemory.com/_faq/FAQ_flash_drive_wear_leveling.pdf

The problem with the statistics given for Dynamic Wear Leveling is that the
flash controller must be able to know which blocks are not in use by the
filesystem.  AFAIK (and this is very difficult to verify one way or the
other), consumer grade devices (at least) are only aware of FAT filesystems and only if there is only one partition on the device. If you use ext2/3 or
have multiple partitions (both typical for Linux) the controller cannot
track blocks being freed by the filesystem and so any block ever written by
the host will looks busy to the flash controller.  Therefore, writes will
only rotate through the (small) pool of extra blocks maintained by the flash
controller.
This is where I hope you are wrong !!!!!!!! See below

Also, AFAIK, few if any consumer-grade and, I suspect, few
professional-grade flash devices do Static Wear Leveling.

The result is that the flash drive will burn out much faster than the
statistics quoted. How much faster, I am unable to compute. It may still
be adequate for your needs.  I don't know.



You have made me have a very careful read of the the Corsair info
and to query my understanding!!

Assuming the Corsair info true then my understanding is as follows

The Flash Voyager GT uses Dynamic Wear Levelling

I am not convinced that it knows about VFAT, NTFS, ext2 ...

The stick does not know about which is static data and which is dynamic,
all its knows is that if it is written to then it has to find
somewhere to put it.

The Corsair info suggests that there is more memory on the stick than
you can get at, See Fig 3, I do not know if this is true.

The stick does not need to know which bits are freed by the
operating system, only which bits have been freed by the
wear levelling look up table.

When the operating system deletes a file then it will free up
a certain number of cylinders/tracks/sectors as "it" understands them.
At a later time the operating system will write to those cylinders/tracks/sectors again. At that time the stick will remap them to different locations on the stick,
freeing up the previously used physical memory.

My F7 stick layout is shown below.

I conclude that there is approx 6GB of "Static" data and 2GB of
Dynamic data (Free Space as far as Linux is concerned).

Thus if there is 2GB "free" as understood by the operating system then this
is "almost" equivalent to 2GB being "free" to be mapped by the stick.

The bit that gets the hammering will be the superblocks - every time
the inode access time is updated for either read or write.
The superblock data will be moving around on the stick as
a result of the wear levelling, however the wear levelling
will only be applied over 2GB instead of 8GB.

If I remember correctly then the superblock (or hopefully only part of it)
will be written every 30 seconds.

http://www.storagesearch.com/siliconsys-art1.html
http://www.stec-inc.com/downloads/AN-0702_STEC_SMALL_CARDS_WEAR_LEVELING_LIFETIME_CALCULATOR.pdf

both give a formulae to compute the lifetime of wear levelled devices.

Filling in the first as follows gives a very large number of years!!
The 100,000 write cycle figure comes from the Corsair info
A bit of superblock = 10kB ????????

[(8000 - 6000)*100,000*(1 - 0)]/[(0.01*2 writes/min)*525,600]
= 19,026 years.

I conclude:

You should NEVER use the stick for long periods when it is
close to full as the wear levelling will only be able
to act over the remaining memory available at that time
and will become ineffective.

I would be very grateful for feed back on this !!

I still have a 10 year warranty to fall back on !!

As an aside
Is it a good idea to run "/" mounted without "atime" ??

John



[root@corsair ~]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2              7549152   4869528   2296148  68% /
/dev/sda1                41053     19301     19632  50% /boot
tmpfs                  1684780         0   1684780   0% /dev/shm
[root@corsair ~]#
Disk /dev/sda: 8287 MB, 8287944704 bytes
241 heads, 32 sectors/track, 2098 cylinders
Units = cylinders of 7712 * 512 = 3948544 bytes

  Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1          11       42400   83  Linux
/dev/sda2   *          12        2000     7669584   83  Linux
/dev/sda3 2001 2098 377888 82 Linux swap / Solaris





--
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list


Blocks that have been freed in the filesystem are still static data as far as the flash controller is concerned, unless the controller can understand the FS metadata. Yes, when they are reallocated by the FS, the controller will be able to move that to another free flash block, freeing the old block. However, as long as they just sit unused by the FS, they are unavailable to the flash controller for wear leveling unless "static" wear leveling is used. As I see it, the worst case is where the filesystem has touched all the blocks that the controller makes visible and then one single-block file (or the superblock) is constantly rewritten in place. The controller will only be able to wear level between that block and its spare-block pool.

So the question is "how big is the spare-block pool"?

http://www.sandisk.com/Assets/File/OEM/WhitePapersAndBrochures/RS-MMC/WPaperWearLevelv1.0.pdf was published by Sandisk 4 years ago. Here's an early paragraph:

"Each memory chip is divided into blocks. A block is an array of memory cells organized as sectors. The number of blocks and sectors vary from product to product. The minimum unit for a write or read operation is a page (or sector). The minimum unit for an erase operation is a block. Physical blocks are logically grouped into zones. For the current technology, a typical zone size is 4 MB. However, this may change from product to product. Wear leveling is done within a zone. The current firmware does not spread the wear across the capacity of the card. Each zone has about 3% additional "spare blocks" beyond what is assigned to meet the logical capacity of the
flash card. This group of blocks is commonly referred to as the "Erase Pool"."

On the other hand, your Corsair PDF talks of a 2GB device having a write page size of 2KB, an erase block size of 128KB (64 pages) and a leveling management unit of 128MB (1024 blocks). Unfortunately, it does not tell us how many spare blocks are in a leveling unit. I'll assume the 3% from Sandisk and this gives us about 32 spare blocks.

Next, we have to ask what happens if only one page is changes in the block? I'm going to assume the worst case that this causes the whole block to be moved to a new block and the old one freed. Corsair states that they use SLC technology for 100,000 erase cycle life instead of MLC with 10,000 so I will use this assumption. (I have read specifications for other devices for 10,000 cycle lifetime so I'd but that the cheaper devices d ouse MLC).

Ok, so assume that the superblock does get rewritten every 30 seconds (I do not know this for a fact). The lifetime of the wear management unit containing the superblock is 32 * 100000 writes or 3 years. OK, that may be ok for you. On the other hand, if the device uses MLC technology, devide by 10 and you only get 3.6 months!!

Now it may be that some devices use better algorithms and will give you a longer life. (I would expect that a US$5000 "sold-state disk" will have implemented things better and would have a much longer life.) On the other hand, if you run out to Fry's and buy a $10 1GB USB stick, don't expect it to last vey long.

One problem I've seen is that no manufacturer gives you all of the information you need. Also, what is true for one device may not be true for another.

It seems to me that one way to extend the life is to use ext3 with a large "ordered" journal that will spread out the writes to the superblock throughout the journal (I assume the superblock is journaled, anyone know for sure?). Also, turn off atime on the filesystem.

What we need is a version of JFFS that can be used on an external device. Assume that the device does not do wear leveling and do it in the OS.


--
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora Magazine]     [Fedora News]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [SSH]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux