On Sun, Jan 19, 2014 at 7:42 AM, Eliezer Croitoru <eliezer@xxxxxxxxxxxx> wrote:
> While working here and there I have seen that ZFS is a very robust FS.
> I will not compare it to any others because there is no need for that.
>
> OK so zfs, ext3, ext4 and others are FS which sits on SPINNING disks or
> flash drives.
> The SATA and SAS interfaces are limited to a serial interface in the limits
> (by a standard) to 3,6 Gbps.
> So SATA is max 6Gbps while a NIC can have a bandwidth of 10Gbps.
> Are there any real reasons to not use a 10Gbps line?
At least two come to mind: first, behind that 10Gbps line there are
still spinning disks and/or SSDs; the transmission line is simply
going to add some latency. Maybe not much (10% overhead over spinning
disks due to processing, propagation, transmission, error correction
etc). Bandwidth is important, but latency is even more so.
Second, packet loss: SATA, SAS and FC guarantee 0% packet loss. If
there is any, it is immediately detected, and the data is
retransmitted. On Ethernet, you're not so sure. On IP-over-ethernet,
even less. I was told that a 1% packet loss is enough to completely
kill transmission performance in a FCoE environment, and that's the
reason why people who do FCoE use special ("converged") adapters,
which look more like FC adapters than to Ethernet adapters.
> For example if I have 10 SAS or SATA disks SSD or Spinning under a machine
> with 128GB of ram it is possible to allow some flow of data to be used and
> be "faster" then only one and even SSD drive.
>
> A dual 10Gbps machine can potentially be faster in lots of aspects then a
> local disk.
>
> I do not have the answer but a NAS might be sometimes the right choice as a
> cache storage.
Benchmarks are welcome :)
> Indeed there are overheads for each and every TCP connection and indeed
> there are many aspects that needs to be tested and verified but I still do
> suspect that there are some assumptions that needs to be verified to make
> sure that a SAN\NAS might worth a lot then it is assumed to be.
The main advantage I can think of for using a NAS is that these
usually have huge RAM caches, which can help by keeping the directory
structure in RAM thus making small file retrieval faster than doing
multiple roundtrips to disk.
--
/kinkie
