Hail devs,
Project Hail was, in part, conceived as an umbrella of libraries and
services enabling the mating of a well known, Internet-standard API with
a back-end that enables distributed storage. tabled is an example of
this: it provides an application front-end compatible with S3 API,
using Hail back-end services chunkd and CLD.
nfs4d[1] is a second, work-in-progress example. nfs4d is a fully
working NFSv4 front-end, waiting to be mated to the Hail back-end services.
A third example is something I poked at long ago, iSCSI. The vinzvault
announcement[2] got me thinking about the iSCSI target[3] daemon that I
had worked on, a while ago. vinzvault, sheepdog, DST, drbd, nbd and
iSCSI all attempt to provide remote network attached storage, usually
for storage on ephemeral virtual machines, similar to Amazon's Elastic
Block Storage (EBS) on their EC2 grid.
I dusted off my "itd" (iSCSI target daemon) project, fixed a bunch of
bugs, and got it working[4] in the hopes that this might be useful to
Hail or vinzvault or so.
itd is a remote iSCSI service exporting one or more slices of storage as
a standard SCSI device on your system. It is based off of
'netbsd-iscsi' in Fedora, which is in turn based off an old, open source
Intel codebase. netbsd-iscsi seemed a more pliable codebase than the
very-nice SCSI TGT project[5].
The web browsable itd tree (with git:// URL for cloning) can be found at
http://git.kernel.org/?p=daemon/distsrv/itd.git
As I write this email, I am borrowing a lot of networking code from
tabled, to convert from GNet over to the more-flexible TCP server
codebase found in tabled -- notably the asynchronous background TCP
writing code in tabled. Hopefully will finish and commit this by the
end of the weekend.
At that point, itd should be a fully compliant SCSI target, capable of
reading/writing -- to a pre-allocated RAM space. Once that milestone is
reached, the RAM storage may be replaced with Hail components, or other
gadgets like MongoDB[6], to provide scalable, distributed storage.
Jeff
[1] https://hail.wiki.kernel.org/index.php/Nfs4d
[2] http://www.mail-archive.com/linux-cluster@xxxxxxxxxx/msg08555.html
[3] a SCSI "target" is a remote network server, in SCSI parlance. It is
mated with an "initiator", which is SCSI's term for client.
[4] well, only small WRITEs work at the moment. but READ is fully
working at high speeds.
[5] http://stgt.sourceforge.net/
[6] http://www.mongodb.org/
--
To unsubscribe from this list: send the line "unsubscribe hail-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
