This marks an interesting development in GlusterFS. If you've been looking for data encryption, you may want to try this patch.
-JM
-------- Original Message --------
Subject: Change in glusterfs[master]: Transparent data encryption and metadata authentication...
From: "Edward Shishkin (Code Review)" <root@xxxxxxxxxxxxxxx>
To:
CC:
Edward Shishkin has uploaded a new change for review.
Change subject: Transparent data encryption and metadata authentication in the systems with non-trusted server
......................................................................
Transparent data encryption and metadata authentication
in the systems with non-trusted server
This new functionality can be useful in various cloud technologies.
It is implemented via a pair of the following interacting translators:
. encryption/crypt, which works on client side and performs encryption
and authentication;
. features/oplock, which works on server side and resolves "conflicts"
1. The class of algorithms for data encryption,
that can be supported by this pair of translators
The mentioned pair of translators can support any atomic symmetric
block cipher algorithms (which require to pad plain/cipher text before
performing encryption/decryption transform (see glossary in atom.c
for definitions). In particular, it can support algorithms with the
EOF issue (which require to pad the end of file by extra-data).
In most cases crypt translator translates the pair (offset, count)
passed by user to different values, and resolves individual ->write()
->truncate(), etc. file operations to read-modify-write sequences.
A volume can contain files encrypted by different algorithms. For
newly created files one can specify desirable algorithm at mount time
via a respective option of crypt translator.
Currently only one algorithm is supported: AES_XTS.
Example of algorithms, which can not be supported by this pair of
translators:
1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA;
2. Symmetric block cipher algorithms with inline MACs for data
authentication.
2. Implementation notes.
a) Atomic algorithms
Since any process in a stackable file system manipulates with local
data (which can be obsoleted by local data of another process), atomic
cipher algorithms without proper support can lead to non-POSIX
behavior. To resolve the "collisions" we introduce a special helper
translator (features/oplock), which works on the server and manages
requests (queues, grants access) of read/write issued by the clients.
When an exclusive access is granted to client, the last one performs
cipher transform and proceeds the stack. After all the client's work is
done on the server, the oplock translator drops the access, and wakes
up the next request in the queue (if any). Our implementation
guarantees that an access will be granted to every concurrent process,
which accesses the same file (i.e. the process won't hang).
b) Algorithms with EOF issue
Such algorithms require to pad the end of file with some extra-data.
Without proper support this will result in losing information about
real file size. Keeping a track of real file size is a responsibility
of the mentioned features/oplock translator. When writing/truncating a
file, the oplock translator cuts the padding and stores the last one
as a special extended attribute with the key "trusted.ceof". When
reading a file, oplock translator appends the respective padding. So,
in the bricks every file has its real size.
Comment. This makes transparent encryption incompatible with GlusterFS
striping and replication translators, which spawn extra-writes to
stripe/replica files without due interaction with the oplock translator.
3. Non-trusted servers and
Metadata authentication
We assume that server, where user's data is stored on is non-trusted.
It means that the server can be subjected to various attacks directed
to reveal user's encrypted personal data. We provide protection
against such attacks.
Every encrypted file has specific private attributes (cipher algorithm
id, atom size, trusted object id, etc), which are packed to a string
(so-called "format string") and stored as a special extended attribute
with the key "trusted.cfmt". We protect the string from tampering.
This protection is mandatory, hardcoded and is always on. Without such
protection various attacks (based on extending the scope of per-file
secret keys) are possible.
Our authentication method has been developed in tight collaboration
with Red Hat security team and is implemented as "metadata loader of
version 1" (see file metadata.c). This method is NIST-compliant and is
based on checking 8-byte per-link MACs created(updated) by
FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the
following unique entities:
. link name;
. trusted file's uuid, specially created on the (trusted) client side
Every time, before manipulating with a file, we check it's MACs at
FOP->open() time. Some FOPs don't require a file to be opened (e.g.
FOP->truncate()). In such cases the crypt translator opens the file
mandatory.
4. Generating keys
Unique per-file keys are derived by NIST-compliant methods (file keys.c)
from the
a) parent key;
b) unique trusted object-id of the file;
Per-volume master key, provided by user at mount time is in the root
of this "tree of keys".
Those keys are used to:
1) encrypt/decrypt file data;
2) encrypt/decrypt file metadata;
3) create per-file and per-link MACs for metadata authentication.
5. Instructions
how to use the new functionality
1) Specify an option "encrypt" when creating a volume.
Example:
# gluster volume create myvol encrypt pepelac:/root/exp8
2) On the client side, when mounting a volume, specify the absolute
name of the file, which contains per-volume master key, overriding
the option "key" of the crypt translator. This file should contain
256-bit AES key in the hex form, i.e. 64 symbols. Crypt translator
accepts the first 64 symbols of the specified file.
Other extra-symbols are ignored.
After successful mount the file with master key may be removed.
Example:
# glusterfs --xlator-option=myvol-crypt.key=/home/edward/mykey \
--volfile-id=myvol --volfile-server=pepelac /mnt/gluster
WARNING! Losing the master key means losing the data of the whole
volume without any chances to recovery.
6. Options of the crypt translator
. "key" (specifies names of the file which contains per-volume master
key); . "kbits" (specifies size of per-file key for data encryption),
possible values:
. "256" default value
. "512"
. "blocksize" (specifies the atom size), possible values:
. "512"
. "1024"
. "2048"
. "4096" default value;
. id of algorithm for data encryption (hidden option);
. id of metadata loader (hidden option);
7. Test cases
Any workload, which involves the following file operations:
->create();
->open();
->readv();
->writev();
->truncate();
->ftruncate();
->link();
->unlink();
->rename();
8. TODOs:
1) Currently iov_len coincides with atom_size (4K by default). We can
introduce larger units for IOs to improve performance.
2) Show encryption status (on/off) of the volume in gluster volume info.
Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee
Signed-off-by: Edward Shishkin <edward@xxxxxxxxxx>
---
M cli/src/cli-cmd-parser.c
M configure.ac
M doc/gluster.8
M xlators/encryption/Makefile.am
A xlators/encryption/crypt/Makefile.am
A xlators/encryption/crypt/src/Makefile.am
A xlators/encryption/crypt/src/atom.c
A xlators/encryption/crypt/src/crypt-common.h
A xlators/encryption/crypt/src/crypt-mem-types.h
A xlators/encryption/crypt/src/crypt.c
A xlators/encryption/crypt/src/crypt.h
A xlators/encryption/crypt/src/data.c
A xlators/encryption/crypt/src/keys.c
A xlators/encryption/crypt/src/metadata.c
A xlators/encryption/crypt/src/metadata.h
M xlators/features/Makefile.am
A xlators/features/oplock/Makefile.am
A xlators/features/oplock/src/Makefile.am
A xlators/features/oplock/src/oplock-mem-types.h
A xlators/features/oplock/src/oplock.c
A xlators/features/oplock/src/oplock.h
M xlators/mgmt/glusterd/src/glusterd-volgen.c
M xlators/mgmt/glusterd/src/glusterd-volume-ops.c
M xlators/mgmt/glusterd/src/glusterd.h
24 files changed, 10,159 insertions(+), 19 deletions(-)
git pull ssh://git.gluster.org/glusterfs refs/changes/67/4667/1
--
To view, visit http://review.gluster.org/4667
To unsubscribe, visit http://review.gluster.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee
Gerrit-PatchSet: 1
Gerrit-Project: glusterfs
Gerrit-Branch: master
Gerrit-Owner: Edward Shishkin <edward.shishkin@xxxxxxxxx>
