Re: poor performance with encryption and SSL enabled

[Adam <adamnkraut@xxxxxxxxx>]

As I understand this error appear because BWA opening hg19.fa.pac twice. One handler for writing out and one for reading.

And second thread is trying to read some data which is not preset on disk yet.

On Mon, Mar 9, 2015 at 11:49 AM, Adam <adamnkraut@xxxxxxxxx> wrote:

Hi Jeff/all,

I took the recommendation of disabled the stripes. Now I just have encryption (at rest) and SSL enabled. The test I am running is a bwa indexing. Basic dd read/writes work fine and I don't see any errors in the gluster logs. Then when I try the bwa index I see the following:

/shared/perftest/bwa/bwa index -a bwtsw hg19.fa

[bwa_index] Pack FASTA... 26.29 sec

[bwa_index] Construct BWT for the packed sequence...

BWTIncConstructFromPacked() : Can't read from hg19.fa.pac : Unexpected end of file

These are my current volume settings:

glusterfs 3.6.2 built on Jan 22 2015 12:58:11

Volume Name: data

Type: Distribute

Volume ID: 55d1c37b-bfba-47d8-8467-0b28b0e04aa2

Status: Started

Number of Bricks: 3

Transport-type: tcp

Bricks:

Brick1: ip-10-9-0-32.ec2.internal:/export/brick

Brick2: ip-10-9-0-141.ec2.internal:/export/brick

Brick3: ip-10-9-0-142.ec2.internal:/export/brick

Options Reconfigured:

performance.open-behind: off

performance.write-behind: off

performance.quick-read: off

encryption.master-key: /root/keystore/master.key

features.encryption: on

auth.ssl-allow: *

server.ssl: on

client.ssl: on

auth.allow: *

There are no messages in the logs during the job. However there are some errors from previous lines:

[2015-03-09 15:21:47.868160] E [socket.c:2481:socket_poller] 0-data-client-0: poll error on socket

[2015-03-09 15:21:47.868184] E [socket.c:2481:socket_poller] 0-data-client-1: poll error on socket

[2015-03-09 15:21:47.868288] E [socket.c:2481:socket_poller] 0-data-client-2: poll error on socket

If I take out the encryption and leave just SSL mode on the bwa index is successful. SSL may be good enough for our needs but I would like to know if we have the option of at rest encryption. Any ideas? Many thanks in advance!

On Tue, Feb 24, 2015 at 12:33 PM, Jeff Darcy <jdarcy@xxxxxxxxxx> wrote:

> SSL certs are self-signed and generated on all servers. Combined into a
> glusterfs.ca in /etc/ssl. By itself the SSL is working well.

Glad to hear it.  ;)

> If I run dd or any i/o operations I see a flurry of these messages in the
> logs.
>
> [2015-02-24 16:58:51.144099] W [stripe.c:5288:stripe_internal_getxattr_cbk]
> (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1e0)[0x3fd0620550]
> (-->
> /usr/lib64/glusterfs/3.6.2/xlator/cluster/stripe.so(stripe_internal_getxattr_cbk+0x36a)[0x7f6a152a12ba]
> (-->
> /usr/lib64/glusterfs/3.6.2/xlator/protocol/client.so(client3_3_fgetxattr_cbk+0x174)[0x7f6a154db284]
> (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5)[0x3fd0e0ea75] (-->
> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x142)[0x3fd0e0ff02] )))))
> 0-data-stripe-3: invalid argument: frame->local


Have you tried encryption (at rest) without striping, or vice versa?  I
suspect some kind of bad interaction between the two, but before we go
down that path it would be nice to make sure they're working separately.

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users