Re: GlusterFS and Kafka

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



OK, seems that this works now.

A couple of questions:
- What do you think, are all these options necessary for Kafka?
- You wrote that there have to be kind of application profiles. So to find out, which set of options work is currently a matter of testing (and hope)? Or are there any experiences for MongoDB / ProstgreSQL / Zookeeper etc.?
- I am using Heketi and Dynamik Storage Provisioning together with Kubernetes. Can I set this volume options somehow by default or by volume plugin?

Thanks for you help... really appreciated.. Christopher

Vijay Bellur <vbellur@xxxxxxxxxx> schrieb am Mo., 22. Mai 2017 um 16:41 Uhr:
Looks like a problem with caching. Can you please try by disabling all performance translators? The following configuration commands would disable performance translators in the gluster client stack:

gluster volume set <volname> performance.quick-read off
gluster volume set <volname> performance.io-cache off
gluster volume set <volname> performance.write-behind off
gluster volume set <volname> performance.stat-prefetch off
gluster volume set <volname> performance.read-ahead off
gluster volume set <volname> performance.readdir-ahead off
gluster volume set <volname> performance.open-behind off
gluster volume set <volname> performance.client-io-threads off

Thanks,
Vijay


On Mon, May 22, 2017 at 9:46 AM, Christopher Schmidt <fakod666@xxxxxxxxx> wrote:
Hi all,

has anyone ever successfully deployed a Kafka (Cluster) on GlusterFS volumes?

I my case it's a Kafka Kubernetes-StatefulSet and a Heketi GlusterFS.
Needless to say that I am getting a lot of filesystem related exceptions like this one:

Failed to read `log header` from file channel `sun.nio.ch.FileChannelImpl@67afa54a`. Expected to read 12 bytes, but reached end of file after reading 0 bytes. Started read from position 123065680.

I limited the amount of exceptions with the log.flush.interval.messages=1 option, but not all...

best Christopher


_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux