Re: Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, May 08, 2017 at 02:23:53PM -0400, Kaleb S. KEITHLEY wrote:
> On 05/08/2017 05:32 AM, Pranith Kumar Karampuri wrote:
> >           We released gluster-block v0.2 just this Friday for which RHEL
> > packages are yet to be built.
> > 
> > +Kaleb,
> >       Could you help with this please?
> 
> Of course this doesn't build in EPEL, where glusterfs-api-devel is not
> available. EPEL does not have packages that exist in RHEL; RHEL has
> glusterfs, but not glusterfs-api-devel, which is in a channel other than the
> base for reasons I don't understand.
> 
> There's no way to do scratch builds in CentOS's Storage SIG. We can add it
> to the Storage SIG after it has been accepted into Fedora. I asked
> previously, but didn't get an answer, about the status is of getting this
> into Fedora.

Actually, we can do scratch build of packages for the CentOS Storage
SIG. We could also do nightly builds of gluster-block and run automated
tests against it. The reason we have not looked into this yet, are:

1. The package is not in Fedora yet.
2. CentOS does not have the required updated versions of tcmu-runner and
   targetcli yet. These will hopefully become available with the next
   RHEL7.x minor release, whenever that is...

> The best option ATM is to do a mockchain build — which anyone can do — using
> the glusterfs src.rpm, e.g. from the Storage SIG, and the gluster-block
> src.rpm.

'mockchain' is a command provided by the 'mock' package, and you can
find that in Fedora EPEL. You will need to take the targetcli and
tcmu-runner source packages from a recent Fedora, and rebuild everything
on RHEL (or CentOS).

Or, you could run gluster-block on Fedora. Building the package there
should be trivial, and all the other dependencies are available.

Good luck!
Niels

> 
> > 
> > Prasanna,
> >       Could you let Abhijit know the rpm versions for tcmu-runner and
> > other packages so that this feature can be used?
> > 
> > 
> > On Mon, May 8, 2017 at 2:49 PM, Pranith Kumar Karampuri
> > <pkarampu@xxxxxxxxxx <mailto:pkarampu@xxxxxxxxxx>> wrote:
> > 
> >     Wait, wait we are discussing this issue only. Expect a reply in some
> >     time :-)
> > 
> >     On Mon, May 8, 2017 at 2:19 PM, Abhijit Paul
> >     <er.abhijitpaul@xxxxxxxxx <mailto:er.abhijitpaul@xxxxxxxxx>> wrote:
> > 
> >         poking for previous mail reply
> > 
> >         On Sun, May 7, 2017 at 1:06 AM, Abhijit Paul
> >         <er.abhijitpaul@xxxxxxxxx <mailto:er.abhijitpaul@xxxxxxxxx>> wrote:
> > 
> >             https://pkalever.wordpress.com/2017/03/14/elasticsearch-with-gluster-block/
> >             <https://pkalever.wordpress.com/2017/03/14/elasticsearch-with-gluster-block/>
> >             here used tested environment is Fedora ,
> >             but i am using RHEL based Oracle linux so does gluster-block
> >             compatible with RHEL as well? What i needs to change & make
> >             it work?
> > 
> >             On Fri, May 5, 2017 at 5:42 PM, Pranith Kumar Karampuri
> >             <pkarampu@xxxxxxxxxx <mailto:pkarampu@xxxxxxxxxx>> wrote:
> > 
> > 
> > 
> >                 On Fri, May 5, 2017 at 5:40 PM, Pranith Kumar Karampuri
> >                 <pkarampu@xxxxxxxxxx <mailto:pkarampu@xxxxxxxxxx>> wrote:
> > 
> > 
> > 
> >                     On Fri, May 5, 2017 at 5:36 PM, Abhijit Paul
> >                     <er.abhijitpaul@xxxxxxxxx
> >                     <mailto:er.abhijitpaul@xxxxxxxxx>> wrote:
> > 
> >                         So should i start using gluster-block with
> >                         elasticsearch in kubernetes environment?
> > 
> >                         My expectation from gluster-block is, it should
> >                         not CorruptIndex of elasticsearch...and issue
> >                         facing in previous mails.
> > 
> >                         Please let me know whether should i
> >                         processed with above mentioned combination.
> > 
> > 
> >                     We are still in the process of fixing the failure
> >                     scenarios of tcmu-runner dying and failingover in
> >                     the multipath scenarios.
> > 
> > 
> >                 Prasanna did test that elasticsearch itself worked fine
> >                 in gluster-block environment when all the machines are
> >                 up etc i.e. success path. We are doing failure path
> >                 testing and fixing things at the moment.
> > 
> > 
> >                         On Fri, May 5, 2017 at 5:06 PM, Pranith Kumar
> >                         Karampuri <pkarampu@xxxxxxxxxx
> >                         <mailto:pkarampu@xxxxxxxxxx>> wrote:
> > 
> >                             Abhijit we just started making the efforts
> >                             to get all of this stable.
> > 
> >                             On Fri, May 5, 2017 at 4:45 PM, Abhijit Paul
> >                             <er.abhijitpaul@xxxxxxxxx
> >                             <mailto:er.abhijitpaul@xxxxxxxxx>> wrote:
> > 
> >                                 I yet to try gluster-block with
> >                                 elasticsearch...but carious to know does
> >                                 this combination plays well in
> >                                 kubernetes environment?
> > 
> >                                 On Fri, May 5, 2017 at 12:14 PM, Abhijit
> >                                 Paul <er.abhijitpaul@xxxxxxxxx
> >                                 <mailto:er.abhijitpaul@xxxxxxxxx>> wrote:
> > 
> >                                     thanks Krutika for the alternative.
> > 
> >                                     @*Prasanna @**Pranith*
> >                                     I was going thorough the mentioned
> >                                     blog post and saw that used tested
> >                                     environment was Fedora ,
> >                                     but i am using RHEL based Oracle
> >                                     linux so does gluster-block
> >                                     compatible with RHEL as well?
> > 
> >                                     On Fri, May 5, 2017 at 12:03 PM,
> >                                     Krutika Dhananjay
> >                                     <kdhananj@xxxxxxxxxx
> >                                     <mailto:kdhananj@xxxxxxxxxx>> wrote:
> > 
> >                                         Yeah, there are a couple of
> >                                         cache consistency issues with
> >                                         performance translators that are
> >                                         causing these exceptions.
> >                                         Some of them were fixed by
> >                                         3.10.1. Some still remain.
> > 
> >                                         Alternatively you can give
> >                                         gluster-block + elasticsearch a
> >                                         try, which doesn't require
> >                                         solving all these caching issues.
> >                                         Here's a blog post on the same -
> >                                         https://pkalever.wordpress.com/2017/03/14/elasticsearch-with-gluster-block/
> >                                         <https://pkalever.wordpress.com/2017/03/14/elasticsearch-with-gluster-block/>
> > 
> >                                         Adding Prasanna and Pranith who
> >                                         worked on this, in case you need
> >                                         more info on this.
> > 
> >                                         -Krutika
> > 
> >                                         On Fri, May 5, 2017 at 12:15 AM,
> >                                         Abhijit Paul
> >                                         <er.abhijitpaul@xxxxxxxxx
> >                                         <mailto:er.abhijitpaul@xxxxxxxxx>>
> >                                         wrote:
> > 
> >                                             Thanks for the reply, i will
> >                                             try it out but i am also
> >                                             facing one more issue "i.e.
> >                                             replicated volumes returning
> >                                             different timestamps"
> >                                             so is this because of Bug
> >                                             1426548 - Openshift Logging
> >                                             ElasticSearch FSLocks when
> >                                             using GlusterFS storage
> >                                             backend
> >                                             <https://bugzilla.redhat.com/show_bug.cgi?id=1426548> ?
> > 
> >                                             *FYI i am using glusterfs
> >                                             3.10.1 tar.gz*
> >                                             *
> >                                             *
> >                                             Regards,
> >                                             Abhijit
> > 
> > 
> > 
> >                                             On Thu, May 4, 2017 at 10:58
> >                                             PM, Amar Tumballi
> >                                             <atumball@xxxxxxxxxx
> >                                             <mailto:atumball@xxxxxxxxxx>> wrote:
> > 
> > 
> > 
> >                                                 On Thu, May 4, 2017 at
> >                                                 10:41 PM, Abhijit Paul
> >                                                 <er.abhijitpaul@xxxxxxxxx <mailto:er.abhijitpaul@xxxxxxxxx>>
> >                                                 wrote:
> > 
> >                                                     Since i am new to
> >                                                     gluster, can please
> >                                                     provide how to turn
> >                                                     off/disable "perf
> >                                                     xlator options"?
> > 
> > 
> >                                                 $ gluster volume set
> >                                                 <volname>
> >                                                 performance.stat-prefetch off
> >                                                 $ gluster volume set
> >                                                 <volname>
> >                                                 performance.read-ahead off
> >                                                 $ gluster volume set
> >                                                 <volname>
> >                                                 performance.write-behind off
> >                                                 $ gluster volume set
> >                                                 <volname>
> >                                                 performance.io-cache off
> >                                                 $ gluster volume set
> >                                                 <volname>
> >                                                 performance.quick-read off
> > 
> > 
> >                                                 Regards,
> >                                                 Amar
> > 
> > 
> >                                                         On Wed, May 3,
> >                                                         2017 at 8:51 PM,
> >                                                         Atin Mukherjee
> >                                                         <amukherj@xxxxxxxxxx
> >                                                         <mailto:amukherj@xxxxxxxxxx>>
> >                                                         wrote:
> > 
> >                                                             I think
> >                                                             there is
> >                                                             still some
> >                                                             pending
> >                                                             stuffs in
> >                                                             some of the
> >                                                             gluster perf
> >                                                             xlators to
> >                                                             make that
> >                                                             work
> >                                                             complete.
> >                                                             Cced the
> >                                                             relevant
> >                                                             folks for
> >                                                             more
> >                                                             information.
> >                                                             Can you
> >                                                             please turn
> >                                                             off all the
> >                                                             perf xlator
> >                                                             options as a
> >                                                             work around
> >                                                             to move forward?
> > 
> >                                                             On Wed, May
> >                                                             3, 2017 at
> >                                                             8:04 PM,
> >                                                             Abhijit Paul
> >                                                             <er.abhijitpaul@xxxxxxxxx
> >                                                             <mailto:er.abhijitpaul@xxxxxxxxx>>
> >                                                             wrote:
> > 
> >                                                                 Dear folks,
> > 
> >                                                                 I setup
> >                                                                 Glusterfs(3.10.1)
> >                                                                 NFS type
> >                                                                 as
> >                                                                 persistence
> >                                                                 volume
> >                                                                 for
> >                                                                 Elasticsearch(5.1.2)
> >                                                                 but
> >                                                                 currently facing
> >                                                                 issue
> >                                                                 with
> >                                                                 *"CorruptIndexException"
> >                                                                 *with
> >                                                                 Elasticseach
> >                                                                 logs and
> >                                                                 due to
> >                                                                 that
> >                                                                 index
> >                                                                 health
> >                                                                 trued
> >                                                                 RED in
> >                                                                 Elasticsearch.
> > 
> > 
> >                                                                 Later
> >                                                                 found
> >                                                                 that
> >                                                                 there
> >                                                                 was an
> >                                                                 issue
> >                                                                 with
> >                                                                 gluster
> >                                                                 < 3.10
> >                                                                 (https://bugzilla.redhat.com/show_bug.cgi?id=1390050
> >                                                                 <https://bugzilla.redhat.com/show_bug.cgi?id=1390050>)
> >                                                                 but even
> >                                                                 after
> >                                                                 *upgrading
> >                                                                 to
> >                                                                 3.10.1
> >                                                                 issue is
> >                                                                 still
> >                                                                 there.*
> >                                                                 *
> >                                                                 *
> >                                                                 *So
> >                                                                 curios
> >                                                                 to know
> >                                                                 what
> >                                                                 would be
> >                                                                 the root
> >                                                                 cause to
> >                                                                 fix this
> >                                                                 issue.*
> > 
> >                                                                 Regards,
> >                                                                 Abhijit
> > 
> >                                                                 _______________________________________________
> >                                                                 Gluster-users
> >                                                                 mailing list
> >                                                                 Gluster-users@xxxxxxxxxxx
> >                                                                 <mailto:Gluster-users@xxxxxxxxxxx>
> >                                                                 http://lists.gluster.org/mailman/listinfo/gluster-users
> >                                                                 <http://lists.gluster.org/mailman/listinfo/gluster-users>
> > 
> > 
> > 
> > 
> > 
> >                                                     _______________________________________________
> >                                                     Gluster-users
> >                                                     mailing list
> >                                                     Gluster-users@xxxxxxxxxxx
> >                                                     <mailto:Gluster-users@xxxxxxxxxxx>
> >                                                     http://lists.gluster.org/mailman/listinfo/gluster-users
> >                                                     <http://lists.gluster.org/mailman/listinfo/gluster-users>
> > 
> > 
> > 
> > 
> >                                                 --
> > Amar Tumballi (amarts)
> > 
> > 
> > 
> >                                             _______________________________________________
> >                                             Gluster-users mailing list
> >                                             Gluster-users@xxxxxxxxxxx
> >                                             <mailto:Gluster-users@xxxxxxxxxxx>
> >                                             http://lists.gluster.org/mailman/listinfo/gluster-users
> >                                             <http://lists.gluster.org/mailman/listinfo/gluster-users>
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> >                             --                             Pranith
> > 
> > 
> > 
> > 
> > 
> >                     --                     Pranith
> > 
> > 
> > 
> > 
> >                 --                 Pranith
> > 
> > 
> > 
> > 
> > 
> > 
> >     --     Pranith
> > 
> > 
> > 
> > 
> > -- 
> > Pranith
> 

Attachment: signature.asc
Description: PGP signature

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux