Re: GitLab AMA Topic: Message Bus

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, 2020-11-23 at 19:29 +0100, Clement Verna wrote:
> 
> 
> On Wed, 18 Nov 2020 at 18:30, Fabio Valentini <decathorpe@xxxxxxxxx>
> wrote:
> > On Mon, Nov 9, 2020 at 3:05 PM Aoife Moloney <amoloney@xxxxxxxxxx>
> > wrote:
> > > 
> > > Hi everyone,
> > > 
> > > I hope you enjoyed the F33 release party this weekend! Getting
> > back to
> > > the GitLab topic mail threads, this weeks topic from the GitLab
> > AMA
> > > session on September 10th is on Message Bus. As always, here are
> > some
> > > links to the resources I have been pulling content from as well:
> > > * Questions and Answers hackmd link
> > https://hackmd.io/RW8HahOeR7OJPON1dwuo3w
> > > * Chat log from session
> > > 
> > https://meetbot.fedoraproject.org/fedora-meeting-1/2020-09-10/ama_session_with_gitlab.2020-09-10-13.31.log.html
> > > * AMA Blog post
> > > 
> > https://communityblog.fedoraproject.org/gitlab-ama-follow-up/#more-9346
> > > * Here is this email in hackmd if you wish to view it there:
> > > https://hackmd.io/tfOqCXNEQtqsGNLAEfZ2zg?view
> > 
> > Sorry for taking so long to respond, the past weeks have been quite
> > busy.
> > 
> 
> Sorry for taking even longer to answer, you know the feeling :P 
>  
> > 
> > (snip)
> > 
> > > ## Topic: Message Bus
> > > - Question: Fedora uses a message bus to integrate different
> > parts of
> > > its infrastructure. How should we onboard GitLab into this
> > message
> > > bus?
> > >      - Answer: Currently we would need to have a service that
> > proxies
> > > GitLab’s events to fedora-messaging something similar to
> > > github2fedmsg.
> > > There were some concerns raised about the order of events sent by
> > > GitLab’s webhooks, these will need to be looked after during a
> > Proof
> > > of Concept stage.
> > 
> > Do we know if such a proxy would even be theoretically possible for
> > GitLab?
> > IIRC, some doubts were raised during the AMA that getting a
> > chronologically consistent stream of events out of GitLab would be
> > ∈
> > [hard, impossible[.
> > What would that mean for fedora? Do services relying on
> > fedora-messaging events related to dist-git need them to be
> > consistent
> > / chronological?
> > What would be the effect of those services not having a reliable
> > stream of events from dist-git?
> > 
> 
> I personally don't have a good answer to these questions, and I don't
> think we will be able to 
> have without actually doing a Proof of Concept and see how that would
> work and scale.
> 
> Regarding the order or messages, I believe that anything related to
> CI testing might need to 
> have the chronological order of messages consistent. 
> I am not sure if there are any other use cases ?
> 

If any service relies on chronological ordering for messaging, I've got
bad news on that front. There's plenty of scenarios where messages can
arrive out-of-order now, regardless of whether they were queued up
sequentially (which, in a distributed system, with multiple nodes
accepting messages for publication...). There's also no guarantees
about messages only being delivered once. Never mind that GitLab's
webhooks may well fire multiple times if a response is not received in
a timely manner.

I would highly recommend not creating message consumers that rely on
any particular message ordering because they're not going to work
properly, GitLab or not.

- Jeremy

_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux