On Mon, Sep 13, 2021 at 12:49:21PM -0600, Tim Flink wrote:
> It turns out that the configuration for rhbz was changed without an announcement and now the max number of bugs returned for any query is 20. Some of the queries that we use in blockerbugs return more than 20 bugs so I have a hotfix to deal with the new pagination requirements. I've attached the patch to this email.
>
> I'd like to apply this patch to production. Unfortunately, we're in the middle of a non-trivial rewrite that's currently on stg so testing the patch isn't really an option and that code isn't ready for production use right now.
>
> Thanks,
>
> Tim
> diff --git a/blockerbugs/util/bz_interface.py b/blockerbugs/util/bz_interface.py
> index 471140f..a9f90d7 100644
> --- a/blockerbugs/util/bz_interface.py
> +++ b/blockerbugs/util/bz_interface.py
> @@ -29,11 +29,14 @@ from xmlrpc.client import Fault
>
> from blockerbugs import app
>
> +# rhbz has been updated to have a max of 20 results returned
> +BUGZILLA_QUERY_LIMIT = 20
If I understood correctly, you can request more than 20 results, but if you
don't specify how many you want, you'll only get 20.
I believe if you set it to 0 you may get everything, but this needs to be
confirmed empirically.
Anyway, I would check first if you can't retrieve 100 or 200 results in one go
instead of doing iterations of 20 ;-)
> base_query = {'o1': 'anywords',
> 'f1': 'blocked',
> 'query_format': 'advanced',
> - 'extra_fields': ['flags']}
> -
> + 'extra_fields': ['flags'],
> + 'limit': BUGZILLA_QUERY_LIMIT}
>
> class BZInterfaceError(Exception):
> """A custom wrapper for XMLRPC errors from Bugzilla"""
> @@ -77,7 +80,8 @@ class BlockerBugs():
> # https://bugzilla.stage.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED
> # &bug_status=POST&bug_status=MODIFIED&classification=Fedora&component=anaconda&f1=component
> # &o1=changedafter&product=Fedora&query_format=advanced&v1=2013-03-21%2012%3A25&version=19
> - def get_bz_query(self, tracker: int, last_update: datetime.datetime = None) -> dict[str, Any]:
> + def get_bz_query(self, tracker: int, last_update: datetime.datetime = None, offset: int = 0
> + ) -> dict[str, Any]:
If you can set the limit to 0 to retrieve everything, you may want to default to
None rather than 0.
> """Build a Bugzilla query to retrieve all necessary info about all bugs which block the
> `tracker` bug.
>
> @@ -129,6 +133,9 @@ class BlockerBugs():
> 'f10': 'CP'
> })
>
> + if offset > 0:
And if you default to None, this will need to be tweaked.
> + query.update({'offset': offset})
> +
> return query
>
> def query_tracker(self, tracker: int, last_update: Optional[datetime.datetime] = None
> @@ -139,8 +146,21 @@ class BlockerBugs():
> :param last_update: If provided, the query is modified to ask only about bugs which have
> recent modifications; otherwise asks about all bugs.
> """
> - query = self.get_bz_query(tracker, last_update)
> - buglist = self.bz.query(query)
> +
> + buglist = []
> + last_query_len = BUGZILLA_QUERY_LIMIT
> +
> +
> + # this is a hotfix hack to work around the sudden config change in rhbz where the max
> + # number of bugs returned for a query is 20
> + # it seems to be working for now but may need more work going forward
> + while last_query_len == BUGZILLA_QUERY_LIMIT:
> +
> + new_query = self.get_bz_query(tracker, last_update, offset=len(buglist))
> + new_buglist = self.bz.query(new_query)
> + buglist.extend(new_buglist)
> + last_query_len = len(new_buglist)
> +
Looks alright to me otherwise.
Pierre
_______________________________________________
infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
