Re: [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On 22 Nov 2019, at 09:41, Markus Armbruster <armbru@xxxxxxxxxx> wrote:

Reviving this old thread, because I'd like to connect it to more recent
discussions.

Christophe de Dinechin <dinechin@xxxxxxxxxx> writes:

Markus Armbruster writes:

Peter Krempa <pkrempa@xxxxxxxxxx> writes:

[...]
From my experience users report non-fatal messages mostly only if it is
spamming the system log. One of instances are very unlikely to be
noticed.

In my experience it's better to notify us in libvirt of such change and
we will try our best to fix it.

How to best alert the layers above QEMU was one of the topic of the KVM
Forum 2018 BoF on deprecating stuff.  Minutes:

   Message-ID: <87mur0ls8o.fsf@xxxxxxxxxxxxxxxxxx>
   https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg05828.html

Relevant part:

* We need to communicate "you're using something that is deprecated".
 How?  Right now, we print a deprecation message.  Okay when humans use
 QEMU directly in a shell.  However, when QEMU sits at the bottom of a
 software stack, the message will likely end up in a log file that is
 effectively write-only.

 - The one way to get people read log files is crashing their
   application.  A command line option --future could make QEMU crash
   right after printing a deprecation message.  This could help with
   finding use of deprecated features in a testing environment.

 - A less destructive way to grab people's attention is to make things
   run really, really slow: have QEMU go to sleep for a while after
   printing a deprecation message.

 - We can also pass the buck to the next layer up: emit a QMP event.

   Sadly, by the time the next layer connects to QMP, plenty of stuff
   already happened.  We'd have to buffer deprecation events somehow.

   What would libvirt do with such an event?  Log it, taint the domain,
   emit a (libvirt) event to pass it on to the next layer up.

 - A completely different idea is to have a configuratin linter.  To
   support doing this at the libvirt level, QEMU could expose "is
   deprecated" in interface introspection.  Feels feasible for QMP,
   where we already have sufficiently expressive introspection.  For
   CLI, we'd first have to provide that (but we want that anyway).

 - We might also want to dispay deprecation messages in QEMU's GUI
   somehow, or on serial consoles.

Sorry for catching up late, this mail thread happened during my PTO.

I remember bringing up at the time [1] that the correct solution needs
to take into account usage models that vary from

- a workstation case, where displaying an error box is easy and
 convenient,

- to local headless VMs where system-level notification would do the job
 better, allowing us to leverage things like system-wide email notifications

- to large-scale collections of VMs managed by some layered product,
 where the correct reporting would be through something like Insights,
 i.e. you don't scan individual logs, you want something like "913 VMs
 are using deprecated X"

To me, that implies that we need to have a clear division of roles, with
a standard way to

a) produce the errors,
b) propagate them,
c) consume them (at least up to libvirt)

Notice that this work has already been done for "real" errors,
i.e. there is a real QAPI notion of "errors". AFAICT, warn_report does
not connect to it, though, it goes through error_vprintf which is really
just basic logging.

So would it make sense to:

1. Add a deprecation_report() alongside warn_report()?

"Grepability" alone would make that worthwhile, I think.

2. Connect warn_report() and all the error_vprintf output to QAPI,
  e.g. using John's suggestion of adding the messages using some
  "warning" or "deprecated" tag?

This is the difficult part.  See my "Exposing feature deprecation to
machine clients (was: [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit
filters)" in this thread.  Quote:

   Propagating errors is painful.  It has caused massive churn all over the
   place.

   I don't think we can hitch deprecation info to the existing error
   propagation, since we need to take the success path back to the QMP
   core, not an error path.

   Propagating a second object for warnings... thanks, but no thanks.

   The QMP core could provide a function for recording deprecation info for
   the currently executing QMP command.  This is how we used to record
   errors in QMP commands, until Anthony rammed through what we have now.
   The commit messages (e.g. d5ec4f27c38) provide no justification.  I
   dimly recall adamant (oral?) claims that recording errors in the Monitor
   object cannot work for us.

   I smell a swamp.

This looks swampy the way you describe it, but this make me think that
the recorder library may provide a somewhat elegant solution to this.

First, my presupposition here is that part of the issues you are describing
is because there is no “one size fits all” reporting of errors, warnings, etc.
Essentially, it’s a policy that is defined outside of libvirt or qemu, and not
all consumers will want the same policy.

The way the recorder library addresses that is to have as many recorders
as you need, and allowing the behavior for each recorder to be configured
individually at run-time, e.g. through env variables.

If you couple that with a naming convention for the recorders, you can
set the policy externally in a rather flexible way. For example, I will typically
have recorders that end with _error or _warning or _info or _verbose.
We could have a _deprecated convention just the same.

With a simple syntax, you can state that you want tracing on
all warnings and errors by activating a regexp like “.*_(warning|error)”,
or disable anything related to deprecation with “.*_deprecated.*=0”.
It wouldn’t be much of a stretch to have similar policies for “create a
qapi error object” or even “abort the process”.

This approach means that we would get to define the default policy
for the various deprecation cases, but that the user would ultimately
be able to override, maybe even individually, e.g.:

// I don’t care about deprecation at all
.*_deprecated=0

// Log all deprecation messages, do nothing else with it
.*_deprecated=<log>

// Log most deprecation, but ignore “blink_tag” deprecation
// messages and abort on “serial_uart” deprecation message
.*_deprecated=<log>
blink_tag_deprecated=0
serial_uart_deprecated=<abort>


   Can we avoid plumbing deprecation info from command code to QMP core?
   Only if the QMP core itself can recognize deprecated interfaces.  I
   believe it can for the cases we can expose in introspecion.  Let me
   explain.

The recorder above (as it exists today) only deals with logging and reporting
messages. It provides no help whatsoever for introspection, which I assume
you would use to warn “ahead of time” that something is deprecated, right?


   Kevin recently added "features" to the QAPI schema language.  The
   implementation is incomplete, but that's detail.  The idea is to tack a
   "deprecated" feature to deprecated stuff in the QAPI schema.

   Commands and arguments need to support features for that.
   Implementation should be relatively straightforward.

   Deprecating an argument's optionalness may require a
   "optional-deprecated" feature.  I've seen more elegant designs, but I've
   also seen plenty of uglier ones.

   Note that features are tied to schema syntax.  To express semantically
   conditional deprecation like "if you specify argument FOO, then not
   specifying argument BAR is deprecated", we'd have to add a language for
   these conditions.  Uh, not now, maybe never.

For these cases, we can still fallback to C code, no? As long as the
“report_deprecated()” being used is somewhat consistent between
the two, the fact that there is a “short, common case” for optiions
and a “complicated, less common case” would be invisible to users.


   The primary use of having deprecation defined in the QAPI schema is
   introspection.  The BoF minutes mention this, too.

   A secondary use could be detecting use of deprecated features right in
   the QMP core.  No need for ad hoc code in commands, no need for plumbing
   information from there to the QMP core.

   I'd like to pursue this idea, then see how well it suits our deprecation
   needs.

I've since explored this idea in

   [RFC PATCH 00/19] Configurable policy for handling deprecated interfaces
   Message-Id: <20191024123458.13505-1-armbru@xxxxxxxxxx>
   https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg06534.html    

I had not paid attention to this one. I’ll take a look and get back to you.




3. Teach libvirt how to consume that new tag and pass it along?

Pertinent:
Message-ID: <20191024140146.GC8381@xxxxxxxxxx>
https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg06604.html

Quoting Dan's conclusion:

   In summary, it is probably reasonable to include this info in the QMP
   command reply, but don't expect much to be done with it beyond possibly
   writing it to a log file.

[1] https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg06131.html


Thanks
Christophe



--
Cheers,
Christophe de Dinechin (IRC c3d)

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list

[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux