Hi Dave!
We have started a similar project in ABRT some year ago. The original
purpose was to automatically determine some crash characteristics
(security impact, unchecked user input etc.). The overall idea is
rebuilding the given package with some compiler plugin and go through
the AST based on coredump's stack.
We have built an infrastructure working on top of ABRT Server[1]. This
is because of its backend synchronizing with several Fedora services
(koji, pkgdb, bugzilla) storing all the data in one database and thus
ability to make fast cross-service queries. The "storage" (that's how we
call it) is more complex - it uses DB for relational data and saves
large files (like RPMs) on the FS.
The storage contains (almost) all packages built in koji. We were trying
to use mock & yum for a long time, but we were not able to make it work
correctly with such a complex repo. That's why we have created our own
dependency solver based on libsolv (using the DB as metadata source) and
use rpm directly to install the resulting set. This resulted into a fast
chroot installer able to interrupt build process in many phases.
Some guys working on static analysis at Masaryk University, Brno, CZ
showed interest in the project. They are working on various clang
plugins for static analysis, but have no real input data. We agreed to
implement a service rebuilding Fedora packages with LLVM/Clang and
providing the output (LLVM bitcode files). They will provide the
analysis tools.
This showed up as a non-trivial problem, because a simple CC=clang is
not enough :). The thing is however implemented and I've personally
rebuilt Fedora a few times, leaving me with some 60% of packages
successfully processed and a few ideas for improvement. The service has
not yet been deployed in Fedora infrastructure, because we 1) don't
consider it ready 2) lack hardware capacity.
This is more or less the current state. We are open to any discussion,
new ideas, use cases or patches :).
Sources: http://git.fedorahosted.org/cgit/faf.git/
RFE: https://fedorahosted.org/abrt/newticket?component=faf
Michal & ABRT
[1] http://abrt.fedoraproject.org
On 2012-12-11 22:52, David Malcolm wrote:
A while back I ran my static checker on all of the Python extension
modules in Fedora 17:
http://fedoraproject.org/wiki/Features/StaticAnalysisOfPythonRefcounts
I wrote various scripts to build the packages in a mock environment that
injects my checker into gcc, then wrote various scripts to triage the
results. I then filed bugs by hand for the most important results,
writing some more scripts along the way to make the process easier.
This led to some valuable bug fixes, but the mechanism for running the
analysis was very ad hoc and doesn't scale.
In particular, we don't yet have an automated way of rerunning the
tests, whilst using the old results as a baseline. For example it would
be most useful if only new problems could be reported, and if the system
(whatever it is) remembered when a report has been marked as a true bug
or as a false positive. Similarly, there's no automated way of saying
"this particular test is bogus; ignore it for now".
I'm wondering if there's a Free Software system for doing this kind of
thing, and if not, I'm thinking of building it.
What I have in mind is a web app backed by a database (perhaps
"checker.fedoraproject.org" ?)
We'd be able to run all of the code in Fedora through static analysis
tools, and slurp the results into the database: primarily my
"cpychecker" work, but we could also run the clang analyzer etc. I've
also been working on another as-yet-unreleased static analysis tool for
which I'd want a db for the results. What I have working is a way to
inject an analysis payload into gcc within a mock build, which dumps
JSON report files into the chroot without disturbing the "real" build.
The idea is then to gather up the JSON files and insert the report data
into the db, tagging it with version information.
There are two dimensions to the version information:
(A) the version of the software under analysis
(name-version-release.arch)
(B) the version of the tool doing the analysis
We could use (B) within the system to handle the release cycle of a
static analysis tool. Initially, any such analysis tools would be
regarded as "experimental", and package maintainers could happily ignore
the results of such a tool. The maintainer of an analysis tool could
work on bug fixes and heuristics to get the signal:noise ratio of the
tool up to an acceptable level, and then the status of the analysis tool
could be upgraded to an "alpha" level or beyond.
Functional Requirements:
* a collection of "reports" (not bugs):
* interprocedural control flow, potentially across multiple source
files (potentially with annotations, such as value of variables,
call stack?)
* syntax highlighting
* capturing of all relevant source (potentially with headers as
well?)
* visualization of control flow so that you can see the path
through the code that leads to the error
* support for my cpychecker analysis
* support for an as-yet-unreleased interprocedural static analysis
tool I've been working on
* support for reports from the clang static analyzer
* ability to mark a report as:
* a true bug (and a way to act on it, e.g. escalate to bugzilla or
to the relevant upstream tracker)
* a false positive (and a way for the analysis maintainer to act
on it)
* other bug associations with a report? (e.g. if the wording from
the tool's message could be improved)
* ability to have a "conversation" about a report within the UI as
a series of comments (similar to bugzilla).
* automated report matching between successive runs, so that the
markings can be inherited
* scriptable triage, so that we can write scripts that mark all
reports matching a certain pattern e.g. as being bogus, as being
security sensitive, etc
* potentially: debug data (from the analysis tool) associated with a
report, so that the maintainers of the tool can analyze a false
positive
* ability to store crash results where some code broke a static
analysis tool, so that the tool can be fixed
* association between reports and builds
* association between builds and source packages
* association between packages and people, so that you can see what
reports are associated with you (perhaps via the pkgdb?)
* prioritization of reports to be generated by the tool
* association between reports and tools (and tool versions)
* "quality marking" of tool versions, so that we can ignore "alpha"
versions of tools and handle phasing in of a new static analysis
tool without spamming everyone
* ability to view the signal:noise ratio of a version of a tool
Nonfunctional requirements:
* Free Software
* sanely deployable within Fedora infrastructure
* sane code, since we're likely to want to extend it (fwiw I'd be most
comfortable with a Python implementation).
* able to scale to running all of Fedora through multiple tools
repeatedly
* many simultaneous users
* will want an authentication system so that we can associate comments
with users. Eventually we may want a way of embargoing
security-sensitive bugs found by the tool so that they're only
visible by a trusted cabal.
* authentication system to support FAS, but not require it, in case
other people want to deploy such a tool. Maybe OpenID?
Implementation ideas:
* as well as a relational database for the usual things, perhaps a
lookaside of source files stored gzipped, with content-addressed storage
e.g. "0fcb0d45a6353e150e26f1fa54d11d7be86726b6" stored gzipped as:
objects/0f/cb0d45a6353e150e26f1fa54d11d7be86726b6
(yes, this looks a lot like git)
Thoughts? Does such a thing already exist?
It might be fun to hack on this at the next FUDcon.
Dave
--
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/devel
