Combining compile-time and run-time instrumentation for testing tools

Dynamic program analysis and testing tools typically require inserting extra instrumentation code into the program to test. The inserted instrumentation then gathers data about the program execution and hands it off to the analysis algorithm. Various analysis algorithms can be used to perform CPU profiling, processor cache simulation, memory error detection, data race detection, etc. Usually the instrumentation is done either at run time or atcompile time – called dynamic instrumentation and compiler instrumentation, respectively. However, each of these methods has to make a compromise between performance and versatil-ity when used in industry software development. This paper presents a combined approach to instrumentationwhich takes the best of the two worlds – the low run-time overhead and unique features of compile-time instrumentation and the flexibility of dynamic instrumentation. Wepresent modifications of two testing tools that benefit from thisapproach: AddressSanitizer and MemorySanitizer. We propose benchmarks to compare different instrumentation frameworks in conditions specific to hybrid instrumenta-tion. We discuss the changes we made to one of the state-of-the-art instrumentation frameworks to significantly improve the performance of hybrid tools.

Source: http://swsys.ru/index.php?page=article&id=3593&lang=en

Does Bug Prediction Support Human Developers? Findings From a Google Case Study

While many bug prediction algorithms have been
developed by academia, they’re often only tested and verified
in the lab using automated means. We do not have a strong
idea about whether such algorithms are useful to guide human
developers. We deployed a bug prediction algorithm across
Google, and found no identifiable change in developer behavior.
Using our experience, we provide several characteristics that bug
prediction algorithms need to meet in order to be accepted by
human developers and truly change how developers evaluate their
code.

Source: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41145.pdf

Why Don’t Software Developers Use Static Analysis Tools to Find Bugs?

Using static analysis tools for automating code
inspections can be beneficial for software engineers. Such tools
can make finding bugs, or software defects, faster and cheaper
than manual inspections. Despite the benefits of using static
analysis tools to find bugs, research suggests that these tools
are underused. In this paper, we investigate why developers
are not widely using static analysis tools and how current tools
could potentially be improved. We conducted interviews with 20
developers and found that although all of our participants felt
that use is beneficial, false positives and the way in which the
warnings are presented, among other things, are barriers to use.
We discuss several implications of these results, such as the need
for an interactive mechanism to help developers fix defects.

Source: http://people.engr.ncsu.edu/ermurph3/papers/icse13b.pdf

AddressSanitizer: A Fast Address Sanity Checker

Memory access bugs, including buffer overflows and
uses of freed heap memory, remain a serious problem for
programming languages like C and C++. Many memory
error detectors exist, but most of them are either slow or
detect a limited set of bugs, or both.
This paper presents AddressSanitizer, a new memory
error detector. Our tool finds out-of-bounds accesses to
heap, stack, and global objects, as well as use-after-free
bugs. It employs a specialized memory allocator and
code instrumentation that is simple enough to be implemented
in any compiler, binary translation system, or
even in hardware.
AddressSanitizer achieves efficiency without sacrificing
comprehensiveness. Its average slowdown is just
73% yet it accurately detects bugs at the point of occurrence.
It has found over 300 previously unknown bugs in
the Chromium browser and many bugs in other software.

Source: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37752.pdf

Searching for Build Debt: Experiences Managing Technical Debt at Google

With a large and rapidly changing codebase,
Google software engineers are constantly paying interest on
various forms of technical debt. Google engineers also make
efforts to pay down that debt, whether through special Fixit
days, or via dedicated teams, variously known as janitors,
cultivators, or demolition experts. We describe several related
efforts to measure and pay down technical debt found in
Google’s BUILD files and associated dead code. We address
debt found in dependency specifications, unbuildable targets,
and unnecessary command line flags. These efforts often expose
other forms of technical debt that must first be managed.

Source: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37755.pdf