Statistical Debugging for Real-World Performance Problems
Design and implementation defects that lead to inefficient computation widely exist in software. These defects are difficult to avoid and discover. They lead to severe performance degradation and energy waste during production runs, and are becoming increasingly critical with the meager increase of single-core hardware performance and the increasing concerns about energy constraints. Effective tools that diagnose performance problems and point out the inefficiency root cause is sorely needed.
The state of the art of performance diagnosis is preliminary. Profiling can tell where computation resources are spent, but not where and why the resources are wasted. Performance-bug detectors can identify specific type of inefficient computation, but are not suited for diagnosing general performance problems. Effective failure diagnosis techniques, such as statistical debugging, have been proposed for functional bugs. However, whether they work for performance problems is still an open question.
In this paper, we first conduct an empirical study to understand how performance problems are observed and reported by real-world users. Our study shows that statistical debugging is a natural fit for diagnosing performance problems, which are often observed through comparison-based approaches and reported together with both good and bad inputs. We then thoroughly investigate different design points in statistical debugging, including three different predicates and two different types of statistical models, to understand which design point works the best for performance diagnosis. Finally, we study how some unique nature of performance bugs allows sampling techniques to lower the overhead of run-time performance diagnosis without extending the diagnosis latency.
Thu 23 OctDisplayed time zone: Tijuana, Baja California change
15:30 - 17:00
DebuggingOOPSLA at Salon E
Chair(s): Caitlin Sadowski Google
|CheckCell: Data Debugging for Spreadsheets|
Dan Barowy University of Massachusetts, Amherst, Dimitar Gochev University of Massachusetts, Amherst, Emery D. Berger University of Massachusetts, AmherstLink to publication
|Finding Minimum Type Error Sources|
Zvonimir Pavlinovic New York University, Tim King New York University, Thomas Wies New York UniversityLink to publication File Attached
|Flint: Fixing Linearizability Violations|
Peng Liu Purdue University, Omer Tripp IBM Thomas J. Watson Research Center, Xiangyu Zhang Purdue UniversityLink to publication
|Statistical Debugging for Real-World Performance Problems|
Linhai Song University of Wisconsin–Madison, Shan Lu University of ChicagoLink to publication