Benchmarking
Benchmarking is the process of comparing the cost, cycle time, productivity, or quality of a specific process or method to another that is widely considered to be an industry standard or best practice. Essentially, benchmarking provides a snapshot of the performance of your business and helps you understand where you are in relation to a particular standard. The result is often a business case for making changes in order to make improvements.
Also referred to as "best practice benchmarking" or "process benchmarking", it is a process used in management and particularly strategic management, in which organizations evaluate various aspects of their processes in relation to best practice, usually within a peer group defined for the purposes of comparison. This then allows organizations to develop plans on how to make improvements or adopt best practice, usually with the aim of increasing some aspect of performance. Benchmarking may be a one-off event, but is often treated as a continuous process in which organizations continually seek to challenge their practices.
Software economics is an increasingly important topic concerned with improving the value created by investments in the development and use of software. The understanding of the relationships between economics and software development practices is the basis for improving software capability. Software managers have often ignored software economics in the past, because they did not have access to reliable benchmarking data. As a result, many (process) improvement initiatives resulted in satisfying standards/models instead of tangibly improving measured capability. Now that data on thousands of projects is available to the software industry, it is becoming possible to make solid business decisions about software development practices and their results in terms of productivity and quality. Benchmarking is a technique that makes use of external comparisons to better evaluate current performance and identify possible actions for the future. As such, it is an important instrument to drive improvement efforts.
Based on experience, the Software Benchmarking Organization has developed a set of KPIs that represent current best practice in industry. In this set, the following categories are distinguished:
- Project performance
- How predictable is the performance of projects?
- Process efficiency
- How efficient is the development process?
- Product scope
- How large and stable is the scope of the planned effort?
- Product quality
- What is the quality of the resulting outcome/product?
| Category | Typical Key Performance Indicators |
|---|---|
| Project performance |
Schedule (calendar months) Effort (staff months) Staffing rate (manpower buildup) Productivity (size unit per time unit) |
| Process efficiency |
Core activities (essential, value-adding) Support activities (essential, nonvalue-adding) Prevention activities (trainings, reviews, inspections) Appraisal/rework activities (testing, defect removal) |
| Product scope Feature size |
Deferral ratio Product size (KLOC or other physical size unit) Re-use level |
|
Product quality |
Complexity (architecture and/or source code level) Test coverage (unit, integration, system testing) Defect density (released defects per size unit) Cumulative defect removal efficiency |
"Within limited budget and time Secure-AG has audited a major FEI software project and revealed some interesting metrics for us. The audit has given us a nice baseline for comparing ourselves with industry peers and with ourselves in the future in time. We were specifically impressed with the graphs and information that the SolidSource tooling provided"
"Since 2009, we have adopted the best practice KPI set as part of our
continuous improvement program. At this stage, most KPIs have been
implemented. Using the set for project monitoring has helped us to
improve the required transparency in our software projects and supports
us to make informed decisions in a pro-active way.
"
"The best practice KPI set turns out to be an excellent instrument to measure real software capability. It enables our customers to baseline and benchmark their current performance and allows us to derive a solid business case for adopting our ASD technology. The supporting workshops have been very well received by both our consultants and our customers."