Every Solution You Can Imagine – and More
What cybersecurity solution do you need? From Zero Trust to ADR, IAM, risk/privacy, data protection, AppSec and threat, securing digital transformation, to resiliency and remediation, we can build the right program to help solve your challenges.
A Single Partner for Everything You Need
Optiv works with more than 400 world-class security technology partners. By putting you at the center of our unmatched ecosystem of people, products, partners and programs, we accelerate business progress like no other company can.
We Are Optiv
Greatness is every team working toward a common goal. Winning in spite of cyber threats and overcoming challenges in spite of them. It’s building for a future that only you can create or simply coming home in time for dinner.
However you define greatness, Optiv is in your corner. We manage cyber risk so you can secure your full potential.
Secure SDLC Lessons Learned: #4 Metrics
Secure SDLC Lessons Learned: #4 Metrics
In this blog series, I am discussing five secure software development lifecycle (SDLC) “lessons learned” from Optiv’s application security team’s conversations with our clients in 2016. Please read previous posts covering:
As the secure SDLC program matures, vulnerabilities should be caught and remediated earlier in the lifecycle. To know if the program is truly working, organizations must capture metrics. The specific metrics chosen should support and align with the organization’s business objectives and risk management program.
Security metrics related to the SDLC are best captured at security checkpoints. These checkpoints or gates may include in-IDE static analysis, code commit analysis, build-time static-analysis, manual code review, dynamic scanning in QA/integration test, and pre- and post-production penetration testing. Internal and external bug bounty programs may also be included as post-production options.
By capturing which checkpoint or tool was used to discover which security defects, you will be able to track trends across your application portfolio. In theory, as the secure SDLC program matures, more security bugs should be caught early in the software lifecycle. This metric will help verify this in practice.
In addition, proper metrics will help identify toolchain issues, or conversely, justify their expense. For example, if your static analysis tools fail to capture security defects that surface during penetration testing, then there may be a problem in code coverage or a scanner ruleset.
Beyond tracking the discovery point, there are two main classes of key performance indicators (KPIs): efficiency and risk (though some indicators may fall in the gray area between). Each KPI may be filtered by vulnerability type, development team or business area, phase of SDLC discovered, etc., and is often reported per time period.
Efficiency indicators may already be available in your bug tracking system. These may include:
Other indicators can be designed to track efficiency in security operations, like time to assign an AppSec resource for internal assurance services, time to complete and so on.
Risk indicators describe areas specific to the confidentiality, integrity and availability of applications.
These metrics are usually presented on some form of dashboard, usually with capabilities for drill down and trend reporting. Remember though that the technologies used to implement KPIs are much less important than the information they’re intended to convey. Avoid developing KPIs and visualizations that bring no value to your secure SDLC program.
Over time, there should be a reduction in post-release vulnerabilities, which again are the most costly to remediate and pose the most risk to the organization. Expect a downward trend in bugs caught later in the SDLC as the program matures. Additionally, vulnerabilities discovered in-cycle should be remediated more quickly. When unexpected indicator values are observed, root cause analysis is often required to address the issue. For example, if severe vulnerabilities continue to be found by an external bug bounty program, then clearly something upstream isn’t working.
Properly designed metrics provide a great way to measure the effectiveness of the secure SDLC program over time, and also provide useful feedback when aspects of the program are tweaked. Capturing key indicators affords organizations the ability to experiment with new vendor tools and alternate approaches to testing, and to determine the impact of different types and sources for developer training.
In short, metrics can take much of the guesswork out of knowing how well your secure SDLC program is working.
In the next post I will cover secure SDLC lesson #5, personnel.
Let us know what you need, and we will have an Optiv professional contact you shortly.