Track Metric Progress
While many vulnerability scanners are good at discovering vulnerabilities, not all are strong at reporting and tracking them.
When purchasing a vulnerability scanner, spend time assessing whether you can manage vulnerabilities outside of the tool, or the tool supports the creation of new and customized reports that meet your metric criteria.
Incorrect Metric Creation
Dashboards and review presentations that report on and track an excessive number of key performance indicators (KPIs) and key risk indicators (KRIs) can have a negative impact on decision makers’ abilities to sufficiently understand the current state of vulnerability exposure to arrive at a high-confidence conclusion. This phenomenon can be described as metrics creep. Some presentations can go on for 20–30 slides of different graphs explaining where vulnerabilities are collected, how many vulnerabilities are attributed to each application, and the duration of their existence.
A slide might be made and presented with the findings no longer relevant, but the metric will persist, driven by the assumption that leadership revisits the question.
At this point, the metric no longer tells the story. Perhaps it never really did. Questions could surface during review meetings with fleeting relevance only to have a metric persist, which adds noise to the real issue of vulnerability management.
2. Create Achievable Vulnerability Management Goals
Many vulnerability management goals are set in the infancy of the program and are built on hope and inexperience. Maturing a program with old goals can become problematic when unachievable goals persist in the face of new information.
To help your vulnerability management program be successful, goals should be set, used, evaluated, and reset as new information comes to light.
Here are a few considerations when setting vulnerability management goals:
- Achieving a vulnerability count of zero is impossible
- Impacts to the rate of vulnerability discovery
- Impacts to the rate of vulnerability remediation
- Use net change in vulnerabilities as a barometer
Achieving a Zero Vulnerability Count is Impossible
When it comes to setting goals for a vulnerability management program, the first key principle to understand is it’s impossible to get a vulnerability count down to zero—assuming the vulnerability scanner is configured properly and receiving a full picture of the vulnerability exposure within the organization on a regular cadence.
A graph of a total number of vulnerabilities on a Y axis and time on the X axis, assuming the organization actively participates in remediation efforts, would look asymptotic, meaning over time the total vulnerabilities may approach but never reach zero.
The vulnerability management team should see sharp decreases in the beginning of the program due to implementing simple remediations that resolve large numbers of vulnerabilities.
However, as the low-hanging fruit is picked, the line flattens out and the remediations become more involved, requiring more resources to resolve a smaller number of overall vulnerabilities.
Impacts to the Rate of Vulnerability Discovery
New vulnerabilities are discovered within your information system in every cycle. This number stays constant with only a few factors that may have an impact, such as:
- Changes to the IT infrastructure
- New vulnerabilities published
- Changes to scanner configurations
Changes to the IT Infrastructure
As new assets are deployed or as new products are implemented, the number of new vulnerabilities discovered will increase. However, this is usually balanced by the number of assets decommissioned or services removed in favor of software as a service (SaaS) applications.
The vulnerability management program may see a large increase in newly discovered vulnerabilities when large scale system expansion projects are undertaken—such as setting up a new data center.
New Vulnerabilities Published
Other factors may have a noticeable impact on the number of newly discovered vulnerabilities, including the number of vulnerabilities published that cycle.
Year over year, more vulnerabilities are published than previously. A recent report from Redscan shows the number of vulnerabilities published in a single year between 2010 and 2020 grew by four and half times.
However, there also seems to be some variability within the year as some months tend to be heavier with publishing than others. Lately, summer months have become particularly heavy for vulnerability publishing, but not all published vulnerabilities will apply to your system.
Changes to Scanning Configurations
Changes to scanning configurations don’t happen regularly, but they can occur as new systems are brought in scope. These may include new technology that requires tweaks to the scanning profile.
Configurations may also fail, causing credentials to no longer work and reducing the number of discovered vulnerabilities.
Impacts to the Rate of Vulnerability Remediation
Just as new vulnerabilities are discovered each cycle, old vulnerabilities should be remediated from your vulnerability exposure.
To effectively allocate resources, it’s crucial to understand your organization’s rate of vulnerability remediation. This is characterized by how many vulnerabilities fall off the next vulnerability scan report as a result of remediation efforts.
Remediation capabilities are mostly impacted by organizational resource allocations dedicated to these efforts, which include:
- Number of individuals allocated to remediation actions
- Technology debt that must be paid off to implement remediation
- Difficulty implementing complex remediation plans
- Technologies used to perform automatic deployment of patches and remediations
The number of vulnerabilities remediated per cycle tends to behave similarly to the number of total vulnerabilities in the environment.
Since fewer resources are needed to remediate easy vulnerabilities, the first few cycles of the vulnerability program will see large amounts of vulnerabilities remediated. However, after picking lower-hanging fruit, the remaining vulnerabilities often require more resources to remediate.
As the vulnerability management program reaches a more mature setting, the number of remediated vulnerabilities becomes more constant and less impressive.
Forces that might impact this metric include those that steal the attention of resources, such as large IT projects, employee turnover, and events that lead to employment uncertainty, such as the COVID-19 pandemic.
Use Net Change in Vulnerabilities as a Barometer
The net change in vulnerabilities metric is a useful barometer for understanding the capabilities of the vulnerability management program.
So long as that net change in vulnerabilities metric is negative, the backlog of vulnerabilities that carry over cycle to cycle will decrease. Once the metric becomes a positive, the backlog of vulnerabilities will increase, which signals to leadership changes may need to be made to resources or goals.
If you start to notice you’re hitting a net change equal to zero and you’re nowhere near the goals you set for the vulnerability management program, then you need to either reevaluate your goals or adjust your resource allocation.
A net change of zero isn’t necessarily bad, provided you’re within your risk tolerance level. This may be the sign that it’s time to stop focusing on which remediation offers the most value in terms of total vulnerabilities remediated to which remediations provide the best overall security.
3. Remediate Vulnerabilities with the Most Impact
Many programs seek to improve vulnerability numbers by focusing on easy wins or remediations that offer the biggest bang-for-your-buck. These tend to be remediations that require a single action, and which will resolve high numbers of vulnerabilities.
For example, removing an unnecessary and vulnerable product may result in a material reduction in total vulnerabilities.
In 2018, 39 vulnerabilities were published for a leading document viewer. If that application was installed on all workstations throughout an organization that managed approximately 500 workstations, the total vulnerability count would be 19,500.
However, focusing on such easy options may result in missing real risk to the organization. A recent report from Kenna Security shows that while the number of published vulnerabilities increases, the number of exploitable vulnerabilities decreases.
An exploitable vulnerability takes the risk a step further with a proven attack that can be utilized to capitalize on that vulnerability. Many vulnerability scanners will designate a vulnerability as being exploitable.
Percentage of Exploitable Vulnerabilities Published
We can assume from these numbers that focusing on a small percentage of vulnerabilities may result in big gains to the organization’s security posture. This is especially important when considering vulnerability remediation is limited by the resources available to the organization.
Why expend valuable resources on easy vulnerabilities that may never see an attempted exploitation, when those resources can be focused on a small subset of vulnerabilities that produce real risk to the organization?
4. View Your Vulnerability Program as an Iterative Process
It’s easy to approach vulnerability management processes as a linear approach, from discovery to remediation. As soon as a new scan is complete, all previous data is seen as outdated and discarded.
However, there’s an error in this thinking because vulnerabilities that have been through remediation should be validated as resolved. Information from the previous cycle should feed into the next. Vulnerability management processes that don’t iterate struggle to mature and respond to the changing vulnerability landscape.
If a vulnerability simply falls off the next report, ask yourself these questions to help inform and tune a vulnerability management program to enter higher levels of maturity.
- Does it mean the vulnerability has been remediated, or is the device no longer present on the system?
- Could the scan have failed?
- Could the initial discovery have been a false positive?
- What if the vulnerability is still present after the remediation has been applied?
- Was there an issue with remediation, or was the device missed?
We’re Here to Help
For guidance in developing or growing your vulnerability management program, contact your Moss Adams professional. You can also visit our Risk & IT Compliance Consulting Services page for additional resources.
