On co-discovery of vulnerabilities

01 Apr 2017 | VEP, CTF, python

Summary: In order to help start informing Vulnerability Equities discussions, we’ve put together metrics for co-discovery of vulnerabilities in purpose-built software.

The United States government established a Vulnerability Equities Process (VEP) to determine if and when information regarding vulnerabilities should be disclosed. The intent of the VEP is to evaluate the impact of privately held vulnerabilities in the context of the public good, law enforcement, or intelligence gathering.

A key element underpinning discussions of any VEP relates to how likely any given vulnerability is going to be co-discovered by other entities. If an adversary is likely to co-discover the vulnerability, then the risk of withholding information is increased.

Without underlying data to understand the potential for co-discovery, those in charge of the VEP must use off-the-cuff guesses.

As an attempt to understand the underlying problem, we’ve gathered information from eight (8) different hacking competitions, and investigated how frequent teams would co-discover a purpose-built vulnerability. Our research focuses on those competitions with publicly disclosed discovery details where the competition does not include sharing details of a vulnerability until after the conclusion of the competition.

We excluded challenges that were trivia, forensics, or other similar areas unrelated to the VEP discussion within the competitions.

While this data is not perfect correlation, the underlying components are the very similar to real world constraints.

The primary problem with analysis of real software is that we have no way of knowing if multiple organizations are investigating the same target software. We don’t know how much time is being invested. We don’t know relative skill differentials.

Given co-discovery, from a equities prospective, is frequently focused on comparing (presumably) extremely well funded organizations, this data has quite a bit of comparable insights.

We have made the tools to gather this information publicly available.