Metricating (Cyber) Security
What's your yardstick?
Published 15:23, 13 May 11
A forensics friend - Jon Care - pinged me on Instant Messenger last night:
Him: Do you remember your "Corporate Security Index", inspired by Dilbert?
Me: Rings a faint bell. Did I write it?
Him: You wrote "8.6" on a bit of paper and stuck it to the wall of your cubicle and told everyone it was a measure of our security.
Me: Ah, yes. [It stayed there for a couple of years. That was the point.]
Him: Well ... Security Experts Launch Cybersecurity Index Resource
The Index of Cyber Security is a sentiment-based measure of the risk to the corporate, industrial, and governmental information infrastructure from a spectrum of cybersecurity threats. It is sentiment-based in recognition of the rapid change in cybersecurity threats and postures, the state of cybersecurity metrics as a practical art, and the degree of uncertainty in any risk-centered field. In short, the Index of Cyber Security aggregates the views of information security industry professionals as expressed through a monthly survey. Its form is an index for reasons that will become apparent.
As you may have inferred from this conversation I am famously suspicious of security metrics - not least because I believe it impossible to measure the number of attacks you're not suffering thereby to compute a decent cost-benefit for "security", and further your figures will always be highly dependent upon your threat model, meaning they'd be unlikely to be directly comparable to anyone else's, at best contributing to a very fuzzy mean.
So I look at stuff like the ICS website - curiously the domainname is the reverse of the acronym - and the only thing that prevents me denouncing it as a white elephant is the name of one of the participants - Dan Geer, another friend whose awesome reputation in the IT Security universe is on par with that of both Yoda and Obi-Wan. Both. At the same time.
There's more on the index over at RWW - but I'm still deeply uncomfortable with anyone generating a "security" number and watching it go up and down, not least because I've tried doing this inside a company and it just made matters worse because people watched the number and not the issues.
Overall, security professionals felt that cyber security in the aggregate has worsened, including that of online transactions they conduct as part of their personal lives.
... I worry.
But I'm half-hoping that Dan is doing this in order to put a stop to cyberpanic, to bring rationality back to those who would otherwise throw state funds at cybersecurity; if an eminent mind publishes a number which only goes up and down by small percentages, maybe we can stop the exponential growth of cyberbudgets and instead bring them in line with inflation, eventually to pop them?