May 2, 2022 11:37:59 AM | xIoT Just in case you didn’t have enough to worry about… (Part 1/3)

I’ll be honest: It’s been quite a while since I seriously worried about anything in cybersecurity other than software vulnerabilities. Almost every serious cyberattack you can name in the last say five years, including Not Petya, SolarWinds, Kaseya, and literally every ransomware attack, was either based on or enabled by at least one software vulnerability.

Of course, when the average cybersecurity person thinks about software vulnerabilities, they probably think of badly-trained (or simply incompetent) software coders who make mistakes and leave vulnerabilities in the software they’re writing – or else malicious actors who plant backdoors (which are just deliberately-inserted vulnerabilities) in the code during development, so they can later come back when the software is deployed and exploit those backdoors. Certainly, both types of people exist.

This was certainly what I thought of when I thought of software vulnerabilities, until I fell in with a bunch of people – loosely, the “SBOM community” – who have been on the front lines of fighting software vulnerabilities for years. These people know that no amount of training, pleading, threats, firings, etc. will ever cause developers to stop creating new vulnerabilities. As the Good Book says, “Software vulnerabilities you have always with you.”

Given that there will always be new vulnerabilities, what’s most important are identifying vulnerabilities, patching them as quickly as possible, and making users aware of both the vulnerabilities and the patches. But there are some serious obstacles standing in the way of accomplishing these goals – and for once, the individual developers aren’t to blame for them. In fact, the most serious obstacles may be due to problems with the various institutions that we’ve created, in a well-intentioned effort to mitigate the risks posed by software vulnerabilities.

One of these institutions is the National Vulnerability Database, which was created in 2005 by NIST and continues to be run by them (MITRE Corp. created the CVE List in 1999, and still maintains it. That list is continually incorporated into the NVD, but isn’t in fact part of it). A foundation of the NVD is the CPE (Common Platform Enumeration) name, which is meant to be a unique machine-readable identifier of a particular version of a particular software product.

CPE is a crucial component (so to speak) of the software vulnerability management ecosystem. This is because the only way a software (or intelligent device) user can learn what CVEs (and by the way, CVE used to stand for something, but now it’s NBI – nothing but initials) apply to a product they use is to search the NVD, using the CPE name for the product (although they may first have to search for the CPE name itself, using a supplier or product name).

In theory, the supplier of your software product should provide, in their SBOMs, a CPE name for every component of the product. Armed with those CPE names, you (or more specifically, a tool that you operate or that a third party operates on your behalf) can search for any CVEs that apply to each component. You repeat this for each component, and you’ll end up with a list of all the CVEs that apply to at least one component in the product. What could be easier?

Well, it turns out that there are a lot of problems with this system, which prevent the user from getting accurate answers. I addressed a few of those problems in the only post I’ve ever written on this issue; these are collectively known as “the naming problem” in SBOM circles. In those circles, the naming problem is thought to be practically insoluble, although it can be managed – just like an incurable disease can often be managed, so that it won’t automatically be fatal.

Until last Friday, I agreed with the idea that the naming problem could be managed, so that the patient wouldn’t die of it. But on Friday I saw a great presentation that Tom Pace of NetRise made to an informal meeting I was part of. While there are a lot of problems with the current CPE/CVE system for identifying software vulnerabilities, Tom pointed out one that perhaps outweighs all the others – and makes me and a few others think that the naming problem needs to be solved in the near future, not just mitigated.

Before I explain the problem, you need to know that NetRise is a company dedicated to firmware security risk management. But just like software (certain theologians argue that firmware is software; however, I don’t get involved in theological arguments), probably the biggest source of risk in firmware is the components included in a firmware product. Also like software, in order to learn about vulnerabilities in those components, the user needs to have an SBOM listing them, with hopefully a CPE name for every component; then the user can look in the NVD to find the complete set of vulnerabilities (CVEs) found in those components.

Now, I already knew that there were problems with looking up CPEs. I mentioned a few of these in the post from 2020 that I referenced above. And since I anticipate I’ll do more posts on this problem as the effort to solve it moves forward, I expect to discuss others. However, the problem Tom discussed was one I hadn’t heard of before, although it follows so clearly from the facts (all of which I already knew), that I wonder why I didn’t think of this.

1. The root of the problem is an unfortunate (to say the least) quirk of the NVD: If you search for CVEs by entering a CPE name, or even a product or supplier name, and there are no CVEs associated with whatever text string you entered, you will receive the message “There are 0 matching records.”
2. Of course, this fact alone doesn’t constitute a problem. If there aren’t any CVEs, that’s what you would want to see. But what do you see if you had mis-entered just one of the characters in a CPE name (for example, you had entered one letter wrong in the CPE name “cpe:2.3:a:altiris:report_pack_for_inventory_solution_for_windows:6.2.1047:*:*:*:*:*:*:*”)? You see the same message.
3. Does the fact that the message is the same mean there aren’t any CVEs associated with the CPE name you intended to enter? Not at all. There could be a ton of them, and if you hadn’t made the mistake, you would have seen them. But – humans being generally an optimistic lot – my guess is most people, upon seeing that message, will breathe a sigh of relief and say, “Thank goodness, my product has no vulnerabilities!” So it’s likely a lot of people will be misled into thinking the software product they’re inquiring about doesn’t have any vulnerabilities, when the problem was that they fat-fingered one of the letters.
4. Of course, that would be bad, but what Tom pointed to was much worse. Suppose you enter the name of a product that doesn’t exist in the database at all? You’re right, you’ll get the same message. Yet there are lots of software products that don’t have a CPE name, since they were never entered in the NVD. Just as in the fat-fingering case above, a user who searches for a CPE name that was never defined will see the same message as they would see if the CPE was defined, but there were no CVEs applicable to it; again, this is likely to give them a false sense of security.
5. Who’s responsible for entering a product in the NVD? Anyone can do it, but in general the supplier of the product should. In fact, Tom pointed out that suppliers of software products or intelligent devices should get in the habit of creating and entering a CPE name as soon as they’ve developed a new version of their product.
6. But most software products and devices (especially ones that are mainly used as components of other products – like firmware) don’t even have a CPE name; this is especially true, when you consider there needs to be a CPE name for every version of every product. In fact, it’s likely that most CPE names are only created when someone wants to enter a vulnerability for the product, since recording that a particular product is vulnerable to a particular CVE requires referring to the CPE for the product.
7. This means that you will never find any vulnerabilities listed for a product if the CPE for that product/version doesn’t exist. In fact, you’ll get the same message as in the cases discussed above: “There are 0 matching records.” Once again, you’ll come away with a false sense of security.
8. But in this case, the problem is much more serious, because the reason there were no matching records was the product was never entered in the NVD. And the product was never entered in the NVD because a vulnerability has never been reported for it.
9. Of course, the reason that a vulnerability has never been reported for a software product might be that the product has never had a single vulnerability…Yes, I think that’s hilarious, too. In fact, if a supplier has never entered a single vulnerability for their product, it’s likely because they’ve never looked for vulnerabilities in the first place, not because the product is so well coded that it’s never had one.

Let’s recap where we stand now: For those of you keeping score at home, the problem I’ve just described is that probably most software products in the US don’t appear in the NVD. This means that a user of one of those products, who wants to learn about vulnerabilities in it, will never find a single vulnerability – but they’ll get the same message that they would get if in fact the product was in the NVD (with its own CPE name), but it was truly vulnerability-free.

You may wonder, “Perhaps most of these products never had a vulnerability, or at least they’ve had such a small number that it really doesn’t matter that they’re not in the NVD. Could that be the case?"

Tom says NetRise has found a lot of products (again, mostly products that are used as components of other products) that seem to have a lot of vulnerabilities, yet there’s no CPE for any of them. He pointed (by name) to one particular CPE-less product – used in many ICS environments – whose manufacturer has never entered a single CVE for that product. Is this because they’re such careful coders that they never make mistakes? Not likely.

But it’s even better: That manufacturer is so good that there exist no CPEs for any of their products (and they have a bunch of them). Is this because they’ve never found any vulnerabilities, despite constantly searching for them in all of their products, all of the time? Again, not likely.

And I’ll add that this supplier is so good that their web site doesn’t contain a single mention about vulnerabilities or even product security in general. Of course, this means they really have their act together, right?

However, Tom, being a naturally skeptical guy, decided to do some scanning on one of that manufacturer’s products – just to see if perhaps there might have been one or two vulnerabilities that had escaped their attention. The good news? He didn’t find one or two vulnerabilities. The bad news? He found 1,237 – in that one product. And he could probably have found a lot more if he’d had the time.

Boys and girls, this is a big problem. Here we’re worrying about individual vulnerabilities - like the log4j vulnerabilities - that are found in thousands of products. Those are legitimate worries, but what about the tens or hundreds of thousands of products that have never even reported a vulnerability – since they’re not even in the NVD in the first place. In fact, there are suppliers who make lots of products, that have never reported a vulnerability on a single one of them. And it turns out that one randomly-chosen product, which has never reported a vulnerability, in fact has at least 1,237 vulnerabilities now.[i]

I think Tom’s presentation makes it very clear that software vulnerabilities are a big problem, and the vulnerabilities that we know about are just the tip of the iceberg. But what would you tell me to do if I told you that this was just the second-biggest problem that Tom pointed out?...Well, I’m not going to do that, but I will describe that problem in one of my next posts. Stay tuned.

This blog is part of a 3-part series, view Part 2 here.

References

[i] Of course, it’s likely that a lot of those vulnerabilities will turn out not to be exploitable, but Tom doesn’t have the resources to validate each one of them. He said he’d have to hire 15-20 people just to do that. But even if say only 5% of those 1,237 vulnerabilities are exploitable, that’s still over 60.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.