Why reconciling CMDB sources is more complicated than it seems

In this article we look at why maintaining a reliable CMDB is harder than it seems, which variables must be managed to succeed, and where the work of technology ends and that of modeling begins.

How many servers do you have? 

It sounds like a trivial question. In theory, it would be enough to open the CMDB—where discovery tools, cloud providers, and ticketing should flow together—and read the number. But try taking any server, one that has been in production for years, and searching for it in each of those sources. In the discovery tool it is called SRV-PROD-01. In the cloud provider console it is i-0a1b2c3d4e5f. In the legacy CMDB it appears as "Milan Production Server". In the ticketing system it is "App-Billing-Host", because whoever opens a ticket cares about the service that is not working, not the host behind it.

Four sources, four names, a single physical object. And without someone to recognize them as the same thing, each source will continue to treat them as four distinct elements because, from its perspective, they truly are. The risk goes in both directions: you can count the same server multiple times, or mistake two different servers for a single one and lose one along the way without realizing it. There is also a third, more subtle case: the total is even correct, but the information remains scattered among the occurrences—the software version in one place, the update status in yet another, the incident history in another—and you end up considering an asset "patched" whose evidence is actually attached to the wrong occurrence.

You only tend to notice this during critical moments: a compliance audit, an impact assessment before authorizing a change, planning which servers to decommission.

The problem is not that the data is dirty 

Faced with this situation, the most natural reaction is to think "we have a data quality problem". It is a wrong diagnosis, and it leads to treating the wrong thing. The discovery tool uses the hostname because that is what the operating system declares about itself. The cloud provider uses the resource ID because it is the only guaranteed unique identifier within its perimeter. The legacy CMDB uses descriptive names because it was populated by people, and people need to read something understandable. The ticketing system uses the application service name for the same reason. Each of these sources does its job well. The problem arises the moment we try to relate them to each other, discovering that nowhere in the architecture is there a layer dedicated to doing just that.

And the picture becomes even more complicated as soon as third-party sources come into play: a vulnerability database that names products according to its own CPE nomenclature, a vendor portal that publishes end-of-support dates with its own release code. You have no control over these sources—you certainly cannot ask a vulnerability database to adapt to your naming—but it is precisely by cross-referencing them with internal sources that you discover whether an asset is vulnerable or near end-of-life, information that no internal source can give you on its own.

Reconciling all this means asking not only what each source calls things, but also when it observes them and with what level of detail. These are different questions, and they generate five variables that must be managed one by one—not with a single rule for all.

The five variables to govern 

The first is naming: it is the example of the server from before, and it also applies to third-party sources. If the occurrences of the same asset remain separate, tickets accumulate on one, the CVE on another, the owner on neither. If, on the other hand, the matching is too aggressive, the opposite risk occurs: a single CI that actually represents two different machines, with conflicting attributes that no longer clearly belong to either.

Then there is granularity. Even when identity is resolved, it remains to be understood what we are actually talking about. SRV-PROD-01 is a host, but four virtual machines run on top of it, and on one of these about ten applications. Whoever looks at the hardware sees one object, whoever looks at the VMs sees four, whoever looks at the applications sees ten: three readings, all correct, taken at different depths, without any source keeping the thread from one level to the next. The result is that action is taken on one level without knowing what happens on the others—a change on the host seems to have zero impact, and instead a service goes down that no one had connected.

Then there is taxonomy: each source classifies according to its own logic. What one system calls a physical server, another calls a compute instance, a third a workload; a vendor identifies a release with an internal code that does not match the version detected by your discovery tool. These are not wrong categories; they were designed to answer different questions, but the consequence is that "how many Linux servers do we have" produces three different numbers depending on who asks and from where, and none of the three holds up in front of an auditor because none is reproducible.

The fourth variable, the most underestimated because it is invisible, is the update frequency. Some sources are continuous, others are weekly or monthly snapshots, and this also applies to third-party sources: a vulnerability database publishes new CVEs several times a day, a vendor portal updates end-of-support dates once a quarter. Two pieces of data can both be correct and yet incompatible, simply because they describe two different instants. Here, trust in the tool pays the price: the CI says the host is active, the host was actually decommissioned eleven days ago, and the ticket ends up on it anyway. After a couple of episodes like this, the service desk stops consulting the CMDB, and that is the worst damage, because it is the one that cannot be measured.

Finally, the most recent variable: the volatility of ephemeral assets, which grows with containers and short-lived workloads. A container that lives for forty minutes, inside a daily synchronization, is not a poorly inventoried asset: it is an asset that spawns and terminates between synchronization cycles an asset that is born and dies between one synchronization cycle and the next. Either it never appears in the CMDB, or it appears there already dead, turning out to be "active" when in reality it has not existed for hours. At that point, it is no longer just the data that needs to be rethought, but the very premise of the inventory—that the inventoried objects last longer than the interval with which you observe them.

Taken together, these five variables explain why there is no easy solution and why it is not enough to solve them once for internal sources: they must be kept under control even when sources that you do not manage come into play.

Why a correspondence table is not enough 

At this point the temptation is to rely on automation: build a mapping table, define correspondence rules, automate the join. Two weeks of work, maybe three. And this is precisely where the matter gets interesting, because reliable matching between heterogeneous sources is not done on names—which are the most unstable thing that exists—but on technical identifiers: serial number, system UUID, MAC address, FQDN, cloud provider resource ID.

The problem is that none of these identifiers covers the entire fleet. The serial number does not exist on a cloud instance, the MAC address changes when the VM is recreated, the FQDN changes with a domain migration, the resource ID is specific to a single provider, and the system UUID can regenerate with certain cloning operations. There is no universal key, so serious matching is necessarily cascading, and where there is no strong key, you end up in probabilistic matching, with similarity thresholds to be calibrated: too permissive and two distinct assets are merged, too strict and duplicates remain. There is no absolute right calibration: there is a right calibration for that fleet, at that moment, with those sources.

And even once found, that calibration does not last, for a precise reason: it depends on the sources as they are today, and the sources continue to change—an extra connector, an updated field format, a migration, an acquisition. A manually built table is recalibrated only when someone notices, that is, late. And even when it works, it only solves the identity of the objects—naming—it says nothing about how they are classified, when the sources observe them, or how they relate to each other.

A complete approach: technology and modeling 

At WEGG we are ITAM/ITSM consultants and we work on CMDB data quality projects. We know firsthand that there is no single solution to govern these five variables. And this is precisely why many CMDB projects fail: in the complexity of managing and updating them over time, preventing the drift from the infrastructure from becoming too wide.

Over time we have found a path that works for us, consisting of two distinct workstreams. On one hand, technology that continuously manages the identity of CMDB objects—naming and classification—always remaining aligned with the sources. On the other hand, work that no technology can do alone: building the reciprocal relationship between objects and deciding the level of detail with which to inventory them.

The technology we refer to is Flexera's IT Visibility: an automatic recognition and normalization engine, anchored to Technopedia, Flexera's reference catalog for hardware and software, which collects and classifies millions of existing IT products on the market in a standardized way. Unlike a static mapping table, it hooks into your internal sources but remains anchored to Technopedia, so it also brings along additional information such as known vulnerabilities and data provided directly by vendors. And when Technopedia updates—which it does several times a day—that update does not remain enclosed in the catalog: it propagates directly to the data in your CMDB, without the need for manual intervention to bring it back into alignment. The result is a unique identifier for each asset, a coherent product classification—category, family, version, etc.—and data aligned with sources almost in real time.

However, the other job remains, the one that technology alone cannot do: deciding the granularity with which you want to read the infrastructure. And this choice is not made in a vacuum: it depends on a service catalog defined upstream, and on how that catalog will need to be read inside the CMDB. Only by knowing which business services you need to represent can you decide down to what level of detail to go for each one.

It is the CMDB modeling work that we help do: adding business classification to the data, understanding which assets are truly critical: because criticality is not a property of the asset itself, but depends on the service it supports. The same server can be negligible or fundamental depending on what is downstream, and only the map of relations with business services tells us this with certainty. Take ephemeral assets: if they support a service, do you still want to track them? The technical channel to bring them in via API is already there, but the choice of whether or not to treat them as fully-fledged CIs remains with the IT staff and is a choice that makes sense only if you already know which service that asset belongs to.

Technology does an enormous job on raw data: it makes it readable, and on that basis it also allows building targeted queries depending on the objective. IT Visibility, for example, has a Power BI Embedded structure that organizes data according to different needs (vulnerabilities, obsolescence), with ready-made or custom-built templates, useful near an audit, remediation on unpatched machines, or technology refreshes. But this visibility stops at the individual asset. Viewing this data in relation to the business—knowing what is critical, what needs to be tracked, at what level of detail—is the added value that modeling brings.

In the October 15 webinar we will see in detail the advantages of this technology and how to manage drift and continuous updating of data sources in the CMDB with a 360-degree approach.

Article signed by Yari Formaggio , ITAM/ITSM Consultant at WEGG - The Impact Factory

02-s pattern02

Vorresti avere dati affidabili e normalizzati all'interno del tuo CMDB?

CONTACT US TO LEARN MORE!