How Does Validation Work?

Understanding Validation requires some patching background ("Truths" 1-4 below), plus an ability to visualize how the validation metadata is used (table examples below). Validation is not merely a "file-by-file" check for component files for each patch you ever installed (which would cause many false "fails" that are not fails.). Instead, validation is based on an understanding of events in the Microsoft patch and operating systems environments, and offers choices in how to validate. The summary below is a snapshot of validation. To fully understand validation, you must read the detail following the summary.

Validation Summary

• Updates often install only a subset of the files in the update.

• Version, Size, and CheckSum checking are methods for catching "back-revved" files, meaning older files that have overwritten newer files, or for catching "corrupted or altered" files.

• "Versions Match Exactly" and "Version installed is greater than or equal" options essentially operate on the principal that it is normal for newer files to supercede older files.

• You can select one, two, or all three validation methods in any combination.

• Validation knows component files per patch, but has no concept of required vs. optional components as this information is not contained in the patch itself.

• Validation can't know whether a file should or should not exist on the target.

• Validation can't know a file was accidentally deleted by a user.

• Validation can't know what other applications have done (installing or deleting files).

Validation Details

Data has to be entered into the Remediation Database at St. Bernard, and propagated to subscribers. The addition of metadata will occur in stages, eventually resulting in complete coverage, at which time the validation additions will continue for new updates that become publicly available.

Validation data includes:

• Filename

• Version

• FileSize (one or more for same version)

• CheckSum (CRC value, one or more for same version/filesize)

Truth-1: UpdateEXPERT validation metadata is based on deconstruction of Microsofts own patch files. This data is NOT based on articles, or documentation of any kind. This results in the most accurate metadata possible.

File Handling

Truth-2: Service-Packs and other updates often contain component files that don't necessarily get installed, depending on the environment of the target machine being patched. Because the validation metadata must account for all potential files, filenames can exist in the table that don't exist on the target machine. Failing a machine because an entry in the table wasn't found on the target could cause many fails that should not be a fail.

Truth-3: UpdateEXPERT cannot "know" what files should or should not be resident on a target system. What it CAN do is check the validity of the files that are there already, that belong to a known installed patch.

Based on the above, files known to UpdateEXPERT, but not found on the target do NOT cause a fail. Files that ARE found on the target can be validated using version, filesize, and checksum. A problem with one or more of those validation checks would cause a patch to fail validation.

This also means that if someone inadvertantly deletes a DLL, validation will NOT detect the deletion and may "pass" the patch. In summary, validation cannot know if a file was deleted, or simply not installed.

Building the Validation Table

When Validation is run for a target machine, the machine is queried to build a list of installed patches. For each installed patch, validation data is pulled from the remediation database and built into a memory resident table. For a given patch, there are typically many files, each one with their own version, size and checksum. The same version of a given file can occur across different patches, yet the size will be different. For a specific version and size, different checksums can occur. Microsoft's product build procedures account for this reality. For example, a different build date causes a different checksum, even though version and size are unchanged.

The table below shows File1.dll occurring across multiple patches, with different versions, filesizes and checksums. Think of this table as showing component-file data by patch. As you can see, File1.dll exists across various patches (Update1.exe etc.).

Note: All tables below (except Table-8) represent memory-resident metadata built by validation. Blue items represent metadata being matched up against target file data on the machine.

Table-1 - Component files by patch

To explain validation, it's more useful to see the table data organized by file and version. Of course, UpdateEXPERT knows the relationships between patches and component files.

Table-2 - Component files by version

FileSize Validation ONLY

If you select size validation ONLY, all filesizes for File1.dll would be candidates for a size match against the file on the target.

When a match is found, as on 27780 below, the updates installed with the corresponding version or older are "passed", while updates using newer versions are "failed". If no match is found, all updates would be failed.

Note that in reality we would expect to get a match on one of the version "c" sizes. The newest version is the "expected version", meaning the version that validation is expecting to find on the target. The match on version "b" is primarily to illustrate which patches would fail or pass based on the match shown in blue (italics).

Table-3 - Matching on FileSize

CheckSum Validation ONLY

If you select checksum validation ONLY, all checksums for File1.dll would be candidates for a checksum match against the file on the target.

When a match is found, as on 923 below, the updates installed with the corresponding version or older are "passed", while updates using newer versions are "failed". If no match is found, all updates would be failed.

Note that in reality we would expect to get a match on one of the version "c" checksums. The newest version is the "expected version", meaning the version that validation is expecting to find on the target. The match on version "a" is primarily to illustrate which patches would fail or pass based on the match shown in blue (italics).

Table-4 - Matching on CheckSum

FileSize & CheckSum Validation

If you select filesize and checksum validation, all filesizes for File1.dll would be candidates for a size match against the file on the target. Note that size & checksums are paired internally, ie., checksum "75" is paired with size "28432", "78" is paired with "28765", and "83/85" are paired with "29442" (See Table-1 entries).

When a match is found, as on 29442 below, the checksums (83 and 85) in the table are compared against the checksum of the file, and assuming a match with either value, the corresponding version or older are "passed", while updates using newer versions are "failed". If no match is found, all updates would be failed.

As expected, we got a match on one of the version "c" sizes. The key point here is to realize that sometimes a specific file can have multiple versions, filesizes within version, and multiple CRC values within filesize.

Table-5 - Matching on FileSize and CheckSum

Version Validation ONLY

Truth-4: Application installs can write versions of files that are either older or newer than the versions known in the validation table. If a product "back-revs" a DLL to an older version, validation will detect that and fail one or more patches, depending on how far back the DLL revision is. If a product (or a Microsoft "private fix") installs a newer DLL, you don't necessarily want to "fail" validation because a newer version of the DLL supercedes all of your patch versions... the "greater than" version match lets you "pass" these items, thus reducing fails.

If you select version validation ONLY, all versions for File1.dll would be candidates for a version match against the file on the target. As always, the newest version is the "expected version", meaning the version that validation is expecting to find on the target.

When a version match is found, as on "c" below, the related updates and updates for older versions are "passed", while updates installed with newer versions would be "failed". If no match is found, all updates would be failed. The use of "Versions Match Exactly" option vs. "Version installed is greater than or equal" option does not matter in this example.

Table-6 - Matching on Expected Version

If "c" is not on the target, but "b" is, then updates 2, 4, 5, & 6 will fail as they were installed with "c," but update 1 and 3 would pass. Again, "Versions Match Exactly" option vs. "Version installed is greater than or equal" option does not matter in this case.

Table-7 - Matching on Non-Expected Version

If "a", "b" & "c" were not on the target, but "d" (newer than a, b, or c) had somehow been installed on the target by an application or a private hotfix from Microsoft, all updates would fail with "Versions Match Exactly" option (see Table-9), but pass with "Version installed is greater than or equal" option (see Table-10).

Table-8 - Newer file found on Target that doesn't exist in the validation metadata table

Table-9 - Matching on Version using "Versions Match Exactly" option, assuming "d" is on the target

Table-10 - Matching on Version using "Version installed is greater than or equal" option, assuming "d" is on the target

Version with Size and/or Checksum Validation

When you combine version with size and/or checksum, the version, size and calculated checksum from the file on the target are matched against the validation table.

When a version, size and checksum match are found, as on "c" below, the related updates and updates for older versions are "passed", while updates installed with newer versions would be "failed".

Table-11 - Matching on Version with Size and/or CheckSum

If "Version installed is greater than or equal" is selected, validation will "pass" the version check for newer versions not in the validation table (such as "d" below), but produce "fails" on size/checksum for these same files because there is no size or checksum data with which to do a compare.

Table-12 - Matching on Version d

If "Versions Match Exactly" is selected, then newer versions not in the validation table (such as "d" above) will "fail" on version, size and checksum (all three checks).

Validation Choices

• Select Version (with "Versions Match Exactly"), Size & CheckSum for the most complete validation possible based on the UpdateEXPERT validation table. Note: You may get some fails on "Versions Match Exactly" if you've deployed Microsoft private fixes with newer file versions than validation knows about, to some target machines.

• Select Version (with "Versions Match Exactly"), and Size to reduce overhead slightly, yet keep a file integrity check.

• Select Version (with "Versions Match Exactly") only for low overhead; failing "unknown" and "older-than-expected" versions.

• Select Version (with "Version installed is greater than or equal") for low overhead. You may get some fails on size/checksum with the "Version installed is greater than or equal" option, but only when the target file is a newer version than anything known in the validation table.

• You can select "Check Size" and/or "Check CheckSum" only, as long as you recognize there is a very slight chance that size and/or checksum could pass a file where the version is not actually a match. This is unlikely, but possible.

From "FILE | AGENT | SETTINGS --> Validation Tab" you can select the validation methods (size, checksum, version) you wish to use.