A result needs its inspection context

Consider three attachments: a spreadsheet containing synthetic customer records, a scanned image of that spreadsheet, and a password-protected archive containing the same file. A detector may inspect the spreadsheet, lack optical character recognition for the image, and be unable to open the archive. Showing all three as No PII unless a rule matches would erase the most important difference between them.

A useful local inspection design starts with a narrow question: what content was actually available to this detector, under this policy, on this device? The answer should describe supported formats, successful extraction, relevant limits and the exact detector revision. Only then does a negative result have a meaning that an investigator can use.

A pattern match is evidence to assess

Microsoft's documentation describes sensitive information types as pattern-based classifiers and explains that supporting elements influence confidence. This is a useful distinction when designing your own evaluation: a sequence of digits may resemble an identifier, while nearby context changes how convincing the match is. A detector confidence label should describe its evidence, not claim a universal probability of correctness.

Use fictional examples to expose ambiguity. A purchasing document may contain product numbers shaped like a national identifier. A customer export may contain real identifiers without the keywords a rule expects. Confidential commercial information may contain no personal identifiers at all. Define separate detection strategies for personal data, organizational labels and organization-specific confidential content, then evaluate each against its intended use.

Represent every inspection outcome

Our recommended contract distinguishes inspected with findings, inspected without findings under named detectors, partially inspected, and uninspected. Attach a reason to incomplete coverage. Unsupported format, encryption, extraction failure, size limit, timeout and unavailable capability are operationally different and may need different follow-up. None should quietly become a clean result because an empty findings array is convenient.

Established products also document boundaries. Microsoft's Endpoint DLP guidance distinguishes supported content scanning from restrictions on unsupported extensions, and identifies additional image coverage when OCR is enabled. That does not establish AI Tool Tracking's coverage. It shows why buyers should ask every product for a tested format-and-capability matrix instead of assuming all files receive equivalent inspection.

  • Record the detected format and whether text extraction completed.
  • Record inspected scope and any limit that stopped processing.
  • Keep classification state independent from transfer evidence and policy response.
  • Show unavailable coverage explicitly in totals, filters and investigations.

Local processing still needs strict boundaries

Keeping inspection on a device can reduce the need to centralize document bodies, but it does not remove the risks of parsing untrusted files. The planned architecture should keep parsers and classifiers in constrained user-mode components. It should bound input size, expanded archive size, recursion, processing time, memory and temporary storage. Kernel components should have a narrowly justified role in system signals rather than parsing arbitrary business documents.

The output also needs minimization. A rule name, count, redacted finding and inspection reason may be sufficient for initial review. File paths, names and snippets can themselves reveal sensitive information. Decide which fields are necessary, protect access to them, and make their collection visible to the tenant administrator. Local inspection is an architectural choice; privacy depends on the entire resulting data flow.

Build a calibration pack before changing policy

Create a versioned synthetic corpus with known outcomes. Include ordinary documents, deliberate personal-data patterns, harmless near-matches, multiple languages, empty files, malformed files and documents that must remain uninspected. For each fixture, record expected extraction coverage separately from expected classification. This makes a parser regression distinguishable from a rule change.

Review false positives and missed cases by category rather than celebrating a single overall accuracy number. If a classifier handles common spreadsheets well but fails on images, an average can hide the gap. Before promoting a detector revision, compare old and new results, check device resource use, and confirm that failure states remain visible. Use a safe rollback path if the revision degrades coverage or disrupts endpoint work.

What is available today

The AI Tool Tracking demo contains fictional personal-data, confidential-data and uninspected scenarios. Its findings illustrate how an investigation interface should handle those states. It does not inspect uploaded documents, run a local classifier, read Microsoft sensitivity labels or deploy an agent. Detector coverage, parser containment and real endpoint performance are development requirements that must be validated before production use.

Sources & further reading