Why an API Key or Secret Was Not Detected

Data type: API keys, tokens, and secrets Article type: Known behavior / troubleshooting

Issue

An API key, token, or secret was present in scanned content but produced no finding, or scored lower than expected. Also covers the frequent report that key detection is "inconsistent" — working for some keys and not others.

Environment

All content integrations and endpoint scan paths.

Before troubleshooting: test the sample in the Nightfall Playground

The Playground returns the model's confidence score for any text you paste, independent of your policies and detection rules.

→ playground.nightfall.ai

Test the key in isolation and then with context, and compare. That single comparison separates a context problem from a pattern problem faster than anything else in this article.

Cause and resolution

1. The key was recognized but had no context around it

A bare key with nothing around it cannot reach Very Likely. Because the final score averages the pattern match with the context score, an isolated key is capped by the weak half of that average no matter how perfect the pattern is.

AWS_ACCESS_KEY_ID=AKIA…        → key pattern plus strong context
api_key: AKIA…                 → key pattern plus strong context
AKIA…  (alone in a message)    → pattern matches, no context, cannot reach Very Likely

This is the most common reason a key that "obviously should have been caught" is missing from a policy set to Very Likely. The detector found it; the policy filtered it out.

Resolution: If bare keys need to be caught, set the rule to Likely and measure the noise impact. Testing at Very Likely with a context-free sample will always understate what the detector is actually finding.

2. The key was in a table and the column header was lost

When a key sits in a table, the column header — API Key, Token, Secret — is usually the only context available. Whether it survives depends on the file type.

Header preserved: CSV, Excel, and other structured data files. Each value stays tied to its column:

service:     payments-api
environment: production
API Key:     sk_live_4eC39HqLyjWDarjtT1zdp7dc
owner:       platform-team

The label sits right next to the key. Strong context, high score.

Header lost: images of tables and HTML tables. Screenshots, photos, PDFs, and HTML flatten into a single run of text:

payments-api production sk_live_4eC39HqLyjWDarjtT1zdp7dc platform-team

The header row appears once, at the top. Every row after it is pushed further away:

Service  Env  API Key  Owner              ← the only place "API Key" appears
Row 1 …  …  sk_live_4eC39…  …             ← header is ~30 characters back
Row 2 …  …  sk_live_9fD21…  …             ← ~110 characters back
Row 40 …  …  sk_live_7bA84…  …            ← thousands of characters back

The detector looks for context within a short window around the value. Once the header falls outside it, each key is evaluated with nothing around it — and by cause 1, that means it cannot reach Very Likely.

This is why partial detection down a long column is such a common report. The first rows are close enough to the header; the rest are not.

Resolution: Scan the source file directly where you can — a CSV or spreadsheet will always outperform a screenshot of the same data. Where image and HTML coverage matters, lower the rule to Likely.

3. One character was misread during OCR, breaking the pattern

This hits API keys harder than any other data type, for a simple reason: keys are long. A provider key can run 40, 50, or more characters, and the pattern has to match all of them. One misread character and the pattern fails — there is no partial match and no lower-confidence finding. The key simply is not there.

The common OCR confusions are exactly the characters keys are built from:

The risk compounds with length. A 40-character key gives OCR forty chances to make one fatal error; a nine-digit SSN gives it nine. This is why key detection from screenshots is noticeably less reliable than key detection from text, even when the image looks perfectly legible.

sk_live_4eC39HqLyjWDarjtT1zdp7dc     → matches the pattern
sk_live_4eC39HqLyjWDarjtTlzdp7dc     → digit 1 read as lowercase L, no match

Resolution: Confirm whether the same key detects correctly as plain text. If it does, the problem is extraction, not detection. Send the original full-resolution image — a re-screenshot of a screenshot compounds the quality loss and often will not reproduce.

4. Coverage or configuration gap

Two different things get reported the same way.

The detector never ran. Check:

• Is secret detection deployed on the integration where the content lives? The common gap in engineering organizations is coverage on chat and file storage but not on code hosting or ticketing.

• Is the detection rule scope set to Content, not File Name only?

• Are minimum confidence or minimum number of findings filtering the result out?

• Is the policy in monitor-only mode with no automation and no end-user notification? That is experienced as "nothing happened."

The key format is one of yours. Internally-issued keys, and keys from providers our model has not yet learned. We can add support for your format.

Resolution: For an internal or unsupported key format, send us sanitized examples that preserve the prefix, length, and character set. Several examples are much better than one, since the pattern has to describe the variation as well as the shape. In the meantime, a custom regex detector anchored on your prefix is usually quick to stand up and often the right long-term answer for internal formats — but check the supported provider list in cause 5 first, so you are not rebuilding something that already exists.

5. A custom key detector is interfering

Custom "API Key or Token" regex detectors are consistently the largest noise source in tuning reviews — over-broad patterns matching any long alphanumeric string. The inverse also occurs: a hand-tightened custom regex misses real keys the native detectors would have caught.

The most common way this starts is a custom detector built for a provider we already cover.

ⓘ Important: before building a custom detector for a specific vendor

Check whether we already support it. A duplicate detector competes with the native one, and the native detector does something a regex cannot — it validates the key against that vendor's service to determine whether it is real and still live.

Vendors and services we support:

Anthropic · Auth0 · AWS · Amazon MWS · Azure · Box · Coinbase · Cohere · Confluence · Confluent · Databricks · Datadog · Dropbox · Elasticsearch · Facebook · GCP · GitHub · GitLab · Google API · Heroku · Hugging Face · Jira · JWT · Nightfall · Notion · Okta · OneLogin · OpenAI · PagerDuty (Events and REST) · PayPal · Perplexity · Ping Identity · Plaid · Postman · Postmark · RapidAPI · Salesforce · SendGrid · Shopify · Slack · Slack Webhook · Snyk · Splunk · Square · Stripe · Telegram Bot · Twilio · Twitter · Zapier

Plus generic token classes, recognized by shape and reported without validation since there is no single service to check them against:

access token · API key · authentication token · broker token · CSRF token · encryption key · geo tag · ID token · OAuth token · private key · refresh token · session token · generic token · unknown token

On the list? Use the native detector and tune it rather than building your own. If it is not firing, the cause is elsewhere — work causes 1, 2, and 3.

Not on the list? A custom detector is the right answer, and worth sending to us as an enhancement request at the same time (cause 4).

Resolution: Where a custom key detector is deployed, test the same samples against the native detectors before tuning either one. For over-broad custom detectors, raise confidence, tighten the regex, and run a tuning session against recent false positive examples. Where a custom detector duplicates a supported vendor, retiring it usually resolves both the noise and the attribution confusion at once.

What not to do

Do not lower confidence to Possible as a general fix. It will surface large volumes of ordinary high-entropy strings — hashes, UUIDs, build identifiers, encoded blobs. Validate any threshold change in the Playground or an isolated test policy first.

Reporting a missed detection to Nightfall Support

Non-findings are not retained. A missed key cannot be investigated retroactively — there is no stored artifact.

Contact Nightfall Support and include these three items at minimum:

Also helpful, in rough order of value:

1. Which provider issued the key, or confirmation that it is an internal format

2. Your Playground result — and whether it scored low or returned nothing. Different problems.

3. Whether the same key detects as plain text but not from the image or table

4. Whether some keys in the file were detected and others were not

5. Detection rule configuration: scope, minimum confidence, minimum findings

6. Whether a custom key detector is deployed alongside the native ones

Triage sequence

1. Paste the key into the Playground twice — once bare, once with it's native code or configuration context. Compare the two scores.

2. Bare scores low, labelled scores high → context, cause 1. The fix is the policy threshold, not the detector.

3. Neither returns anything → the pattern did not match. If the source was an image, work cause 3. If the key is an internal format, work cause 4.

4. Came from a table or spreadsheet → cause 2. Re-test the same data as a direct file upload.

5. Scored in the Playground but the policy did not fire → configuration, cause 4.

6. A custom key detector is in play → compare against the native detectors, cause 5.

7. Still unexplained → contact Nightfall Support with a sanitized sample, the time of the test, and the environment.