Article

    Regex Library | Nightfall Documentation

    4 min read
    Last updated 1 month ago

    Nightfall provides detectors for the most common data protection use cases. For unique situations, you can build custom detectors using regular expressions.

    Secrets Detection

    Please double-check our Nightfall Detector Glossary before creating your own, including the API and cryptographic key detectors listed below, as regex detectors can introduce noise.

    Nightfall's API key Detector

    Nightfall's API key supports specific detection and validation of API keys for the top 50 vendors and use cases, as shown below.

    • AWS

    • Azure

    • Confluence

    • Confluent

    • Datadog

    • ElasticSearch

    • Facebook • GCP

    • Google API

    • GitHub

    • GitLab

    • Hugging Face

    • JIRA

    • Nightfall

    • Notion

    • Okta

    • OpenAI

    • PagerDuty

    • Paypal

    • Plaid

    • Postmark

    • Postman

    • Salesforce

    • Sendgrid

    • Slack

    • Snyk

    • Square

    • Stripe

    • Twitter

    • Twilio

    • Zapier

    • Authentication Token

    • CSRF Token

    • OAuth Token

    • Generic API Key

    • Generic Token

    • JWT

    • Private Key

    • Refresh Token

    • Session Token

    Nightfall's Cryptographic Key Detector

    Nightfall's identifies popular keys for locking or unlocking cryptographic functions, including authentication, authorization, and encryption.

    • DSA Private Key

    • RSA Private Key

    • EC Private Key

    • OpenSSH Private Key

    • Private Key

    • Encrypted Private Key

    • PGP Private Key Block

    You can send us a request for new ML detectors directly in Nightfall.

    REGEX Library

    Here is a list of regex detectors used by other Nightfall customers.

    Name

    Detector

    Category

    google_two_factor_backup

    ^(?:BACKUP VERIFICATION CODES|SAVE YOUR BACKUP CODES)[\s\S]{0,300}@$

    Credentials

    heroku_key

    ^(heroku_api_key|HEROKU_API_KEY|heroku_secret|HEROKU_SECRET)[a-z_ =\s"'\:]{0,10}[^a-zA-Z0-9-]\w{8}(?:-\w{4}){3}-\w{12}[^a-zA-Z0-9\-]$

    Credentials

    MailGun API Key

    ^key-[0-9a-zA-Z]{32}$

    Credentials

    microsoft_office_365_oauth_context

    ^https://login.windows.net/common/oauth2/token$

    Credentials

    PayPal Braintree Access Token

    ^access_token\$production\$[0-9a-z]{16}\$[0-9a-f]{32}$

    Credentials

    Picatic API Key

    ^sk_live_[0-9a-z]{32}$

    Credentials

    ECDSA Private Key

    ^-----BEGIN ECDSA PRIVATE KEY-----\s.*,ENCRYPTED(?:.|\s)+?-----END ECDSA PRIVATE KEY-----$

    Credentials

    KeePass 1.x CSV Passwords

    ^"Account","Login Name","Password","Web Site","Comments"$

    Credentials

    KeePass 1.x XML Passwords

    ^<pwlist>\s*?<pwentry>[\S\s]*?<password>[\S\s]*?<\/pwentry>\s*?<\/pwlist>$

    Credentials

    Password etc passwd

    ^[a-zA-Z0-9\-]+:[x|\*]:\d+:\d+:[a-zA-Z0-9/\- "]*:/[a-zA-Z0-9/\-]*:/[a-zA-Z0-9/\-]+$

    Credentials

    Password etc shadow

    ^[a-zA-Z0-9\-]+:(?:(?:!!?)|(?:\*LOCK\*?)|\*|(?:\*LCK\*?)|(?:\$.*\$.*\$.*?)?):\d*:\d*:\d*:\d*:\d*:\d*:$

    Credentials

    MailChimp API Key

    ^[0-9a-f]{32}-us[0-9]{1,2}$

    Credentials

    PGP Header

    ^-{5}(?:BEGIN|END)\ PGP\ MESSAGE-{5}$

    Credentials

    PKCS7 Encrypted Data

    ^(?:Signer|Recipient)Info(?:s)?\ ::=\ \w+|[D|d]igest(?:Encryption)?Algorithm|EncryptedKey\ ::= \w+$

    Credentials

    PuTTY SSH DSA Key

    ^PuTTY-User-Key-File-2: ssh-dss\s*Encryption: none(?:.|\s?)*?Private-MAC:$

    Credentials

    PuTTY SSH RSA Key

    ^PuTTY-User-Key-File-2: ssh-rsa\s*Encryption: none(?:.|\s?)*?Private-MAC:$

    Credentials

    Samba Password config file

    ^[a-z]*:\d{3}:[0-9a-zA-Z]*:[0-9a-zA-Z]*:\[U\ \]:.*$

    Credentials

    SSH DDS Public

    ^ssh-dss [0-9A-Za-z+/]+[=]{2}$

    Credentials

    SSH RSA Public

    ^ssh-rsa AAAA[0-9A-Za-z+/]+[=]{0,3} [^@]+@[^@]+$

    Credentials

    SSL Certificate

    ^-----BEGIN CERTIFICATE-----(?:.|\n)+?\s-----END CERTIFICATE-----$

    Credentials

    Lightweight Directory Access Protocol

    ^(?:dn|cn|dc|sn):\s*[a-zA-Z0-9=, ]*$

    Credentials

    Arista network configuration

    ^via\ \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3},\ \d{2}:\d{2}:\d{2}$

    Network

    John the Ripper

    ^[J,j]ohn\ [T,t]he\ [R,r]ipper|john-[1-9].[1-9].[1-9]|Many\ salts:|Only\ one\ salt:|openwall.com/john/|List.External:[0-9a-zA-Z]*|Loaded\ [0-9]*\ password hash|guesses:\ \d*\ \ time:\ \d*:\d{2}:\d{2}:\d{2}|john\.pot$

    Network

    Huawei config file

    ^sysname\ HUAWEI|set\ authentication\ password\ simple\ huawei$

    Network

    Metasploit Module

    ^require\ 'msf/core'|class\ Metasploit|include\ Msf::Exploit::\w+::\w+$

    Network

    Network Proxy Auto-Config

    ^proxy\.pac|function\ FindProxyForURL\(\w+,\ \w+\)$

    Network

    Nmap Scan Report

    ^Nmap\ scan\ report\ for\ [a-zA-Z0-9.]+$

    Network

    Cisco Router Config

    ^service\ timestamps\ [a-z]{3,5}\ datetime\ msec|boot-[a-z]{3,5}-marker|interface\ [A-Za-z0-9]{0,10}[E,e]thernet$

    Network

    Simple Network Management Protocol Object Identifier

    ^(?:\d\.\d\.\d\.\d\.\d\.\d{3}\.\d\.\d\.\d\.\d\.\d\.\d\.\d\.\d\.\d{4}\.\d)|[a-zA-Z]+[)(0-9]+\.[a-zA-Z]+[)(0-9]+\.[a-zA-Z]+[)(0-9]+\.[a-zA-Z]+[)(0-9]+\.[a-zA-Z]+[)(0-9]+\.[a-zA-Z]+[)(0-9]+\.[a-zA-Z0-9)(]+\.[a-zA-Z0-9)(]+\.[a-zA-Z0-9)(]+\.[a-zA-Z0-9)(]+$

    Network

    Bank of America Routing Numbers - California

    ^(?:121|026)00(?:0|9)(?:358|593)$

    Finance

    BBVA Compass Routing Number - California

    ^321170538$

    Finance

    Chase Routing Numbers - California

    ^322271627$

    Finance

    Citibank Routing Numbers - California

    ^32(?:11|22)71(?:18|72)4$

    Finance

    USBank Routing Numbers - California

    ^12(?:1122676|2235821)$

    Finance

    United Bank Routing Number - California

    ^122243350$

    Finance

    Wells Fargo Routing Numbers - California

    ^121042882$

    Finance

    SWIFT Codes

    ^[A-Za-z]{4}(?:GB|US|DE|RU|CA|JP|CN)[0-9a-zA-Z]{2,5}$

    Finance

    CVE Number

    ^CVE-\d{4}-\d{4,7}$

    General

    Dropbox Links

    ^[l)/\\S+$](https://www.dropbox.com/(?:s)

    General

    Box Links

    ^[l\]/\\S+$](https://app.box.com/\\[s)

    General

    Large number of US Zip Codes

    ^(\d{5}-\d{4}|\d{5})$

    General

    MySQL database dump

    ^DROP DATABASE IF EXISTS(?:.|\n){5,200}CREATE DATABASE(?:.|\n){5,200}DROP TABLE IF EXISTS(?:.|\n){5,200}CREATE TABLE$

    Database

    MySQLite database dump

    ^DROP\ TABLE\ IF\ EXISTS\ \[[a-zA-Z]*\];|CREATE\ TABLE\ \[[a-zA-Z]*\];$

    Database

    If you need help with regexes or have regexes you'd like to share, please reach out to support@nightfall.ai.