How It Works Under the Hood
The inspector iterates over Unicode scalar code points cleanly using `for (const ch of input)` (keeping emoji surrogate pairs intact) rather than naive `.split("")`, encodes UTF-8 byte sequences with `TextEncoder`, and classifies dangerous format controls: Zero-Width characters (`U+200B`–`U+200D`, `U+2060`, `U+FEFF`), Bidirectional overrides (`U+202A`–`U+202E`, `U+2066`–`U+2069`), and non-ASCII space variants (`U+00A0` NBSP).
Step-by-Step Instructions
Paste suspicious source code, a domain name, markdown copy, or error-causing config string.
Inspect the Diagnostic Alert banner highlighting any hidden Zero-Width chars, BiDi Trojan Source controls, or Non-Breaking Spaces (`U+00A0`).
Browse the per-code-point breakdown table showing glyph badge, `U+XXXX`, decimal scalar, UTF-8 hex bytes, and hazard classification.
Click Strip Invisible Controls, Replace Special Spaces with ASCII Space, or Normalize (`NFC`, `NFD`, `NFKC`, `NFKD`) and copy the clean string.
Key Features & Performance
- Detects CVE-2021-42574 Trojan Source BiDi override characters (`RLO`, `LRO`, `PDI`, etc.)
- Flags invisible `U+200B` ZWSP, `U+200C`/`U+200D` joiners, `U+FEFF` BOM, and `U+00A0` NBSPs that silently break parsers/URLs
- Proper astral-plane handling for 4-byte UTF-8 emojis (never splits surrogate pairs)
- One-click `NFC` / `NFD` / `NFKC` / `NFKD` normalization and control stripping
- 100% browser-only execution safe for security incident strings
Methodology, Formulas & Technical Standards
Characters are split via Unicode code point iteration (`Array.from(text)`), while UTF-8 byte lengths derive from `new TextEncoder().encode(ch)`. Normalization invokes built-in ECMAScript UAX #15 (`String.prototype.normalize`). Note that legitimate ZWJ (`U+200D`) inside compound emojis (e.g., family emojis) is flagged visibly so you choose whether to strip controls.
Common Real-World Use Cases
- Finding why two visually identical YAML keys, JSON keys, or Python identifiers fail equality checks (`U+200B` or mixed NFC vs NFD Vietnamese accents)
- Auditing untrusted code snippets for hidden Trojan Source directional override spoofing before merging
- Cleaning copy-pasted ChatGPT/Word text full of `U+00A0` non-breaking spaces or stray `U+200B` watermarks
- Verifying exact UTF-8 byte representation of characters for protocol debugging
Worked Examples & Edge Cases
Hidden Zero-Width Space in Variable Name
apiKey = "123"
Flagged U+200B (ZERO WIDTH SPACE) at index 3 -> Sanitized: apiKey = "123"
Visually looks like `apiKey` in most fonts but causes `ReferenceError` or auth header mismatches.
Vietnamese NFD vs NFC Normalization
tiếng (decomposed e + combining acute + circumflex = 7 chars)
NFC -> tiếng (composed = 5 chars, identical visually)
macOS HFS+/APFS filenames often use NFD while Linux/Windows/Web use NFC.
Limitations, Scope & Accuracy Notes
- • Stripping all `U+200D` (Zero Width Joiner) will split multi-person compound ZWJ emojis into individual base emojis.
- • Cross-script visual homoglyphs (e.g., Cyrillic `а` vs Latin `a`) are distinguishable by viewing their `U+XXXX` hex scalar (`U+0430` vs `U+0061`) rather than stripped automatically so legitimate Cyrillic/Greek text is never mangled.
Local Browser Processing & Privacy
Traditional web converters upload your documents, snippets, or media files to remote servers. Unicode Inspector & Hidden Character Sanitizer runs inside your browser tab:
Your input text and files are processed locally without uploading payloads to a processing server.
Microdeck does not log or store the files or inputs you paste into tool workspaces (normal static hosting/CDN requests still load page assets).
Working outputs stay in your browser tab and are released when you reset the tool or close the tab.
Frequently Asked Questions
What is a Trojan Source BiDi attack?
Tracked as CVE-2021-42574, attackers embed invisible Bidirectional override characters (`U+202E`, `U+2066`, etc.) inside comments or strings so human reviewers see different logic order than the compiler executes.
When should I choose NFC vs NFKC normalization?
Use `NFC` for standard text and Vietnamese/European accents. Use `NFKC` when you also want compatibility characters (like fullwidth ASCII `A`, superscript `²`, or ligatures `fi`) collapsed to plain ASCII equivalents.
Why does macOS sometimes show different Unicode lengths than Linux for the same word?
macOS filesystem paths frequently store accented letters in decomposed `NFD` form (base letter + combining mark), whereas web forms and Linux generally emit `NFC` precomposed characters.
Does the inspector alter emojis cleanly?
Yes, code-point iteration reads full 21-bit scalars (`U+1F680` etc.) instead of splitting UTF-16 surrogates into broken halves.
Explore Related Micro-Tools
Text Case & Vietnamese-Aware SEO Slug Converter
Convert between 11 cases (camelCase, snake_case, kebab-case, Title Case) and clean URL slugs
Text Diff Checker & Patch Generator
Compare two text blocks line-by-line or word-by-word and export unified .patch files
Base64, URL & Hex Multi-Format Encoder/Decoder
Auto-detecting multi-codec converter for Base64, URL percent-encoding, Hex & HTML