How It Works Under the Hood
List Cleaner normalizes lines through an explicit pipeline: optional line trimming, blank-line removal, deduplication (preserving original first-seen order via `Set`), sorting (`Alphabetical`, `Numeric/Natural` via `Intl.Collator`, `By Length`, or Fisher-Yates `Shuffle`), and optional prefix/suffix wrapping or custom delimiter joining. Set Compare maps List A and List B into hashed sets for instant exact set algebra.
Step-by-Step Instructions
Paste your raw list into the Clean & Sort workspace (or switch to Set Compare with List A and List B).
Toggle `Trim whitespace`, `Remove empty lines`, `Deduplicate`, and `Case-Insensitive` options.
Pick a Sort Mode (`Original Order`, `A → Z`, `Z → A`, `Natural Numeric`, `Line Length`, or `Shuffle`).
Optionally add Prefix/Suffix quotes (e.g., `'` and `'`) and Join Separator (`, `) to generate instant SQL `IN (...)` tuples, or read exact counts of `Intersection`, `Only in A`, and `Only in B`.
Key Features & Performance
- Preserves original order during deduplication unless a sort mode is picked
- Fisher-Yates unbiased shuffle and `Intl.Collator` natural sorting (`item2` before `item10`)
- Built-in Prefix / Suffix / Joiner to convert newline lists into SQL `IN ('a', 'b')` or JSON lines instantly
- Full 4-way Venn Set Comparator (`Only A`, `Only B`, `A ∩ B`, `A ∪ B`)
- Processes email lists, customer IDs, or firewall IPs with zero uploads
Methodology, Formulas & Technical Standards
Deduplication runs in $O(n)$ using a JavaScript `Set` keyed either on raw string or `.toLocaleLowerCase()`. Set algebra maintains canonical strings from List A first on `A ∩ B` when case-insensitive comparison is toggled.
Common Real-World Use Cases
- Comparing yesterday’s deployed microservice list against today’s running pods (`A − B` / `B − A`)
- Turning 200 pasted order IDs from Excel into a deduplicated comma-separated SQL `WHERE id IN (...)` clause
- Cleaning duplicate SEO keywords or domain allowlists without opening a spreadsheet
- Sorting numbered build artifacts (`release-2` before `release-10`) naturally
Worked Examples & Edge Cases
Excel Column to SQL IN List
ord_101\n ord_102 \nord_101\n
'ord_101', 'ord_102'
Enable Trim + Remove empty + Deduplicate, Prefix=`'`, Suffix=`'`, Join=`, `.
Set Comparison (A vs B)
A: [alpha, beta, gamma] | B: [beta, delta]
Intersection (A∩B): [beta] | Only A: [alpha, gamma] | Only B: [delta]
Useful for quick database seed verification or recipient list exclusions.
Limitations, Scope & Accuracy Notes
- • Set operations match whole line values (trimmed optionally); structured multi-column CSV joins belong in SQL/CSV tools.
- • Very large pasted text (>10 MB / several hundred thousand lines) can momentarily pause browser tab rendering during DOM updates.
Local Browser Processing & Privacy
Traditional web converters upload your documents, snippets, or media files to remote servers. Line List Cleaner, Deduplicator & Set Comparator runs inside your browser tab:
Your input text and files are processed locally without uploading payloads to a processing server.
Microdeck does not log or store the files or inputs you paste into tool workspaces (normal static hosting/CDN requests still load page assets).
Working outputs stay in your browser tab and are released when you reset the tool or close the tab.
Frequently Asked Questions
Does deduplication shuffle my list if I leave Sort on "Keep Original Order"?
No. We preserve the first occurrence index of every unique item in order.
What is Natural Numeric sorting?
Standard ASCII sorting puts `file10.png` before `file2.png`. Natural sorting compares embedded digit chunks numerically so `file2.png` comes first.
Can I compare two email lists without uploading customer PII?
Yes. All Set Compare operations run completely inside your browser tab RAM.
How does Case-Insensitive Set Compare pick casing for Intersection results?
When two lines differ only in case (`Alice@x.com` vs `alice@x.com`), the Intersection output retains the spelling from List A.
Explore Related Micro-Tools
Text Diff Checker & Patch Generator
Compare two text blocks line-by-line or word-by-word and export unified .patch files
CSV & Log PII Data Anonymizer
Mask, hash, or redact Emails, Phones, IPs, Cards & API tokens before sharing with AI
Text Case & Vietnamese-Aware SEO Slug Converter
Convert between 11 cases (camelCase, snake_case, kebab-case, Title Case) and clean URL slugs