The core rule: identical lines
The entire operation rests on one rule: lines that are exactly identical count as duplicates. Character for character, including spaces and punctuation — 'Apple' and 'Apple ' (with a trailing space) are different lines to the tool, as are 'Apple' and 'apple'.
This strictness is a feature, not a limitation. Exact matching is predictable: you always know precisely what will be removed, with no judgment calls hidden inside. Fuzzy matching would guess; exact matching just reports.
What 'kept' means
For each group of identical lines, one copy survives and the rest are removed. The surviving copy is the first occurrence — your list keeps its original order, with each unique line appearing where it first appeared.
Everything that is not a duplicate passes through untouched: unique lines, blank lines, formatting, and line order all stay exactly as you pasted them. The output is your input minus the repeats — nothing added, nothing rearranged.
What dedup does not do
Dedup does not fix near-duplicates: 'John Smith' and 'Jon Smith' are different lines and both survive. It does not trim spaces, normalize case, or correct typos — those are separate cleanup jobs to handle before or after dedup.
It also does not sort, reformat, or validate your data. An email list with invalid addresses comes out with the same invalid addresses, just without repeats. Dedup solves duplication and only duplication — plan your other cleanup steps separately.
Preparing text for the best result
The quality of your dedup depends on the consistency of your input. If trailing spaces or mixed case are creating 'duplicates' that are not really duplicates, normalize the text first — trim spaces, settle the case — then dedup.
Remove header rows from data exports before pasting if you process files in pieces: a header appearing in each chunk would survive as a 'unique' line per chunk but read as a repeat in the merged result. Clean inputs, clean outputs.
When in doubt, test on a small sample first: dedup ten lines, check the result, then run the full list with confidence.
A mental model to remember
Think of dedup as a bouncer with a guest list: the first 'John Smith' walks in, and every later identical 'John Smith' is turned away. The bouncer checks names exactly — no nicknames, no close-enoughs — and never rearranges the party inside.
With that model, every dedup question answers itself: will it catch this repeat? Only if it is character-identical. Will my order change? No — the party stays as it was, minus the turned-away duplicates. Simple, predictable, reliable.
Frequently asked questions
Are 'Apple' and 'apple' duplicates?
No. Matching is exact and character-for-character, so different cases (or trailing spaces) make lines distinct. Normalize case beforehand if you want them treated as one.
Does dedup change the order of my lines?
No. Each unique line stays where it first appeared; only the repeat copies are removed.
Will it fix typos or near-duplicates?
No. 'John Smith' and 'Jon Smith' are different lines to the tool. Handle typos as a separate cleanup step.