1. Home
  2. Text
  3. Remove Duplicate Lines

Remove Duplicate Lines

Remove repeated lines from a list and see what was duplicated.

Runs in your browser · No upload
Total lines
7
Lines kept
4
Lines removed
3
Result
anna@example.com ben@example.com chris@example.com Ben@example.com
Duplicated lines
LineOccurrences
anna@example.com3×
chris@example.com 2×

How to use

  1. Paste a list with one item per line, such as keywords, email addresses, SKUs or order numbers.
  2. The deduplicated list appears right away, along with counts of total, kept and removed lines.
  3. Leave Show duplicated lines on to see which lines repeated and how many times.
  4. Use Copy or Download .txt to take the result with you.

What the options do

Option Behavior Example
Ignore leading/trailing spaces Spaces and tabs at either end are ignored when comparing. The line you keep is output exactly as it was typed apple and apple count as the same
Ignore case Uppercase and lowercase letters count as equal Apple and apple count as the same
Remove empty lines Drops blank lines and lines with only spaces When this is off, blank lines are deduplicated like any other line
First / last occurrence Chooses which copy of a repeated line to keep. Either way, the remaining lines stay in their original order a b a c → first: a b c, last: b a c

Example

The sample list has seven email addresses: anna three times, chris twice (once with a trailing space) and ben twice (once as Ben). With the default options, it shrinks to four lines, because ben and Ben differ only in case. Turn on Ignore case and you get three lines (anna, ben, chris), with four removed.

Good to know

  • Lines are compared as whole lines. Repeated words within a single line are left alone.
  • Windows (CRLF), classic Mac (CR) and Unix (LF) line endings are all recognized, and the result uses LF. A single newline at the very end of your input doesn’t count as an extra empty line.
  • Text is Unicode-normalized (NFC) before comparing, so accented letters that look the same but were typed differently still match.

FAQ

Can I remove duplicates from an Excel or Google Sheets column?

Yes. Copy the column and paste it here, and each cell becomes one line. Then paste the result back into your sheet.

I also want the list sorted.

The sort lines tool sorts alphabetically or naturally and can drop duplicates at the same time. Use this tool when you need to keep the original order.

How do I compare two lists?

Paste them into text diff to see added and removed lines highlighted.

Can't find the tool you need?

Tell us what you want to do. We build the most requested tools first.