LocalCSVRuns on your device · nothing is uploaded

Remove duplicate rows from a CSV

Find and remove repeated rows — exactly, or by close match with every group shown before anything goes. Your file never leaves your computer.

How this works
  1. Add your file. Drop the spreadsheet onto the page.
  2. Choose what counts as a duplicate. Compare the whole row, or just the columns that identify a record.
  3. Check what will go. Close matches are shown group by group for you to accept, and the rows that would be removed are shown before anything is.
  4. Download. Save the cleaned file, and the removed rows if you want a record of them.
What it cannot do
  • Keeping the largest or smallest value compares numbers as numbers and dates written year-first as dates. A date written 31/01/2022 is compared as text, because reading it needs to know whether the file puts the day or the month first, and guessing would keep the wrong row.
  • Close matching compares each row against the rows that look like it rather than against every other row, which is what keeps it fast — but it can miss a pair that shares nothing at the start, the end, or the letters of the value.
  • Close matching reads the compared columns into the page to compare them, so it stops at 250,000 rows. It runs in a background thread, so the page stays usable while it works and changing a setting stops it. Removing exact duplicates has no limit and never leaves the engine.
  • Groups are joined up transitively: if A matches B and B matches C, all three are one group even when A and C are not alike. That is usually what "the same customer" means, and it is also why every group is shown to you rather than removed.
  • Exact matching holds one entry per distinct row in memory, so files with tens of millions of unique rows may not finish in a browser tab.
  • Rows are compared as written unless you turn on the options for spacing, capitalisation, or email addresses.
Questions
Can I remove rows that are nearly the same but not identical?
Yes. Close matching groups rows that look like the same record — spelling variants, a missing initial, a transposed pair of letters — and shows you every group with its rows before anything is removed. Nothing goes until you say which groups are right, because grouping rows that merely look alike is a guess rather than a rule.
Which copy of a repeated row is kept?
Your choice: the first one, the last one, or the one with the fewest blank fields. That last option is usually what people want when merging records from two systems.
Can I see what was removed?
Yes, and you can download it. The removed rows are a separate result, because someone always needs to prove afterwards exactly what went.
Are a.b@gmail.com and ab@gmail.com the same address?
For Gmail, yes — but for most company domains they are two different people. Email matching is therefore an option you turn on deliberately, not a default.
Is my file uploaded anywhere?
No. When you remove duplicate rows from a CSV here, the work happens inside this page and the browser is told to block outbound connections, so there is no upload to intercept. Watch the Network tab of your developer tools while you remove duplicate rows from a CSV: the only requests are for this site’s own code, and none carries your file.
Is it free?
Yes, with no signup and no row limit. Paid features cover saving your steps and re-running them on next month’s file, not the tools themselves.

Your file is processed in this page and never uploaded — the privacy page shows how to confirm that yourself.