REMOVE DUPLICATES

How Does 2csv Decide Two Rows Are Duplicates?

Two rows are duplicates when the values you compare are the same. By default every column is compared, so only identical rows match. Choosing columns narrows the test: comparing only the email column treats two rows with the same email as duplicates even when the names or dates differ.

• Ignore upper/lower case, so Ann@Example.com and ann@example.com match.
• Ignore spaces at the ends, so a trailing space does not hide a duplicate.
• Keep the first or the last row of each group.

Can You See the Duplicates Before Removing Them?

Yes. Find duplicates lists every group, largest first, with how many copies it has and the row numbers of each copy. Nothing is removed at this stage, so you can check that the comparison is right before you commit. If the groups look wrong, change the columns or options and look again.

1. Press Find duplicates and read the groups.
2. Adjust columns or options if the groups are not what you expected.
3. Press Remove to delete the extra copies.
4. Press Undo if you need the removed rows back.

What Does Remove Duplicates Not Do?

Matching is exact, after the case and space options. Rows that are only similar, such as Jon Smith and John Smith, or two spellings of one street, are not treated as duplicates, because fuzzy matching is not in this version. Running Trim whitespace first removes invisible characters that can stop real duplicates from matching.

Removing duplicates works inside one file. When the same customers appear across several monthly exports, stack those files into one first with 2join, then open the combined file here and remove duplicates across all of it in a single pass.

What Else Can You Do With a CSV Here?

• CSV viewer: Open a CSV of any size, scroll every row, and see a profile of each column.
• Sort CSV: Sort rows by one or more columns, as text, numbers, or dates, ascending or descending.
• Filter CSV: Show only the rows that match an expression such as price > 10 AND state = "NY".
• CSV validator: Find ragged rows, broken quotes, bad headers, encoding problems, and values of the wrong type.
• Trim CSV whitespace: Trim leading and trailing spaces, collapse double spaces, and drop invisible characters and empty rows.
• Find and replace in CSV: Find text in one column or every column and replace it, with match case, whole cell, and regex options.
• Remove rows from CSV: Delete rows by number or range, the first or last rows, empty rows, or rows that match a rule.
• CSV editor: Edit cells, add or delete rows, and add, delete, rename, or move columns.
• Split CSV column: Split one column into several at a delimiter, such as a full name into first and last.
• Merge CSV columns: Join two or more columns into one with the separator you choose, skipping empty values.
• CSV column type converter: Convert a column to clean numbers, ISO dates, true/false, or upper, lower, or title case.
• CSV to JSON converter: Convert CSV to a JSON array of objects or arrays, or to JSON Lines, with typed values.
• JSON to CSV converter: Convert a JSON array or JSON Lines to CSV, flattening nested objects into dotted column names.
• CSV to SQL converter: Turn a CSV into SQL INSERT statements, with an optional CREATE TABLE, for PostgreSQL, MySQL, SQLite, or SQL Server.
• CSV to TSV converter: Convert a comma-separated file to tab-separated values that paste cleanly into spreadsheets.
• TSV to CSV converter: Convert a tab-separated file to a standard, properly quoted comma-separated CSV.
• CSV splitter: Split a large CSV into smaller files of a set number of rows, with the header repeated in each.

All eighteen tools are on the 2csv home page, and once a file is open you can switch between them without reloading it. Part of the 2suite.io privacy-first ecosystem.