//  home   //  advanced search   //  news   //  categories   //  sql build chart   //  downloads   //  statistics
 ASP FAQ 
Home
ASP FAQ Tutorials

   8000XXXX Errors
   Alerts
   ASP.NET 2.0
   Classic ASP 1.0
   Databases
      Access DB & ADO
      General SQL Server & Access Articles
      MySQL
      Other Articles
      Schema Tutorials
      Sql Server 2000
      Sql Server 2005
   General Concepts
   Search Engine Optimization (SEO)

Contact Us
Site Map

Search

Web
aspfaq.com
tutorials.aspfaq.com
databases.aspfaq.com

ASP FAQ Tutorials :: Databases :: Other Articles :: How do I remove duplicates from a table?


How do I remove duplicates from a table?

If you have a primary key in the table (let's call it PK_Column), you can do something like this with a sub-query: 
 
CREATE TABLE dbo.Dupes 

    PK_Column INT IDENTITY(1,1), 
    Dupe_String VARCHAR(32) 

GO 
 
SET NOCOUNT ON 
GO 
 
INSERT dbo.Dupes(Dupe_String) 
    SELECT 'foo' 
    UNION ALL SELECT 'foo' 
    UNION ALL SELECT 'foo' 
    UNION ALL SELECT 'bar' 
    UNION ALL SELECT 'bar' 
    UNION ALL SELECT 'splunge' 
    UNION ALL SELECT 'splunge' 
    UNION ALL SELECT 'mort' 
GO 
 
SELECT * FROM dbo.Dupes 
GO 
 
DELETE dbo.Dupes 
    FROM dbo.Dupes d 
    LEFT OUTER JOIN  
    ( 
        SELECT Dupe_String, pk_column = MIN(PK_Column) 
        FROM dbo.Dupes 
        GROUP BY Dupe_String 
    ) x 
    ON d.pk_column = x.pk_column 
    WHERE x.pk_column IS NULL 
GO 
 
SELECT * FROM dbo.Dupes 
GO 
 
DROP TABLE dbo.Dupes 
GO
 
There are also some platform-specific ways to handle this (for example, if you don't have a primary key): 
 
SQL Server 
 
There is a straightforward method outlined in KB #139444
 
Another approach might be to create a second table with a primary key or unique constraint on that column, and insert into the new table all the rows from the old table. For each unique value, the first insert will succeed but all subsequents will fail due to violation of key or unique constraint. Once you're done, you can rename the old table to something else, and rename the new table with the old name. 
 
Finally, see this article for some other ideas. 
 
Access 
 
There is a really ugly kludge in KB #209573 -- requiring manual labor. 
 

Once you've removed the duplicate rows, please do yourself a favor and make it so that it doesn't happen again. You can see some techniques for preventing duplicates in Article #2509.

Related Articles

How do I build a query with optional parameters?
How do I calculate the median in a table?
How do I create a store locator feature?
How do I deal with MEMO, TEXT, HYPERLINK, and CURRENCY columns?
How do I deal with multiple resultsets from a stored procedure?
How do I debug my SQL statements?
How do I determine if a column exists in a given table?
How do I enable or disable connection pooling?
How do I enumerate through the DSNs on a machine?
How do I find a stored procedure containing <text>?
How do I get a list of Access tables and their row counts?
How do I get the latest version of the JET OLEDB drivers?
How do I handle alphabetic paging?
How do I handle BIT / BOOLEAN columns?
How do I handle error checking in a stored procedure?
How do I ignore common words in a search?
How do I page through a recordset?
How do I present one-to-many relationships in my ASP page?
How do I prevent duplicates in a table?
How do I prevent my ASP pages from waiting for backend activity?
How do I prevent NULLs in my database from mucking up my HTML?
How do I protect my Access database (MDB file)?
How do I protect my stored procedure code?
How do I protect myself against the W32.Slammer worm?
How do I rename a column?
How do I retrieve a random record?
How do I return row numbers with my query?
How do I send a database query to a text file?
How do I simulate an array inside a stored procedure?
How do I solve 'Could not find installable ISAM' errors?
How do I solve 'Operation must use an updateable query' errors?
How do I temporarily disable a trigger?
How do I use a SELECT list alias in the WHERE or GROUP BY clause?
How do I use a variable in an ORDER BY clause?
Should I index my database table(s), and if so, how?
Should I store images in the database or the filesystem?
Should I use a #temp table or a @table variable?
Should I use a view, a stored procedure, or a user-defined function?
Should I use recordset iteration, or GetRows(), or GetString()?
What are all these dt_ stored procedures, and can I remove them?
What are the limitations of MS Access?
What are the limitations of MSDE?
What are the valid styles for converting datetime to string?
What datatype should I use for my character-based database columns?
What datatype should I use for numeric columns?
What does "ambiguous column name" mean?
What is this 'Multiple-step OLE DB' error?
What is wrong with 'SELECT *'?
What naming convention should I use in my database?
What should I choose for my primary key?
What should my connection string look like?
When should I use CreateObject to create my recordset objects?
Where can I get this 'Books Online' documentation?
Where do I get MSDE?
Which database platform should I use for my ASP application?
Which tool should I use: Enterprise Manager or Query Analyzer?
Why are there gaps in my IDENTITY / AUTOINCREMENT column?
Why can I not 'open a database created with a previous version...'?
Why can't I access a database or text file on another server?
Why can't I use the TOP keyword?
Why do I get 'Argument data type text is invalid for argument [...]'?
Why do I get 'Not enough space on temporary disk' errors?
Why does ASP give me ActiveX errors when connecting to a database?
Should I use COALESCE() or ISNULL()?
Where can I get basic info about using stored procedures?

 

 


Created: 12/16/2002 | Last Updated: 9/12/2005 | broken links | helpful | not helpful | statistics
© Copyright 2006, UBR, Inc. All Rights Reserved. (162)

 

Copyright 1999-2006, All rights reserved.
Finding content
Finding content.  An error has occured...