Re: Duplicate rows

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Depends on how you determine if something's a duplicate or not.  For example, if it's just one column that can be used, you can do something like this:

select ItemName, count(ItemName) from ItemListTable group by ItemName
having count(ItemName) > 1


That'll show you if "ItemName" is repeated.   Then you can go back through and just search for "ItemName" and remove the ones you don't want.

You can do pretty much the same thing as above but CONCATenating multple columns if that's what you need to do to determine uniqueness.

I know you're dealing with 15k rows still, so you probably want something a little more automated.

Without more info though, it's hard to say exactly what can be done.

Hope that helps a little bit.

-TG

= = = Original message = = =

My dear beloved friends,

I have a catalog of products that a product provider gave, sadly for me, in
this CSV file there are many duplicated rows.
I edited the file in my Linux system with the "uniq -u" command, and it
worked somewhat fine, it eliminated some duplicated rows, originally the
file had 24K rows, and now it has been reduced to 15k rows.

Anyhow, there are still duplicated rows, and since this is a catalog, it
should not have duplicated rows!!!
Now the catalog has been has been loaded into the DB.

How can I continue eliminating duplicated rows?
As far as I remember the is a sentence in SQL to only show ONE row of
duplicated rows, maybe if I do a select using this sentence and then put
this new recordset in another table, it will work!!

Any ideas?

-----------------------
Miguel Guirao Aguilera
Logistica R8 TELCEL
Tel. (999) 960.7994


Este mensaje es exclusivamente para el uso de la persona o entidad a quien esta dirigido; contiene informacion estrictamente confidencial y legalmente protegida, cuya divulgacion es sancionada por la ley. Si el lector de este mensaje no es a quien esta dirigido, ni se trata del empleado o agente responsable de esta informacion, se le notifica por medio del presente, que su reproduccion y distribucion, esta estrictamente prohibida. Si Usted recibio este comunicado por error, favor de notificarlo inmediatamente al remitente y destruir el mensaje. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Radiomovil Dipsa, S.A. de C.V. o alguna de sus empresas controladas, controladoras, afiliadas y subsidiarias. Este mensaje intencionalmente no contiene acentos.

This message is for the sole use of the person or entity to whom it is being sent.  Therefore, it contains strictly confidential and legally protected material whose disclosure is subject to penalty by law.  If the person reading this message is not the one to whom it is being sent and/or is not an employee or the responsible agent for this information, this person is herein notified that any unauthorized dissemination, distribution or copying of the materials included in this facsimile is strictly prohibited.  If you received this document by mistake please notify  immediately to the subscriber and destroy the message. Any opinions contained in this e-mail are those of the author of the message and do not necessarily coincide with those of Radiomovil Dipsa, S.A. de C.V. or any of its control, controlled, affiliates and subsidiaries companies. No part of this message or attachments may be used or reproduced in any manner whatsoever.


___________________________________________________________
Sent by ePrompter, the premier email notification software.
Free download at http://www.ePrompter.com.

-- 
PHP Database Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [PHP Users]     [Postgresql Discussion]     [Kernel Newbies]     [Postgresql]     [Yosemite News]

  Powered by Linux