Re: Duplicate rows
| From: | tg-php at gryffyndevelopment dot com | Date: | Wed, 01 Mar 2006 18:17:10 +0000 |
| Subject: | Re: Duplicate rows | ||
| Groups: | php.db | ||
| Request: | Send a blank email to php-db+get-41358@lists.php.net to get a copy of this message | ||
Depends on how you determine if something's a duplicate or not. For example, if it's just
one column that can be used, you can do something like this:
select ItemName, count(ItemName) from ItemListTable group by ItemName
having count(ItemName) > 1
That'll show you if "ItemName" is repeated. Then you can go back through and just
search for "ItemName" and remove the ones you don't want.
You can do pretty much the same thing as above but CONCATenating multple columns if that's what
you need to do to determine uniqueness.
I know you're dealing with 15k rows still, so you probably want something a little more
automated.
Without more info though, it's hard to say exactly what can be done.
Hope that helps a little bit.
-TG
= = = Original message = = =
My dear beloved friends,
I have a catalog of products that a product provider gave, sadly for me, in
this CSV file there are many duplicated rows.
I edited the file in my Linux system with the "uniq -u" command, and it
worked somewhat fine, it eliminated some duplicated rows, originally the
file had 24K rows, and now it has been reduced to 15k rows.
Anyhow, there are still duplicated rows, and since this is a catalog, it
should not have duplicated rows!!!
Now the catalog has been has been loaded into the DB.
How can I continue eliminating duplicated rows?
As far as I remember the is a sentence in SQL to only show ONE row of
duplicated rows, maybe if I do a select using this sentence and then put
this new recordset in another table, it will work!!
Any ideas?
-----------------------
Miguel Guirao Aguilera
Logistica R8 TELCEL
Tel. (999) 960.7994
Este mensaje es exclusivamente para el uso de la persona o entidad a quien esta dirigido; contiene
informacion estrictamente confidencial y legalmente protegida, cuya divulgacion es sancionada por la
ley. Si el lector de este mensaje no es a quien esta dirigido, ni se trata del empleado o agente
responsable de esta informacion, se le notifica por medio del presente, que su reproduccion y
distribucion, esta estrictamente prohibida. Si Usted recibio este comunicado por error, favor de
notificarlo inmediatamente al remitente y destruir el mensaje. Todas las opiniones contenidas en
este mail son propias del autor del mensaje y no necesariamente coinciden con las de Radiomovil
Dipsa, S.A. de C.V. o alguna de sus empresas controladas, controladoras, afiliadas y subsidiarias.
Este mensaje intencionalmente no contiene acentos.
This message is for the sole use of the person or entity to whom it is being sent. Therefore, it
contains strictly confidential and legally protected material whose disclosure is subject to penalty
by law. If the person reading this message is not the one to whom it is being sent and/or is not an
employee or the responsible agent for this information, this person is herein notified that any
unauthorized dissemination, distribution or copying of the materials included in this facsimile is
strictly prohibited. If you received this document by mistake please notify immediately to the
subscriber and destroy the message. Any opinions contained in this e-mail are those of the author of
the message and do not necessarily coincide with those of Radiomovil Dipsa, S.A. de C.V. or any of
its control, controlled, affiliates and subsidiaries companies. No part of this message or
attachments may be used or reproduced in any manner whatsoever.
___________________________________________________________
Sent by ePrompter, the premier email notification software.
Free download at http://www.ePrompter.com.