Does anyone know of a tool for working with CSV files and then importing them into MySQL?
I've got a flatfile database which has a series of fields that I need to import into MySQL.
Most of the fields in the orginal will map to new ones in the MySQL, while some on both sides will be ignored
There's about a thousand entries in the database, so I'd rather not do it manually, as it's not simply a matter of copying and inserting - I will also have to manually change one element associated with each entry....
Any suggestions would be welcome if they run on windows or linux (I don't have OSX yet)
CSV / MySQL Tools?
Categories:
9 Comments
Rate This Blog
Categories
- Cinema (103)
- Domains & Hosting (18)
- General (108)
- Harry Potter (17)
- Linguistics (28)
- Podcasting (21)
- Techie :: Techno :: (627)
- Thoughts :: Pensieri :: (238)
- Work :: Lavoro :: (96)
- accessibility (16)
- adsense (2)
- adwords (14)
- amazon (8)
- apache (4)
- apple (27)
- audio (2)
- barcamp (6)
- blacknight (47)
- blogging (55)
- books (12)
- cool (77)
- copyright (4)
- data retention (1)
- design (12)
- ebay (12)
- ecommerce (55)
- events (10)
- gaming (2)
- google (26)
- humour (74)
- iedr (8)
- iia (25)
- interviews (3)
- jaiku (3)
- linksys (4)
- marketing (26)
- microsoft (9)
- movabletype (11)
- paypal (3)
- philately (4)
- phones (2)
- photography (24)
- rant (8)
- ripe (6)
- security (7)
- seo (20)
- skycon (4)
- technoratifeedback (3)
- travel (12)
- tv (1)
- twitter (7)
- weird (4)
- wordpress (7)
- yahoo (2)
Monthly Archives
- October 2008 (2)
- September 2008 (14)
- August 2008 (29)
- July 2008 (24)
- June 2008 (19)
- May 2008 (36)
- April 2008 (43)
- March 2008 (41)
- February 2008 (51)
- January 2008 (35)
- December 2007 (22)
- November 2007 (21)
- October 2007 (42)
- September 2007 (71)
- August 2007 (61)
- July 2007 (39)
- June 2007 (37)
- May 2007 (41)
- April 2007 (39)
- March 2007 (63)
- February 2007 (47)
- January 2007 (82)
- December 2006 (25)
- November 2006 (34)
- October 2006 (28)
- September 2006 (36)
- August 2006 (40)
- July 2006 (31)
- June 2006 (45)
- May 2006 (47)
- April 2006 (40)
- March 2006 (39)
- February 2006 (54)
- January 2006 (63)
- December 2005 (38)
- November 2005 (61)
- October 2005 (68)
- September 2005 (46)
- August 2005 (54)
- July 2005 (85)
- June 2005 (25)
- May 2005 (20)
- April 2005 (18)
- March 2005 (19)
- February 2005 (13)
- January 2005 (15)
- December 2004 (9)
- November 2004 (14)
- October 2004 (13)
- September 2004 (11)
- August 2004 (23)
- July 2004 (7)
- June 2004 (1)
- May 2004 (1)
- April 2004 (2)
- March 2004 (5)
- February 2004 (2)
- January 2004 (10)
- December 2003 (8)
- November 2003 (4)
OpenID accepted here
Learn more about OpenID
Search
About this Entry
This page contains a single entry by Michele Neylon published on January 3, 2007 1:49 PM.
.eu pricing 2007 was the previous entry in this blog.
Google Provides Phone Support? Adwords UK & Ireland Telephone Numbers! is the next entry in this blog.
Find recent content on the main index or look in the archives to find all content.

sqlyog has CSV importing as far as I remember (I use the command line now rather than a GUI.)
A low tech way though is to open the CSV in Excel or some spreadsheet app. You then put your "INSERT INTO abc" command in the first column and then break up the rest of the columns with new columns having "','" in them. Remember to include brackets where necessary. Do it for one line and then use Excel's repeat-copy feature to copy it all the way down. I then save it as a text file and run it through MySQL. It can be a bit tricky getting the quotes right but it works.
Try the LOAD DATA INFILE command http://dev.mysql.com/doc/refman/5.0/en/load-data.html
Or if you have more time on your hands get to know Pentaho's Kettle tool http://kettle.pentaho.org/
It's a powerful open source GUI fronted ETL tool, will load almost any format into any database.
Tom
I generally use a mixture of Openoffice and bash tools when I want to work with csv files. Awk is wonderful for reaaranging columns!
I'll bring in a copy of Perl For Dummies to you ;)
Niall
Does that come in a "retards" edition? :)
Donncha - I've been trying something similar but ended up losing an entire field...
Tom - I couldn't find any sign of kettle on their site, but thanks anyway
Paul - Too much like hard work :) It would have been as easy to copy and paste into the new DB
The might have it in a "Cark" edition all right :)
We managed to get a list of 50,000 feeds into a database using the Excel/OpenOffice method. Took about 10 minutes to setup.
You could always post it on Mechanical Turk or RentACoder and pay 1c a line ;)
I had to do something similar a few months back and ended up using the LOAD DATA INFILE command as suggested by Tom above. I had a CSV file with around 1000 entries and around 10 fields per entry.
The first thing I did was to create a new table describing the various fields I planned to import. This was the trickiest part.
MySQL> create table players (name varchar(20), club varchar(20), value double);
Them, I imported from the file as follows:
MySQL> load data local infile "abc.csv" into table players fields terminated by ',';
The key things here are the use of the "local" keyword before the infile and the "fields terminated by ','
Best of luck
I found myself in the same situation and hacked a small utility to import CSV files to MySQL.
It recognizes fieldtypes and -lengths automatically on import.
Drawback, because my English is really bad, i have coded it in German language.
Youll find the tool here:
http://mcyra.ath.cx/csvutil/index.html
And can download it here:
http://mcyra.ath.cx/dl/download.php?dldfile=3