PEPPLER.ORG
Michael Peppler
Sybase Consulting
Menu
Home
Sybase on Linux
Install Guide for Sybase on Linux
General Sybase Resources
General Perl Resources
Freeware
Sybperl
Sybase::Simple
DBD::Sybase
BCP Tool
Bug Tracker
Mailing List Archive
Downloads Directory
FAQs
Sybase on Linux FAQ
Sybperl FAQ
Personal
Michael Peppler's resume

sybperl-l Archive

Up    Prev    Next    

From: Michael Peppler <mpeppler at peppler dot org>
Subject: RE: dbi and bulk insert
Date: Jun 27 2001 2:07PM

Peter.Collard@barclayscapital.com writes:
 > The main argument for implementing a bulk insert module is that bcp only
 > takes data from files.
 > 
 > With large amounts of data (3gb) needing transposition before bcp'ing, I
 > found that the I/O costs were quite high as the data had to be 1)read
 > 2)written 3)read and 4)sent to ASE, which even with careful disk placement
 > took some time (20 mins for steps 1& 2) and added complexity to the error
 > handling,control and reporting.

Agreed.

But on most systems you can used a named pipe to avoid additional file
system space, and also to avoid a lot of IO (the IO becomes Unix type
sockets instead of file IO).

Michael

 > -----Original Message-----
 > From: Michael Peppler [mailto:mpeppler@peppler.org]
 > Sent: 25 June 2001 17:55
 > To: SybPerl Discussion List
 > Subject: Re: dbi and bulk insert
 > 
 > 
 > Stephen.Sprague@morganstanley.com writes:
 >  > Do you know of any plans to augment DBI with a 'bulk insert' mechansism?
 >  > I would think this would fall under this module, no?
 > 
 > I've not heard of any plans for any bulk insert mechanism in DBI. I
 > could add something for the DBD::Sybase module specifically (i.e. in a
 > non-portable way), but I've found that doing BCP style operations in
 > perl is mostly a waste of time, and there are usually ways of
 > achieving the same results by combining perl and Sybase's bcp utility.
 > 
 > However, my opinion could be changed by a good argument :-)
 > 
 > Michael
 > -- 
 > Michael Peppler - Data Migrations Inc. - mpeppler@peppler.org
 > http://www.mbay.net/~mpeppler - mpeppler@mbay.net
 > International Sybase User Group - http://www.isug.com
 > *Looking for new project to tackle starting 8/1/01*
 > 
 > 
 > ------------------------------------------------------------------------
 > For more information about Barclays Capital, please
 > visit our web site at http://www.barcap.com.
 > 
 > 
 > Internet communications are not secure and therefore the Barclays 
 > Group does not accept legal responsibility for the contents of this 
 > message.  Although the Barclays Group operates anti-virus programmes, 
 > it does not accept responsibility for any damage whatsoever that is 
 > caused by viruses being passed.  Any views or opinions presented are 
 > solely those of the author and do not necessarily represent those of the 
 > Barclays Group.  Replies to this email may be monitored by the Barclays 
 > Group for operational or business reasons.
 > 
 > ------------------------------------------------------------------------
 > 

-- 
Michael Peppler - Data Migrations Inc. - mpeppler@peppler.org
http://www.mbay.net/~mpeppler - mpeppler@mbay.net
International Sybase User Group - http://www.isug.com