PEPPLER.ORG
Michael Peppler
Sybase Consulting
Menu
Home
Sybase on Linux
Install Guide for Sybase on Linux
General Sybase Resources
General Perl Resources
Freeware
Sybperl
Sybase::Simple
DBD::Sybase
BCP Tool
Bug Tracker
Mailing List Archive
Downloads Directory
FAQs
Sybase on Linux FAQ
Sybperl FAQ
Personal
Michael Peppler's resume

sybperl-l Archive

Up    Prev    Next    

From: Larry Bertolini <bertolini dot 1 at osu dot edu>
Subject: Mix BCP & CT-Lib? Write to bcp via fifo?
Date: Oct 19 2001 7:05PM

I have a couple questions about Sybperl and bcp:

1. can I use the Sybase::BCP functions
    from within a Sybase::CTlib program?

2. Right now, I'm trying to use perl to pipe
    data to the bcp process.  Pseudo-code is:

    open input file
    start a background bcp process that reads from a fifo file
    open fifo file for output
    loop:
       read record
       munge data
       write to fifo
    endloop
    close fifo file
    close input file

I'm having weird problems with this approach;
bcp dies with the message:

    Unexpected EOF encountered in BCP data-file.
    bcp copy in partially failed

Sometimes the program works (processes 500,000 records),
but usually it fails; the number of records that it
processes is fairly random (a couple hundred, a couple
thousand, or several thousand).  I suspect that maybe it's
related to perl's file buffering, but I've played with
the "$|" variable, and it seems to have no effect (that is
to say, it doesn't solve this problem).

If I write the data to an actual filesystem file, and then
invoke bcp, it works fine.  I'd just like to eliminate that
overhead.

(Sybase 11.9.2, perl 5.005_03, sybperl 2.10_02, Solaris 2.6.)

Actual code below:

#
# Open the input file
#
$INFILE="myfile.txt";
print "NOTE: myfile is $INFILE\n";
open(INFILE,$INFILE) || die "May Day! May Day! Open INFILE $INFILE failed\n";

#
# Define the BCPFIFO file, and the bcp command
#
$BCPFIFO="mytable.fifo";
$bcpcmd="bcp $SYBDBNM.dbo.mytable_W in $BCPFIFO -U$USER -S$DSQUERY -c -t~ 
-r~\\\\n  < $pwdfile \&";
print "NOTE: bcp command: $bcpcmd\n";

#
# Start the bcp process
#
$rc = system($bcpcmd);
if ($rc > 0) {die "May Day! May Day! bcp returned $rc\n";}

#
# Open the BCPFIFO
#
open(BCPFIFO, "> $BCPFIFO") || die "May Day! May Day! Open BCPFIFO $BCPFIFO 
failed\n";
#
# Read the input file, reformat the data, and write it to
# the fifo file that bcp is reading.
#
$recs_in = 0;
#select BCPFIFO;
#$| = 1;
while () {
   ($col1,$col2,$col3,$col4) = split /:/, $_, 4;
   $namen =~ tr/A-Z/a-z/;
   print BCPFIFO "$col1~$col2~$col3~$col4~\n";
   $recs_in ++;
}

close(BCPFIFO);
close(INFILE);
print "NOTE: $recs_in records read from $INFILE\n";