Michael Peppler
Sybase Consulting
Sybase on Linux
Install Guide for Sybase on Linux
General Sybase Resources
General Perl Resources
BCP Tool
Bug Tracker
Mailing List Archive
Downloads Directory
Sybase on Linux FAQ
Sybperl FAQ
Michael Peppler's resume

sybperl-l Archive

Up    Prev    Next    

From: Larry Bertolini <bertolini dot 1 at osu dot edu>
Subject: Mix BCP & CT-Lib? Write to bcp via fifo?
Date: Oct 19 2001 7:05PM

I have a couple questions about Sybperl and bcp:

1. can I use the Sybase::BCP functions
    from within a Sybase::CTlib program?

2. Right now, I'm trying to use perl to pipe
    data to the bcp process.  Pseudo-code is:

    open input file
    start a background bcp process that reads from a fifo file
    open fifo file for output
       read record
       munge data
       write to fifo
    close fifo file
    close input file

I'm having weird problems with this approach;
bcp dies with the message:

    Unexpected EOF encountered in BCP data-file.
    bcp copy in partially failed

Sometimes the program works (processes 500,000 records),
but usually it fails; the number of records that it
processes is fairly random (a couple hundred, a couple
thousand, or several thousand).  I suspect that maybe it's
related to perl's file buffering, but I've played with
the "$|" variable, and it seems to have no effect (that is
to say, it doesn't solve this problem).

If I write the data to an actual filesystem file, and then
invoke bcp, it works fine.  I'd just like to eliminate that

(Sybase 11.9.2, perl 5.005_03, sybperl 2.10_02, Solaris 2.6.)

Actual code below:

# Open the input file
print "NOTE: myfile is $INFILE\n";
open(INFILE,$INFILE) || die "May Day! May Day! Open INFILE $INFILE failed\n";

# Define the BCPFIFO file, and the bcp command
$bcpcmd="bcp $SYBDBNM.dbo.mytable_W in $BCPFIFO -U$USER -S$DSQUERY -c -t~ 
-r~\\\\n  < $pwdfile \&";
print "NOTE: bcp command: $bcpcmd\n";

# Start the bcp process
$rc = system($bcpcmd);
if ($rc > 0) {die "May Day! May Day! bcp returned $rc\n";}

# Open the BCPFIFO
open(BCPFIFO, "> $BCPFIFO") || die "May Day! May Day! Open BCPFIFO $BCPFIFO 
# Read the input file, reformat the data, and write it to
# the fifo file that bcp is reading.
$recs_in = 0;
#select BCPFIFO;
#$| = 1;
while () {
   ($col1,$col2,$col3,$col4) = split /:/, $_, 4;
   $namen =~ tr/A-Z/a-z/;
   print BCPFIFO "$col1~$col2~$col3~$col4~\n";
   $recs_in ++;

print "NOTE: $recs_in records read from $INFILE\n";