PEPPLER.ORG
Michael Peppler
Sybase Consulting
Menu
Home
Sybase on Linux
Install Guide for Sybase on Linux
General Sybase Resources
General Perl Resources
Freeware
Sybperl
Sybase::Simple
DBD::Sybase
BCP Tool
Bug Tracker
Mailing List Archive
Downloads Directory
FAQs
Sybase on Linux FAQ
Sybperl FAQ
Personal
Michael Peppler's resume

sybperl-l Archive

Up    Prev    Next    

From: Stephen dot Sprague at morganstanley dot com
Subject: Re: blk_init and image columns
Date: Sep 5 2002 7:52PM

Hi Michael-
Ahh... found the SEGV problem.  on the blk_init I had $has_identity set to
true and I only had one column - a image type.  bonehead move on my part on
setting up the test case.

After creating a table with an identity column and an image  column  and
setting CS_OPT_TEXTSIZE to 1_000_000 things appear to be working -  damn
slow - but they're working.

I'll increase packetsize and hopefully see some performance gain.

Thanks for sticking with me on this. I owe you one!

Cheers,
Steve



On 5 Sep 2002 @ 3:36am, an entity claiming to be Michael Peppler scribbled:

mpeppl :Have you tried setting CS_OPT_TEXTSIZE to a smaller value, to see what
mpeppl :happens?
mpeppl :
mpeppl :I just looked at the code, and the place where I would *guess* that it
mpeppl :dies is in the following loop:
mpeppl :
mpeppl :    for(i = 0; i < num_cols; ++i) {
mpeppl :        ret = blk_describe(info->bcp_desc, i+1, &info->datafmt[i]);
mpeppl :        if(ret != CS_SUCCEED)
mpeppl :            goto FAIL;
mpeppl :    }
mpeppl :
mpeppl :I haven't tried to reproduce it, but the CTlib/t/xblk.t test script
mpeppl :*does* insert data to an image column, so it *should* work.
mpeppl :
mpeppl :My laptop runs linux (with sybase on it, of course :-), so I may try to
mpeppl :reproduce it here.
mpeppl :
mpeppl :Michael
mpeppl :
mpeppl :On Wed, 2002-09-04 at 14:10, Stephen.Sprague@morganstanley.com wrote:
mpeppl :> Thanks Michael for replying. Revisiting, as  things  would  have  it,  I
mpeppl :> still am croaking with a SEGV.  Any further pointers most welcome.
mpeppl :>
mpeppl :> If it helps any here's my perl code fragment:
mpeppl :>
mpeppl :>
mpeppl :> #create handle
mpeppl :>   my ($cs, $packetsize) = ('iso_1', 8192);
mpeppl :>   my $props = { CON_PROPS => { CS_BULK_LOGIN => CS_TRUE,
mpeppl :>                                CS_SYB_CHARSE=> $cs,
mpeppl :>                                CS_PACKETSIZE => $packetsize } };
mpeppl :>   my $dbh = Sybase::CTlib->new( $u, $p, $s, $programName, $props );
mpeppl :>
mpeppl :>
mpeppl :> #set textsize to 1gb
mpeppl :>   $dbh->ct_options(CS_SET, CS_OPT_TEXTSIZE, 1_000_000_000, CS_INT_TYPE);
mpeppl :>
mpeppl :>
mpeppl :> #init blk copy
mpeppl :>    $dbh->blk_init( $tab, $num_cols, $has_identity, $id_column )
mpeppl :>
mpeppl :> Signal SEGV
mpeppl :>
mpeppl :>
mpeppl :> The table on the 11.9 server looks like this I'm trying to bcp into:
mpeppl :>
mpeppl :>  Column_name     Type            Length Prec Scale Nulls Default_name    Rule_name       Identity
mpeppl :>  --------------- --------------- ------ ---- ----- ----- --------------- --------------- --------
mpeppl :>  i               image               16 NULL  NULL     0 NULL            NULL                   0
mpeppl :>
mpeppl :>
mpeppl :>
mpeppl :>
mpeppl :>
mpeppl :>
mpeppl :> On 4 Sep 2002 @ 8:05am, an entity claiming to be Michael Peppler scribbled:
mpeppl :>
mpeppl :> mpeppl :[I'm traveling - currently in England - so email access is spotty... ]
mpeppl :> mpeppl :
mpeppl :> mpeppl :CS_PACKETSIZE must be set in a power of 2, starting at 512 (so 512,
mpeppl :> mpeppl :1024, 2048, etc). I think there's a max, and of course the server has to
mpeppl :> mpeppl :be configured to handle larger packet sizes (at least up to 12.0 - 12.5
mpeppl :> mpeppl :*may* handle this dynamically).
mpeppl :> mpeppl :
mpeppl :> mpeppl :However, I'm pretty sure that the packet size has nothing to do with the
mpeppl :> mpeppl :SEGV that you are seeing. I suspect instead that it may be linked with
mpeppl :> mpeppl :the TEXTSIZE value. You can set this via SET TEXTSIZE xxxx or via a
mpeppl :> mpeppl :ct_option() call.
mpeppl :> mpeppl :
mpeppl :> mpeppl :Michael
mpeppl :> mpeppl :
mpeppl :> mpeppl :
mpeppl :> mpeppl :On Fri, 2002-08-30 at 13:39, Stephen.Sprague@morganstanley.com wrote:
mpeppl :> mpeppl :> sorry. last email for the weekend.  I set CS_PACKETSIZE => 9000 via
mpeppl :> mpeppl :> CON_PROPS and I still SEGV'ed on the blk_init.
mpeppl :> mpeppl :>
mpeppl :> mpeppl :> Time to set this thing down and grab a beer.
mpeppl :> mpeppl :>
mpeppl :> mpeppl :> Cheers,
mpeppl :> mpeppl :> Steve
mpeppl :> mpeppl :>
mpeppl :> mpeppl :>
mpeppl :> mpeppl :>
mpeppl :> mpeppl :> On Fri, 30 Aug 2002 @ 4:23pm, an entity claiming to be Stephen.Sprague@morg...:
mpeppl :> mpeppl :>
mpeppl :> mpeppl :> Stephe :Ahhh. Digging further I'm pretty sure this can be solved  by  increasing
mpeppl :> mpeppl :> Stephe :the CS_PACKETSIZE connection property. Now if only I had sa priv to bump
mpeppl :> mpeppl :> Stephe :the max size up from 512 on the server...
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :Thanks,
mpeppl :> mpeppl :> Stephe :Steve
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :cf http://www.sybase.com/detail/1,6904,954,00.html
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :On most platforms, the packet size defaults to 512 bytes.  This  default
mpeppl :> mpeppl :> Stephe :is provided for backward compatibility. Larger packet sizes 2048 to 4096
mpeppl :> mpeppl :> Stephe :bytes, for example almost  always  yield  better  performance.  This  is
mpeppl :> mpeppl :> Stephe :generally true for applications that fetch rows with large columns, rows
mpeppl :> mpeppl :> Stephe :with lots of columns, or lots of rows with a few small  columns.  It  is
mpeppl :> mpeppl :> Stephe :especially true for applications that do bulk  copy  data  transfers  or
mpeppl :> mpeppl :> Stephe :that handle text and image data.
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :To increase a connection's packet size, set the CS_PACKETSIZE connection
mpeppl :> mpeppl :> Stephe :property, via ct_con_props, prior to opening the connection.
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :--------------------------------------------------------------------------------
mpeppl :> mpeppl :> Stephe :Note
mpeppl :> mpeppl :> Stephe :You may have to reconfigure  SQL  Server  to  accept  a  larger  maximum
mpeppl :> mpeppl :> Stephe :network packet size. See the SQL Server System Administration Guide  for
mpeppl :> mpeppl :> Stephe :instructions on reconfiguring the maximum network packet size.
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :On Fri, 30 Aug 2002 @ 3:34pm, an entity claiming to be Stephen.Sprague@morg...:
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :Stephe :hi folks. I'm using Michael's BLK routines to do bulk inserts. Should  I
mpeppl :> mpeppl :> Stephe :Stephe :be able to use blk_init on a table with an image column?
mpeppl :> mpeppl :> Stephe :Stephe :
mpeppl :> mpeppl :> Stephe :Stephe :Currently I'm getting a Signal SEGV.
mpeppl :> mpeppl :> Stephe :Stephe :
mpeppl :> mpeppl :> Stephe :Stephe :Here's the versions:
mpeppl :> mpeppl :> Stephe :Stephe :   Perl          v5.6
mpeppl :> mpeppl :> Stephe :Stephe :   Sybase::CTlib v2.14
mpeppl :> mpeppl :> Stephe :Stephe :   OpenClient    v12.0
mpeppl :> mpeppl :> Stephe :Stephe :
mpeppl :> mpeppl :> Stephe :Stephe :I couldn't find any doc to the contrary and I see Sybase's command  line
mpeppl :> mpeppl :> Stephe :Stephe :bcp references switch -T [image size] so I think it's possible. The question
mpeppl :> mpeppl :> Stephe :Stephe :is does the Perl API support something analogous?
mpeppl :> mpeppl :> Stephe :Stephe :
mpeppl :> mpeppl :> Stephe :Stephe :Any pointers welcome.
mpeppl :> mpeppl :> Stephe :Stephe :
mpeppl :> mpeppl :> Stephe :Stephe :Thanks,
mpeppl :> mpeppl :> Stephe :Stephe :Steve
mpeppl :> mpeppl :> Stephe :Stephe :
mpeppl :> mpeppl :> Stephe :Stephe :
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :> Stephe :
mpeppl :> mpeppl :>
mpeppl :> mpeppl :>
mpeppl :> mpeppl :
mpeppl :>
mpeppl :>
mpeppl :