|
|
sybperl-l Archive
Up Prev Next
From: JMiller at pressherald dot com
Subject: RE: memory problems - SOLVED
Date: May 6 2002 4:50PM
thank you all for your responses -
I upgraded perl to 5.6.1 and sybperl to 2.14.
I had tried to upgrade in the past but could never
get it to compile - a newer gcc solved that problem.
my script now takes up about 15 MB - instead of 200+ !!! :)
thanks again!
JJM
> -----Original Message-----
> From: Michael Peppler [SMTP:mpeppler@peppler.org]
> Sent: Saturday, May 04, 2002 6:19 PM
> To: SybPerl Discussion List
> Subject: RE: memory problems
>
> On Fri, 2002-05-03 at 08:38, JMiller@pressherald.com wrote:
> >
> > Hi Michael -
> > thanks for responding - we are running perl 5.003
>
> That's pretty old. If you get the chance - upgrade to 5.6.1 (and the
> latest sybperl).
>
> > %pgs = "";
> > %pages = "";
>
> As we mentioned elsewhere - this should be
>
> %pgs = ();
> %pages = ();
>
> However, below you use $pages as an array, not a hash, so you should
> really write:
>
> @pages = ();
>
> > $numresp= 0;
> > $dbh->ct_execute($query);
> > while ($dbh->ct_results($res_type) == CS_SUCCEED) {
> > next unless $dbh->ct_fetchable($res_type);
> > while ((%pgs) = $dbh->ct_fetch(CS_TRUE)) {
> > $pages[$numresp]{id} = $pgs{id};
>
> You're working too hard:
>
> push(@pages, {%pgs});
> }
> }
>
> $numresp = scalar(@pages);
>
> > now here is the meat of the script - it loops through %pages and gets
> text
> > data based on that
> > and then save the data to a file and then go back and get the next row
> from
> > the db.
> >
> > $numpages = @pages;
> > $loop = 0;
> > while ( $loop < $numpages ) {
> >
> > $dbh->ct_sql("set textsize 2048000");
> > $currentpage = $pages[$loop]{id};
> > $query1 = qq{
> > select id, name, text, layoutdt
> > from story, storypages where story.id = storypages.storyid
> > and storypages.pagesid = $currentpage
> > and name not like "ag.%" and name not like "%folio%"
> > };
> >
> > $dbh->ct_execute($query1);
> > while ($dbh->ct_results($res_type) == CS_SUCCEED) {
> > next unless $dbh->ct_fetchable($res_type);
> > while ((%story) = $dbh->ct_fetch(CS_TRUE)) {
> >
> > @spline = "";
>
> That should be @spline = ();
>
>
> > thanks for any help or suggestions you might have.
>
> Actually I strongly suggest that you run this script with the -w flag,
> and with
>
> use strict;
>
> at the top. This will force you to declare your variables as lexicals
> (my $foo), which in turn will tell perl to free them when they go out of
> scope. This is particularly important for the second part of the script
> where you fetch large chunks of data, but perl probably never realizes
> that you are done with them.
>
> Michael
>
>
> > > -----Original Message-----
> > > From: Michael Peppler [SMTP:mpeppler@peppler.org]
> > > Sent: Friday, May 03, 2002 10:37 AM
> > > To: SybPerl Discussion List
> > > Subject: Re: memory problems
> > >
> > > On Fri, 2002-05-03 at 06:16, JMiller@pressherald.com wrote:
> > > >
> > > > Hi,
> > > > I have a sybperl (ver 2.06) script that chews up gobs of memory
> > > > on my Solaris system. I am retrieving text fields from sybase.
> > > > I read each result into a hash and process it. my script will
> > > > reach 250+ MB plus before it runs out of memory. this was
> > > > exacerbated when I had to bump up the textsize on the query
> > > > to 2048000. I have tried to reinitialize the hash ( %hash = ""; )
> > > > after each fetch but it doesn't help.
> > >
> > > John Gilmore-Baldwin's comments are on the money:
> > >
> > > You empty a hash by doing:
> > >
> > > %hash = ();
> > >
> > > Reconnecting to Sybase won't help. Sybperl itself *shouldn't* be
> leaking
> > > (although 2.06 is pretty old, so it's not impossible that it has some
> > > problems in that respect).
> > >
> > > (as an aside, you disconnect from Sybase in a sybperl script either by
> > > calling $dbh->dbclose() [Sybase::DBlib] or by having the handle go out
> > > of scope/become undefined: $dbh = undef;)
> > >
> > > Is there any way for you to post a section of the script that
> > > illustrates the problem? There might be some issue with variables
> > > getting set but never getting freed due to the way they are used.
> > >
> > > Also I'm guessing that you have a fairly old version of perl, and that
> > > may also contribute to the problem.
> > >
> > > Michael
> > > --
> > > Michael Peppler Data Migrations, Inc.
> > > mpeppler@peppler.org *or* mpeppler@mbay.net
> > > http://www.mbay.net/~mpeppler
> > > International Sybase User Group: http://www.isug.com
> >
> --
> Michael Peppler Data Migrations, Inc.
> mpeppler@peppler.org *or* mpeppler@mbay.net
> http://www.mbay.net/~mpeppler
> International Sybase User Group: http://www.isug.com
|
