|
|
sybperl-l Archive
Up Prev Next
From: "Sabherwal, Balvinder K dot " <BKSabherwal at lmus dot leggmason dot com>
Subject: RE: out of memory -- CTlib
Date: Feb 8 2001 4:00PM
You can open a file in write mode and put all your records in that file
using just one variable rather than using an array. once you have completed
fetching the records from the database, close the file u r writing to and
open it back in a read mode and get one record at a time from the file and
do the steps u want to.
HTH
Balvinder.
> Any sugestions or help would be welcome.
>
> I am using ct_lib to select large look-up tables from the data base for
> feed
> processing. I tend to assign all of the info in the data base into a hash
> keyed on a specific value in the database and then read the file line by
> line using the key as a quick lookup. What I am running into however is
> that
> if I try to read in more than 100,000 records or so I get an 'Out of
> Memory!' error. Is there a more efficient way to read in a large number
> of
> records into a hash table? Any help or suggestions would be most welcome.
>
> Thanks
> Mark
>
> sub get_records {
> my ($dbh) = @_;
> my ($y, @record, $restype, $sql);
>
> $sql = "SELECT a bunch of rows " .
> "FROM table ";
> $dbh->ct_execute($sql);
> while ($dbh->ct_results($restype) == CS_SUCCEED) {
> next unless $dbh->ct_fetchable($restype);
> while (@record = $dbh->ct_fetch){
> if ( !$record_list{$record[3]}) {
> $record_list{$record[3]} = [@record];
> ############### record[3] is the key value
> } else {
> print OUT
> "=\"$record[3]\",=\"$record[0]\",=\"Duplicate\"";
> }
> $y++;
> if (!($y % 10000) && ($y !=0)) {
> print "$y Records processed at " , `date`;
> }
> }
> }
> $dbh->ct_cancel(CS_CANCEL_ALL);
> print "read $y total records\n";
> }
>
IMPORTANT: The security of electronic mail sent through the Internet
is not guaranteed. Legg Mason therefore recommends that you do not
send confidential information to us via electronic mail, including social
security numbers, account numbers, and personal identification numbers.
Delivery, and timely delivery, of electronic mail is also not
guaranteed. Legg Mason therefore recommends that you do not send time-sensitive
or action-oriented messages to us via electronic mail, including
authorization to "buy" or "sell" a security or instructions to conduct any
other financial transaction. Such requests, orders or instructions will
not be processed until Legg Mason can confirm your instructions or
obtain appropriate written documentation where necessary.
|
