Memory leak in TSQLQuery (TSQLQuery.next)
Original Reporter info from Mantis: mith
-
Reporter name: Michał
Original Reporter info from Mantis: mith
- Reporter name: Michał
Description:
I'm trying to export data to a *.csv file using a TSQLQuery (connected to a postgres database). Everything works fine for a small results. When the result is bigger (for example 80k rows, each with 6 fields of varchar 255 - which isn't of course huge amount of data), application exhausts all possible memory (more than 1,5GB).
FIRST TEST (saving data to a file)
BEGIN CODE ----->
try
AssignFile(plik, Utf8ToAnsi(SaveDialog.FileName));
Rewrite(plik);
Form1.QueryReports.First;
Count := Form1.QueryReports.FieldCount - 1;
pg.Max := Form1.QueryReports.RowsAffected;
pg.Position := 0;
Form1.Datasource1.Enabled := False;
Form1.Datasource1.DataSet := nil;
//header
for i := 0 to Count do
begin
if i = Count then
Writeln(plik, '"' + Form1.QueryReports.FieldDefs[i].DisplayName + '"')
else
Write(plik, '"' + Form1.QueryReports.FieldDefs[i].DisplayName + '"' + ',');
end;
//content
while not Form1.QueryReports.EOF do
begin
for i := 0 to Count do
begin
if i = Count then
begin
tmp_string:= '"' + StringReplace(Form1.QueryReports.Fields[i].AsString, '"', '^', [rfIgnoreCase, rfReplaceAll]) + '"';
tmp_string:= StringReplace(tmp_string, #13, ' ', [rfIgnoreCase, rfReplaceAll]);
tmp_string:= StringReplace(tmp_string, #10, ' ', [rfIgnoreCase, rfReplaceAll]);
Writeln(plik, tmp_string);
end
else
begin
tmp_string:= '"' + StringReplace(Form1.QueryReports.Fields[i].AsString, '"', '^', [rfIgnoreCase, rfReplaceAll]) + '"' + ',';
tmp_string:= StringReplace(tmp_string, #13, ' ', [rfIgnoreCase, rfReplaceAll]);
tmp_string:= StringReplace(tmp_string, #10, ' ', [rfIgnoreCase, rfReplaceAll]);
Write(plik, tmp_string);
end;
end;
Form1.QueryReports.Next;
pg.Position := pg.Position + 1;
end;
<----- END CODE
SECOND TEST
I turned off writing to a file, keeping simply iterating through SQLQuery by using '.next').
I made a DumpHeap after each 10000 records. You can see results below.
### After first 10000 records: ###
Heap dump by heaptrc unit
1002361 memory blocks allocated : 886168757/890603048
944126 memory blocks freed : 452070306/456341968
58235 unfreed memory blocks : 434098451
True heap size : 448954368 (112 used in System startup)
True free heap : 10104368
Should be : 10966136
Call trace for block $087554F8 size 28
$004CABEA TFORMA_RAPORTY__MENUITEM14CLICK (comment: procedure code is above), line 200 of form_raporty.pas
$00601568 TMENUITEM__CLICK, line 83 of ./include/menuitem.inc
$00601B26 TMENUITEM__DOCLICKED, line 278 of ./include/menuitem.inc
$0040C0F6
$75877694
$75878BAA
$75878468
$75879C40
Call trace for block $087A07B8 size 14
$004CABBD
$00601568
$00601B26
$0040C0F6
$75877694
$75878BAA
$75878468
$75879C40
Call trace for block $4140A270 size 42764
[...]
### On 20000 records: ###
Heap dump by heaptrc unit
1828722 memory blocks allocated : 1735404672/1743811016
1760487 memory blocks freed : 873666230/881869944
68235 unfreed memory blocks : 861738442
True heap size : 885948416 (112 used in System startup)
True free heap : 18618432
Should be : 19640192
[...]
### On 30000 records: ###
Heap dump by heaptrc unit
2660904 memory blocks allocated : 2585303174/2597715304
2582669 memory blocks freed : 1295924723/1308094224
78235 unfreed memory blocks : 1289378451
True heap size : 1322942464 (112 used in System startup)
True free heap : 27132464
Should be : 28314232
[...]
After a while: RunError(203), EOutOfMemory
Steps to reproduce:
- Connect throught TPQConnection to PostgreSQL database
- Use the code above to retrieve data
Additional information:
I made tests on Windows 7 64b, Win 8.1.
The application was compiled on Lazarus (stable) 1.0.10 and 1.0.14 (stable).
Mantis conversion info:
- Mantis ID: 25477
- OS: Windows
- OS Build: 7 64b/8.1 64b
- Build: 43446
- Platform: Intel
- Version: 1.0.14
- Target version: 2.7.1