View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0033424 | FPC | Compiler | public | 2018-03-13 18:14 | 2020-10-20 16:53 |
| Reporter | John | Assigned To | Jonas Maebe | ||
| Priority | normal | Severity | minor | Reproducibility | sometimes |
| Status | resolved | Resolution | unable to reproduce | ||
| Platform | Intel | OS | Lubuntu | ||
| Product Version | 3.0.0 | ||||
| Summary | 0033424: possible problem with dynamic arrays are a lot of use | ||||
| Description | I think that there is a problem with dynamic arrays, but I’m not 100% sure. I’m getting exceptions/crashes when I call setlength to set the initial size of a dynamic array, or perhaps to change its size. Here are some details. I’m writing an image processing application. It takes scanned images and, in part, removes dirt/noise/crud from those images prior to OCR. The images are loaded from files as contiguous bitmaps (dynamic arrays of bytes) and converted to a series of lines. Hence, a page comprises a dynamic array of lines, each of which comprises a dynamic array of pixels (which, in my case, are bytes). I chose to organize the data in this way because the nature of the image processing that I was doing is somewhat non-linear and it is much easier to write code that addresses pixels by their X, Y indices, and it seems like it runs faster too. The algorithms seem to work well the vast majority of the time, but occasionally, when I call setlength, the application generates an exception. This can be after many pages and therefore very many calls to setlength, e.g., 500,000 (a number suspiciously close to 2 ^ 19). However, in an early incarnation of the software, I successfully loaded all of the pages (400+) into RAM prior to doing any image processing (an approach that I abandoned after I determined that it was slower than doing the image processing on a page by page basis, reading and writing the files before and after respectively), so I don’t believe that the problem is directly related to number of calls to setlength per se. For awhile, I wondered about whether there is enough “time” for the memory management algorithms to complete between calls to setlength, but after having introduced Application.ProcessMessages after calls to setlength without effect, I’m pretty sure that this is not the problem. | ||||
| Steps To Reproduce | run the software on many scanned pages | ||||
| Additional Information | Below, find some call stack traces that I’ve recorded. #0 SYSTEM_$$_SYSGETMEM_VAR$QWORD$$POINTER at :0 0000001 ?? at :0 0000002 ?? at :0 0000003 .Ld241 at :0 0000004 ?? at :0 0000005 ?? at :0 0000006 SYSTEM_$$_SYSGETMEM$QWORD$$POINTER at :0 0000007 ?? at :0 0000008 ?? at :0 0000009 ?? at :0 0000010 SYSTEM_$$_GETMEM$POINTER$QWORD at :0 0000011 ?? at :0 0000012 fpc_dynarray_setlength at :0 0000013 WRITE_TIFF_FILE({FILE_NAME = 0x7ffff7fca9d8 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups/cleaned/page-238.tif', FILE_SIZE = 3018112, WAS_ERROR = false, LAST_ERROR = 0x0, IMAGE_COUNT = 1, IMAGES = 0x7ffff7fd0530}) at sf_tiff_io.pas:819 0000014 WRITE_PAGE({FILE_NAME = 0x7ffff7feedd8 'page-238.tif', SAMPLES_PER_PIXEL = 1, BITS_PER_SAMPLE = 1, PAGE = 0x7fffd1793960}) at do_cleanup_scans.pas:260 0000015 CLEAN_BOOK at do_cleanup_scans.pas:379 0000016 DIRECTORYEDIT1CHANGE(0x7ffff7faaad0, 0x7ffff7fa15d0) at do_cleanup_scans.pas:456 0000017 EDITCHANGE(0x7ffff7fa15d0) at editbtn.pas:1715 0000018 INTERNALONEDITCHANGE(0x7ffff7fa15d0, 0x7ffff7fa1d30) at editbtn.pas:1128 0000019 CHANGE(0x7ffff7fa1d30) at include/customedit.inc:582 0000020 CHANGE(0x7ffff7fa1d30) at maskedit.pp:1592 0000021 TEXTCHANGED(0x7ffff7fa1d30) at include/customedit.inc:574 0000022 TEXTCHANGED(0x7ffff7fa1d30) at maskedit.pp:1577 0000023 CMTEXTCHANGED(0x7ffff7fa1d30, {MSG = 45074, UNUSEDMSG = 0, WPARAM = 0, LPARAM = 0, RESULT = 0}) at include/control.inc:1109 0000024 SYSTEM$_$TOBJECT_$__$$_DISPATCH$formal at :0 0000025 .Ld240 at :0 0000026 .Ld239 at :0 0000027 ?? at :0 0000028 ?? at :0 0000029 ?? at :0 0000030 ?? at :0 0000031 WSREGISTERCLASS(0x7fffffffd1c0) at include/speedbutton.inc:852 0000032 WNDPROC(0x7ffff7fa1d30, {MSG = 45074, UNUSEDMSG = 0, WPARAM = 0, LPARAM = 0, RESULT = 0}) at include/wincontrol.inc:5383 0000033 DELIVERMESSAGE(0x7ffff7fa1d30, 0) at lclmessageglue.pas:112 0000034 SETTEXT(0x7ffff7fa3370, 0x7ffff7fa1d30, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at gtk2wsstdctrls.pp:1113 0000035 WSSETTEXT(0x7ffff7fa1d30, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at include/wincontrol.inc:5388 0000036 REALSETTEXT(0x7ffff7fa1d30, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at include/wincontrol.inc:8231 0000037 REALSETTEXT(0x7ffff7fa1d30, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at include/customedit.inc:523 0000038 REALSETTEXT(0x7ffff7fa1d30, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at maskedit.pp:1076 0000039 SETTEXT(0x7ffff7fa1d30, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at include/control.inc:4782 0000040 SETTEXT(0x7ffff7fa15d0, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at editbtn.pas:1677 0000041 SETDIRECTORY(0x7ffff7fa15d0, 0x7ffff7fd0e38 '/home/jhb/Images/Scans/Motorola Zener Diode Manual 1980/backups') at editbtn.pas:2424 0000042 RUNDIALOG(0x7ffff7fa15d0) at editbtn.pas:2491 0000043 BUTTONCLICK(0x7ffff7fa15d0) at editbtn.pas:2453 0000044 INTERNALONBUTTONCLICK(0x7ffff7fa15d0, 0x7ffff7fa0010) at editbtn.pas:1122 0000045 CLICK(0x7ffff7fa0010) at include/control.inc:2736 0000046 CLICK(0x7ffff7fa0010) at include/speedbutton.inc:111 0000047 WMLBUTTONUP(0x7ffff7fa0010, {MSG = 514, KEYS = 0, XPOS = 18, YPOS = 13, POS = {X = 18, Y = 13}, DUMMY = 851986, RESULT = 0}) at include/speedbutton.inc:907 0000048 SYSTEM$_$TOBJECT_$__$$_DISPATCH$formal at :0 0000049 .Ld21 at :0 0000050 .Ld20 at :0 0000051 ?? at :0 0000052 ?? at :0 #53 ?? at :0 #54 ?? at :0 0000055 ?? at :0 0000056 ?? at :0 0000057 ?? at :0 0000058 ?? at :0 0000059 ?? at :0 0000060 WNDPROC(0x7ffff7fa0010, {MSG = 514, UNUSEDMSG = 0, WPARAM = 0, LPARAM = 851986, RESULT = 0}) at include/control.inc:2124 0000061 PERFORM(0x7ffff7fa0010, 514, 0, 851986) at include/control.inc:1453 0000062 ISCONTROLMOUSEMSG(0x7ffff7fa15d0, 0) at include/wincontrol.inc:4713 0000063 WNDPROC(0x7ffff7fa15d0, {MSG = 514, UNUSEDMSG = 32767, WPARAM = 0, LPARAM = 852667, RESULT = 0}) at include/wincontrol.inc:5348 0000064 DELIVERMESSAGE(0x7ffff7fa15d0, 0) at lclmessageglue.pas:112 0000065 DELIVERMESSAGE(0x7ffff7fa15d0, 0) at gtk2proc.inc:3628 0000066 DELIVERMOUSEUPMESSAGE(0xc47a20, 0xd500f0, 0x7ffff7fa15d0) at gtk2callback.inc:2361 0000067 GTKMOUSEBTNRELEASE(0xc47a20, 0xd500f0, 0x7ffff7fa15d0) at gtk2callback.inc:2436 0000068 ?? at :0 0000069 g_closure_invoke at :0 0000070 ?? at :0 0000071 g_signal_emit_valist at :0 0000072 g_signal_emit at :0 0000073 ?? at :0 0000074 gtk_propagate_event at :0 0000075 gtk_main_do_event at :0 0000076 ?? at :0 0000077 g_main_context_dispatch at :0 0000078 ?? at :0 0000079 g_main_context_iteration at :0 0000080 APPWAITMESSAGE(0x7ffff7f6ca50) at gtk2widgetset.inc:2404 0000081 IDLE(0x7ffff7f6c430, true) at include/application.inc:405 0000082 HANDLEMESSAGE(0x7ffff7f6c430) at include/application.inc:1261 0000083 RUNLOOP(0x7ffff7f6c430) at include/application.inc:1395 0000084 APPRUN(0x7ffff7f6ca50, {Proc = {procedure (POINTER)} 0x7fffffffe400, Self = 0x7ffff7f6c430}) at include/interfacebase.inc:54 0000085 RUN(0x7ffff7f6c430) at include/application.inc:1383 0000086 main at cleanup_scans.lpr:19 #0 SYSTEM_$$_SYSGETMEM_VAR$QWORD$$POINTER at :0 0000001 ?? at :0 0000002 ?? at :0 0000003 .Ld241 at :0 0000004 ?? at :0 0000005 ?? at :0 0000006 SYSTEM_$$_SYSGETMEM$QWORD$$POINTER at :0 0000007 ?? at :0 #0 SYSTEM_$$_TRY_CONCAT_FREE_CHUNK$PMEMCHUNK_VAR$$PMEMCHUNK_VAR at :0 0000001 ?? at :0 0000002 ?? at :0 0000003 U_$SYSTEM_$$_FREELISTS at :0 0000004 SYSTEM_$$_SYSFREEMEM_VAR$PFREELISTS$PMEMCHUNK_VAR$$QWORD at :0 0000005 .Ld241 at :0 0000006 ?? at :0 0000007 U_$SYSTEM_$$_FREELISTS at :0 0000008 ?? at :0 0000009 ?? at :0 0000010 SYSTEM_$$_SYSFREEMEM$POINTER$$QWORD at :0 0000011 ?? at :0 0000012 ?? at :0 0000013 ?? at :0 0000014 RTTI_$DO_CLEANUP_SCANS_$$_LINE at :0 0000015 ?? at :0 0000016 SYSTEM_$$_FREEMEM$POINTER$$QWORD at :0 0000017 ?? at :0 0000018 fpc_dynarray_clear at :0 0000019 fpc_finalize at :0 0000020 .Ld241 at :0 0000021 ?? at :0 0000022 ?? at :0 0000023 fpc_finalize_array at :0 0000024 ?? at :0 0000025 ?? at :0 0000026 ?? at :0 0000027 RTTI_$DO_CLEANUP_SCANS_$$_PAGE_BM at :0 0000028 ?? at :0 0000029 fpc_dynarray_clear at :0 0000030 fpc_dynarray_assign at :0 0000031 ?? at :0 0000032 ?? at :0 0000033 ?? at :0 0000034 fpc_copy at :0 0000035 ?? at :0 #0 SYSTEM_$$_SYSGETMEM_VAR$QWORD$$POINTER at :0 0000001 ?? at :0 0000002 ?? at :0 0000003 .Ld241 at :0 0000004 ?? at :0 0000005 ?? at :0 0000006 SYSTEM_$$_SYSGETMEM$QWORD$$POINTER at :0 0000007 ?? at :0 They seem to point to the problem being related to something happening in a FreePascal library or its interface to an OS library. I have this problem on two different (Intel based) computers, and, although the crash point varied between them to begin with, having removed calls to setlength wherever possible, they now seem to converge to the same page. One computer has 8GB of RAM; the other 16GB. In both cases when running the software, RAM use hovers around 1GB. It is clear that the page image being processed is NOT the problem. If I start the software on a different page, it will process the offending page without issues. I can provide source code and image files to help you demonstrate/find the problem. | ||||
| Tags | No tags attached. | ||||
| Fixed in Revision | |||||
| FPCOldBugId | |||||
| FPCTarget | |||||
| Attached Files |
| ||||
|
|
There is probably not much we can do about this: heavy setlength calls might cause memory fragmentation at a certain point. To avoid it: - reduce number of setlength calls, especially reallocating is a problem - use a different memory manager like cmem (this does not mean that cmem handles fragmentation better, it depends on the specific allocation pattern were it might be better or worse) |
|
|
Without a simplified program that demonstrates the problems, it will be hard to work on this. |
|
|
For the sake of getting some more data, I compiled and ran the identical software with Lazarus on a 32 bit version of Linux. Although it ran slowly on my relatively old Atom processor, the software completed without errors. In other words, the problem seems to be related to the 64-bit version. Below find a code snippet that is directly from the software that I'm writing. The failure tends to occur when running this function. line = array of byte; // corresponds to one scan line page_bm = array of line; // the scan lines that comprise the image of the page function expanded_binary_bit_map(input: TIFF_image): page_bm; // returns a binary bit map expanded to an 8 bit bitmap var sample, X_mod: byte; X, Y: word; index: longword; begin // allocate memory for the image data setlength(result, input.height); // convert from 1 bit per pixel bit map to 8 bits per pixel page index := 0; sample := input.image_data[index]; for Y := 0 to input.height - 1 do begin setlength(result[Y], input.width); for X := 0 to input.width - 1 do begin X_mod := X and 7; if bytebool(sample and ($80 >> X_mod)) then result[Y][X] := white else result[Y][X] := black; if (X_mod = 7) then begin inc(index); sample := input.image_data[index]; end; end; if (X_mod <> 7) then inc(index); end; end; I can send the complete project, including the image files, as a .zip file if that is of interest. I just didn't want to burden you with that as a start point. Please let me know how I might help. |
|
|
More feedback. I tried Florian’s suggestion about using cmem instead of the native memory manager. This induces failures on both 32 bit and 64 bit Lubuntu at exactly the same point in the code, and rather quickly – before having completed the entire process of reading, correcting, and writing a single page. Below are the stack traces. // // on 32 bit Lubuntu with cmem // #0 __kernel_vsyscall at :0 0000001 __GI_raise(6) at ../sysdeps/unix/sysv/linux/raise.c:54 0000002 __GI_abort at abort.c:89 0000003 __libc_message(2, 0xb74ace54 '*** Error in `%s'': %s: 0x%s ***'0000010) at ../sysdeps/posix/libc_fatal.c:175 0000004 malloc_printerr(<optimized out>, 0xb74acec4 'double free or corruption (!prev)', <optimized out>, 0xb74ff780 <main_arena>) at malloc.c:5006 0000005 _int_free(0xb74ff780 <main_arena>, <optimized out>, 0) at malloc.c:3867 0000006 CMEM_$$_CFREEMEM$POINTER$$LONGWORD(0xbfffe9c8, 0x2e0d88 <error: Cannot access memory at address 0x2e0d88>) at :2823 0000007 SYSTEM_$$_FREEMEM$POINTER$$LONGWORD at :0 0000008 fpc_dynarray_clear at :0 0000009 fpc_finalize at :0 0000010 ?? at :0 0000011 CLEAN_BOOK at do_cleanup_scans.pas:377 0000012 TFORM1__DIRECTORYEDIT1CHANGE(0x834fc64, 0x834c494) at do_cleanup_scans.pas:454 0000013 TCUSTOMEDITBUTTON__EDITCHANGE(0x834fc64) at editbtn.pas:1715 0000014 TCUSTOMEDITBUTTON__INTERNALONEDITCHANGE(0x8350a04, 0x834fc64) at editbtn.pas:1128 0000015 TCUSTOMEDIT__CHANGE(0x8350a04) at ./include/customedit.inc:582 0000016 TCUSTOMMASKEDIT__CHANGE(0x8350a04) at maskedit.pp:1592 0000017 TCUSTOMEDIT__TEXTCHANGED(0x8350a04) at ./include/customedit.inc:574 0000018 TCUSTOMMASKEDIT__TEXTCHANGED(0x8350a04) at maskedit.pp:1577 0000019 TCONTROL__CMTEXTCHANGED({MSG = 45074, WPARAM = 0, LPARAM = 0, RESULT = 0, WPARAMLO = 0, WPARAMHI = 0, LPARAMLO = 0, LPARAMHI = 0, RESULTLO = 0, RESULTHI = 0}, 0x8350a04) at ./include/control.inc:1109 0000020 SYSTEM$_$TOBJECT_$__$$_DISPATCH$formal at :0 // // on 64 bit Lubuntu with cmem // #0 __GI_raise(6, 6) at ../sysdeps/unix/sysv/linux/raise.c:54 0000001 __GI_abort at abort.c:89 0000002 __libc_message(2, 2, 0x7ffff5832ed8 '*** Error in `%s'': %s: 0x%s ***'0000010, 0x7ffff5832ed8 '*** Error in `%s'': %s: 0x%s ***'0000010) at ../sysdeps/posix/libc_fatal.c:175 0000003 malloc_printerr(<optimized out>, 0x7ffff5833008 'double free or corruption (!prev)', 3) at malloc.c:5006 0000004 _int_free(<optimized out>, <optimized out>, 0) at malloc.c:3867 0000005 __GI___libc_free(<optimized out>) at malloc.c:2968 0000006 CMEM_$$_CFREEMEM$POINTER$$QWORD at :0 0000007 ?? at :0 0000008 SYSTEM_$$_FREEMEM$POINTER$$QWORD at :0 0000009 ?? at :0 0000010 fpc_dynarray_clear at :0 0000011 fpc_finalize at :0 0000012 ?? at :0 0000013 ?? at :0 0000014 ?? at :0 0000015 WRITE_PAGE({FILE_NAME = 0xf7fde0 'page-001.tif', SAMPLES_PER_PIXEL = 1, BITS_PER_SAMPLE = 1, PAGE = 0x3f0df98}) at do_cleanup_scans.pas:227 0000016 CLEAN_BOOK at do_cleanup_scans.pas:377 0000017 DIRECTORYEDIT1CHANGE(0xccf008, 0xcd4698) at do_cleanup_scans.pas:454 0000018 EDITCHANGE(0xcd4698) at editbtn.pas:1715 0000019 INTERNALONEDITCHANGE(0xcd4698, 0xd0cb68) at editbtn.pas:1128 0000020 CHANGE(0xd0cb68) at include/customedit.inc:582 0000021 CHANGE(0xd0cb68) at maskedit.pp:1592 0000022 TEXTCHANGED(0xd0cb68) at include/customedit.inc:574 0000023 TEXTCHANGED(0xd0cb68) at maskedit.pp:1577 0000024 CMTEXTCHANGED(0xd0cb68, {MSG = 45074, UNUSEDMSG = 0, WPARAM = 0, LPARAM = 0, RESULT = 0}) at include/control.inc:1109 0000025 SYSTEM$_$TOBJECT_$__$$_DISPATCH$formal at :0 Here is the code that was being executed. function binary_bit_map(image: page_bm; bright: byte): byte_buffer; // takes a binary black & white page and returns a binary bitmap // if a pixel = bright, then the output is 1 var height, width, bytes_per_line, bm_size, X, Y, index: longword; begin height := length(image); width := length(image[0]); bytes_per_line := width div 8; if bytes_per_line * 8 <> width then inc(bytes_per_line); bm_size := height * bytes_per_line; setlength(result, bm_size); index := 0; result[index] := 0; // write out the image line by line for Y := 0 to height - 1 do for X := 0 to width - 1 do begin if image[Y][X] = bright then result[index] := result[index] or ($80 >> (X and 7)); if ((X and 7) = 7) or (X = width - 1) then begin inc(index); result[index] := 0; end; end; end; function write_page(input: page_type): TIFF_write_result; // writes the input page to a TIFF output file var file_record: TIFF_file_record; bitmap: byte_buffer; Y, line_length: longword; bits_per_sample: word_buffer; begin Form1.StatusBar1.SimpleText := input.file_name + ': writing'; Application.ProcessMessages; // give other system processes a chance to work // put everything into a TIFF_file_record format for the TIFF writer if input.bits_per_sample = 1 then bitmap := binary_bit_map(input.page, white) else // write the lines as a single bitmap begin line_length := length(input.page[0]); setlength(bitmap, length(input.page) * line_length); for Y := 0 to length(input.page) - 1 do move(input.page[Y][0], bitmap[Y * line_length], line_length); end; setlength(bits_per_sample, input.samples_per_pixel); for Y := 0 to input.samples_per_pixel - 1 do bits_per_sample[Y] := input.bits_per_sample; file_record.file_name := output_directory + short_file_name(input.file_name) + '.tif'; file_record.was_error := false; file_record.last_error := ''; file_record.image_count := 1; file_record.file_size := length(bitmap); setlength(file_record.images, 1); file_record.images[0].was_error := false; file_record.images[0].last_error := ''; file_record.images[0].description := ''; file_record.images[0].bits_per_sample := bits_per_sample; file_record.images[0].samples_per_pixel := input.samples_per_pixel; file_record.images[0].height := length(input.page); file_record.images[0].width := length(input.page[0]); file_record.images[0].image_data := bitmap; result := write_tiff_file(file_record); end; function cleaned_page(file_name: ansistring): page_type; // cleans the page associated with file_name var image_record: TIFF_file_record; page_num: word; begin image_record := read_tiff_file(file_name); if (image_record.was_error) then begin Form1.StatusBar1.SimpleText := 'error reading ' + file_name + ': ' + image_record.last_error; setlength(result.page, 0); end else begin result := read_page(image_record); Application.ProcessMessages; // give other system processes a chance to work page_num := file_number(result.file_name) - 1; if result.bits_per_sample = 1 then begin mark_halftone_areas(result); Form1.StatusBar1.SimpleText := result.file_name + ': removing single speckle strings'; Application.ProcessMessages; // give other system processes a chance to work result := remove_single_pixel_strings(result); Form1.StatusBar1.SimpleText := result.file_name + ': removing 2 x 2 blocks'; Application.ProcessMessages; // give other system processes a chance to work result := remove_single_pixel_strings(remove_two_by_two_blocks(result)); Form1.StatusBar1.SimpleText := result.file_name + ': aligning the page'; Application.ProcessMessages; // give other system processes a chance to work result := align_page(result); crop_width := min(crop_width, book_data[page_num].right_margin - book_data[page_num].left_margin); if (book_data[page_num].bottom_margin <> length(result.page)) and (not outlier(book_data[page_num].bottom_margin, crop_height_stats)) then crop_height := min(crop_height, book_data[page_num].bottom_margin); end;; Application.ProcessMessages; // give other system processes a chance to work end; end; procedure clean_book; // cleans all of the pages within the book and saves them to the cleaned subdirectory var file_count: longword; begin crop_width := $1000000; crop_height := $1000000; for file_count := 0 to length(files) - 1 do write_page(cleaned_page(files[file_count].directory + files[file_count].info.Name)); end; It seems that, on both 32 bit and 64 bit Lubuntu, the failure occurs immediately after the function write_page, that is, during whatever cleanup operation is undertaken prior to returning to the calling procedure, clean_book. I’m not sure why this should be and why cmem is so much worse than the native memory manager, but, near as I can tell, it doesn’t seem to have anything to do with my code, but rather the implementation of dynamic arrays. I can confirm that the file writing process was complete and that the file was properly closed. Dynamic arrays are an extremely valuable capability in FreePascal since they free the programmer from having to worry about memory management details. I use them extensively in other software that I’ve written, especially in a geometric algebra calculator that I’d like to release to the public domain. Since dynamic arrays are an intrinsic part of the FreePascal language definition, I would hope that one can count on being able to use them without concern that they will at some time fail. Yes, I know that I’m pushing hard on them (and the underlying memory manager) in this software, but as a programmer, it should be reasonable to believe that these functions will not fail, although, due to memory fragmentation, they might take longer over time. Again, if I can be of help in tracking down and correcting this problem let me know. I’m happy to help make this feature of FreePascal bug free. |
|
|
Are you sure this is due to memory fragmentation? 500000 allocations is not a lot. I see crashes near FreeMem usually after some form of heap corruption, most likely caused by out-of-bounds writes. The memory manager tends to be the first thing that breaks. |
|
|
Crash with cmem as well and the point Martok mentioned points for me also into the direction of memory corruptions. Did you try to turn on all checks fpc offers (e.g. range, overflow, heap)? |
|
|
[code=pascal] for Y := 0 to input.height - 1 do begin setlength(result[Y], input.width); [/code] Is highly inefficient code and a possible cause of fragmentation, because it keeps creating copies. Can you bring the setlength outside of the loop? That results in width-1 less (re-)allocations. You can set it in one go, since the result is rectanglar. See http://wiki.freepascal.org/Example:_Multidimensional_dynamic_array Setlength(Result,input.Height,input.width); |
|
|
The cmem traceback has double free warnings? Anyway first -CRriot, and otherwise valgrind. |
|
|
I re-tested and have no problems with well (somewhat) written code. As per my remark. |
|
|
Try compiling with -gv and running under valgrind to ensure you don't have buffer overflows. Otherwise please provide a compilable example that demonstrates the problem. |
|
|
Wondering if this problem is related to 35187? https://bugs.freepascal.org/view.php?id=35187 PB |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2018-03-13 18:14 | John | New Issue | |
| 2018-03-13 19:52 | Florian | Note Added: 0107089 | |
| 2018-03-14 11:09 | Marco van de Voort | Note Added: 0107103 | |
| 2018-03-15 18:18 | John | Note Added: 0107139 | |
| 2018-03-15 18:29 | John | Note Edited: 0107139 | View Revisions |
| 2018-03-17 18:49 | John | Note Added: 0107212 | |
| 2018-03-17 20:07 | Martok | Note Added: 0107216 | |
| 2018-03-17 22:41 | Florian | Note Added: 0107220 | |
| 2018-03-18 07:32 | Thaddy de Koning | Note Added: 0107228 | |
| 2018-03-18 07:33 | Thaddy de Koning | Note Edited: 0107228 | View Revisions |
| 2018-03-18 07:39 | Thaddy de Koning | Note Edited: 0107228 | View Revisions |
| 2018-03-18 07:41 | Thaddy de Koning | Note Edited: 0107228 | View Revisions |
| 2018-03-18 13:46 | Marco van de Voort | Note Added: 0107237 | |
| 2018-03-23 18:32 | Thaddy de Koning | Note Added: 0107375 | |
| 2018-12-24 16:20 | Jonas Maebe | Note Added: 0112862 | |
| 2018-12-24 16:20 | Jonas Maebe | Status | new => resolved |
| 2018-12-24 16:20 | Jonas Maebe | Resolution | open => unable to reproduce |
| 2018-12-24 16:20 | Jonas Maebe | Assigned To | => Jonas Maebe |
| 2020-10-20 16:53 | Peter Blackman | Note Added: 0126426 |
