View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0037179 | Lazarus | IDE | public | 2020-06-06 04:26 | 2020-10-15 14:11 |
Reporter | Cyrax | Assigned To | Juha Manninen | ||
Priority | normal | Severity | minor | Reproducibility | random |
Status | resolved | Resolution | reopened | ||
Platform | Linux x86_64 | OS | Arch | ||
Product Version | 2.1 (SVN) | ||||
Summary | 0037179: [lazbuild] [patch] Lazbuild randomly aborts project building process. | ||||
Description | procedure TExternalToolThread.SetTool(AValue: TExternalTool); begin if FTool=AValue then Exit; if FTool<>nil then FTool.Thread:=nil; FTool:=AValue; if FTool<>nil then FTool.Thread:=Self; end; destructor TExternalToolThread.Destroy; begin Tool:=nil; inherited Destroy; end; Setting FTool field to NIL (aka trying to prepare it and thread associated to it to be destroyed as well) via property setter causes already running thread to product an access violation and thus breaking the build process along side of it. Attached patch will fix this by accessing FTool field directly. Valgrind didn't show any leaks. | ||||
Tags | No tags attached. | ||||
Fixed in Revision | r63321, r64005 | ||||
LazTarget | - | ||||
Widgetset | GTK 2 | ||||
Attached Files |
|
related to | 0036318 | resolved | Juha Manninen | lazbuild crashes when compiling packages |
related to | 0036959 | resolved | Juha Manninen | LazBuild crashes with --recursive builds |
|
exttools.pas.diff (267 bytes)
diff --git a/ide/exttools.pas b/ide/exttools.pas index f81f0f27e2..598948ce2a 100644 --- a/ide/exttools.pas +++ b/ide/exttools.pas @@ -1678,7 +1678,7 @@ end; destructor TExternalToolThread.Destroy; begin - Tool:=nil; + FTool:=nil; inherited Destroy; end; |
|
After applying this patch, no crashes anymore when building Lazarus. The crash also happened very often when building lazpaint. This patch also solved the lazpaint crash. So, it seems to work. And. If valgrind shows no leaks, this could mean that the issues can be considered as solved. https://bugs.freepascal.org/view.php?id=36318 https://bugs.freepascal.org/view.php?id=36959 @juha: please review ! |
|
The patch omits calling SetThread. This might work in lazbuild, but not in the IDE. CallAutoFree in SetThread was not initialized. I fixed that. |
|
@Mattias Your fix works on Windows. Even with lazbuild max-process-count set to a value > 1. Going to test on Catalina now. Will report re-occurrence of issue. You might close related issues I guess. |
|
Celebrating too early I guess ... :-( After 12 x building lazpaint ... it happened again unfortunately. |
|
Stack trace $081A02A0 file: components/buildintf/ideexterntoolintf.pas : 1237; $08757A34 file: ide/exttools.pas : 475; $0875C958 file: ide/exttools.pas : 1383; $0875DF6F file: ide/exttools.pas : 1689; $0806D702 file: ../../../../../../fpc/source/git_source/rtl/inc/objpas.inc : 336; $080A4062 file: ../../../../../../fpc/source/git_source/rtl/unix/tthread.inc : 137; $0808C465 file: ../../../../../../fpc/source/git_source/rtl/unix/cthreads.pp : 317; $F7EE8687 $00000000 |
|
@Mattias. Latest fix looks good. 25x rebuild/install of lazpaint : all ok. Will run for max 100x. |
|
After 62 times, the (repeated) failures start again. (3104) Compiling lcvectormultishape.pas (1008) 15619 lines compiled, 3.6 sec (1021) 31 warning(s) issued (1022) 226 hint(s) issued (1023) 63 note(s) issued An unhandled exception occurred at $778B3D63: EAccessViolation: Access violation $778B3D63 $77888911 $778860F9 $004172B9 $00416BA6 $00490998 TABSTRACTEXTERNALTOOL__ENTERCRITICALSECTION, line 1231 of ideexterntoolintf.pas $0072B44F TEXTERNALTOOL__SETTHREAD, line 469 of exttools.pas $0072E788 TEXTERNALTOOLTHREAD__SETTOOL, line 1389 of exttools.pas $0072F5ED TEXTERNALTOOLTHREAD__DESTROY, line 1695 of exttools.pas (1002) Target OS: Win32 for i386 (3104) Compiling lazpaint.lpr $004132F2 $00409D6F $768F6359 $778B7C24 $778B7BF4 I will start my own investigation again. Sorry about this. |
|
SynUniHighlighter.pas(1206) Hint: (8028) Change of bind type of symbol INIT_$SYNUNIHIGHLIGHTER_$$_TSYNUNISYN from external to global after use SynUniHighlighter.pas(1206) Hint: (8028) Change of bind type of symbol RTTI_$SYNUNIHIGHLIGHTER_$$_TSYNUNISYN from external to global after use /mnt/shares/ohjelmointi/32-bit/kielet/pascal/free_pascal/lazarus/projects_from_others/double_commander/git_double_commander/source/components/synunihighlighter/synuni.pas(10,3) Hint: (5023) Unit "SynUniClasses" not used in SynUni TExternalToolsConsole.HandleMesages: Calling CheckSynchronize! TExternalToolsConsole.HandleMesages: Calling CheckSynchronize! /mnt/shares/ohjelmointi/32-bit/kielet/pascal/free_pascal/lazarus/projects_from_others/double_commander/git_double_commander/source/components/synunihighlAn unhandled exception occurred at $08072419: EStackOverflow: Stack overflow $08072419 $080F05DA $080F15F9 $08072B31 $0916FF88 $087320C8 $08733287 $0806B7E2 $080A2142 $0808A545 $F7EFC687 ighter/synuni.pas(10,37) Hint: (5023) Unit "SynUniRules" not used in SynUni (1008) 4023 lines compiled, 5.9 sec (1021) 3 warning(s) issued (1022) 77 hint(s) issued (1023) 9682 note(s) issued TExternalToolsConsole.HandleMesages: Calling CheckSynchronize! An unhandled exception occurred at $080612EB: EAccessViolation: Access violation $080612EB $081DF627 $081EAE04 $081EACFF $0823A467 $0804C55F $0804BD50 $0804A47C $0804F301 $08051A67 |
|
Lazarus trunk r63325. FPC trunk r45588. Still getting random crashes at lazbuild. ----- Stack trace $0819E380 file: components/buildintf/ideexterntoolintf.pas : 1237; $0872E430 file: ide/exttools.pas : 475; $087320C8 file: ide/exttools.pas : 1389; $08733287 file: ide/exttools.pas : 1695; $0806B7E2 file: ../../../../../../fpc/source/git_source/rtl/inc/objpas.inc : 336; $080A2142 file: ../../../../../../fpc/source/git_source/rtl/unix/tthread.inc : 137; $0808A545 file: ../../../../../../fpc/source/git_source/rtl/unix/cthreads.pp : 317; $F7ED7687 |
|
> EStackOverflow: Stack overflow That is a new one, isn't it? Interesting. I have seen only Access violation so far in debugger backtraces of Cyrax and Alfred. |
|
The patch in https://bugs.freepascal.org/view.php?id=37345 may lower the possibility of triggering this bug, but I have to come conclusion that it is a thread race condition on freeing FWorkerMessages in TAbstractExternalTool.Destroy (components/buildintf/ideexterntoolintf.pas, line 1226) https://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/components/buildintf/ideexterntoolintf.pas?view=markup&revision=63144&root=lazarus#l1226 EDIT : Also moving clause FTool:=nil at https://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/ide/exttools.pas?view=markup&revision=63456&root=lazarus#l1389 to line 1391 may lessen crashes |
|
Attached patch will move clause FTool:=nil to line 1391 in ide/exttools.pas exttools-2.pas.diff (347 bytes)
diff --git a/ide/exttools.pas b/ide/exttools.pas index 59eab75172..c19cbeb569 100644 --- a/ide/exttools.pas +++ b/ide/exttools.pas @@ -1386,9 +1386,9 @@ var begin if FTool=AValue then Exit; OldTool:=FTool; - FTool:=nil; if OldTool<>nil then OldTool.Thread:=nil; + FTool:=nil; if AValue<>nil then begin FTool:=AValue; |
|
Attached patch is an attempt to fix the race condition by creating subclass of TObjectList, which will free instance of FWorkerMessages class later time by utilising a timer functionality. ideexterntoolintf-2.pas.diff (2,299 bytes)
diff --git a/components/buildintf/ideexterntoolintf.pas b/components/buildintf/ideexterntoolintf.pas index 2da4a39ad7..b1d69d1e2f 100644 --- a/components/buildintf/ideexterntoolintf.pas +++ b/components/buildintf/ideexterntoolintf.pas @@ -727,6 +727,26 @@ function dbgs(s: TExternalToolStage): string; overload; implementation +uses + contnrs, fptimer; + +type + + { TDelayedObjectFreeList } + + TDelayedObjectFreeList = class(TObjectList) + private + FFPTimer : TFPTimer; + procedure DoOnTime(Sender: TObject); + public + constructor Create; + destructor Destroy;override; + procedure Tick; + end; + +Var + ADelayedObjectFreeList : TDelayedObjectFreeList = NIL; + function CompareMsgLinesSrcPos(MsgLine1, MsgLine2: Pointer): integer; var Line1: TMessageLine absolute MsgLine1; @@ -787,6 +807,34 @@ begin WriteStr(Result,s); end; +{ TDelayedObjectFreeList } + +procedure TDelayedObjectFreeList.DoOnTime(Sender: TObject); +begin + FFPTimer.Enabled:=False; + Self.Delete(0); +end; + +constructor TDelayedObjectFreeList.Create; +begin + inherited Create(True); + FFPTimer := TFPTimer.Create(NIL); + FFPTimer.Interval := 1000; + FFPTimer.Enabled:=False; + FFPTimer.OnTimer := @DoOnTime; +end; + +destructor TDelayedObjectFreeList.Destroy; +begin + FreeAndNil(FFPTimer); + inherited Destroy; +end; + +procedure TDelayedObjectFreeList.Tick; +begin + FFPTimer.Enabled:=True; +end; + { TIDEExternalToolData } constructor TIDEExternalToolData.Create(aKind, aModuleName, aFilename: string); @@ -1197,6 +1245,7 @@ begin FEstimatedLoad:=1; FEnvironmentOverrides:=TStringList.Create; FReferences:=TStringList.Create; + ADelayedObjectFreeList.Add(FWorkerMessages); end; destructor TAbstractExternalTool.Destroy; @@ -1223,7 +1272,7 @@ begin finally LeaveCriticalsection; end; - FreeAndNil(FWorkerMessages); + ADelayedObjectFreeList.Tick; end; procedure TAbstractExternalTool.EnterCriticalSection; @@ -2523,6 +2572,10 @@ initialization // on single cores there is delay due to file reads // => use 2 processes in parallel by default DefaultMaxProcessCount:=Max(2,GetSystemThreadCount); + ADelayedObjectFreeList:=TDelayedObjectFreeList.Create; + +finalization + FreeAndNil(ADelayedObjectFreeList); end. |
|
FPC trunk r46453. Lazarus trunk r63750. --- TExternalToolsConsole.HandleMesages: Calling CheckSynchronize! TExternalToolsConsole.HandleMesages: Calling CheckSynchronize! /mnt/shares/ohjelmointi/32-bit/kielet/pascal/free_pascal/lazarus/projects_from_others/double_commander/git_double_commander/source/components/synunihighlighter/synuni.pas(10,37) Hint: (5023) Unit "SynUniRules" not used in SynUni (1008) 4023 lines compiled, 3.9 sec (1021) 4 warning(s) issued (1022) 77 hint(s) issued (1023) 9685 An unhandled exception occurred at $08072609: EStackOverflow: Stack overflow $08072609 $080F0F0A $080F1F29 $08072D1F $09E785E8 $0873ACDE $0873BEA7 $0806B942 $080A2462 $0808A885 $F7EC8687 note(s) issued TExternalToolsConsole.HandleMesages: Calling CheckSynchronize! TProject.DoLoadStateFile Statefile not found: /mnt/shares/ohjelmointi/32-bit/kielet/pascal/free_pascal/lazarus/projects_from_others/double_commander/git_double_commander/source/units/i386-linux-gtk2-30301/.compiled Hint: (lazarus) Project needs building: No state file for Project An unhandled exception occurred at $0814D095: EAccessViolation: Access violation $0814D095 $0814D1C6 $08152D81 $08241B68 $082421DC $0804C8A6 $0804BD68 $0804A48C $0804F361 $08051AD7 Stack trace $08072609 file: ../../../../../../fpc/source/git_source/rtl/inc/system.inc : 1037; $080F0F0A file: ../../../../../../fpc/source/git_source/rtl/objpas/sysutils/sysutils.inc : 194; $080F1F29 file: ../../../../../../fpc/source/git_source/rtl/objpas/sysutils/sysutils.inc : 407; $08072D1F file: ../../../../../../fpc/source/git_source/rtl/inc/system.inc : 1325; $09E785E8 $0873ACDE file: ide/exttools.pas : 1390; $0873BEA7 file: ide/exttools.pas : 1697; $0806B942 file: ../../../../../../fpc/source/git_source/rtl/inc/objpas.inc : 336; $080A2462 file: ../../../../../../fpc/source/git_source/rtl/unix/tthread.inc : 137; $0808A885 file: ../../../../../../fpc/source/git_source/rtl/unix/cthreads.pp : 317; $F7EC8687 Stack trace $0814D095 file: components/codetools/codecache.pas : 578; $0814D1C6 file: components/codetools/codecache.pas : 607; $08152D81 file: components/codetools/codetoolmanager.pas : 1264; $08241B68 file: ide/buildmanager.pas : 1902; $082421DC file: ide/buildmanager.pas : 1987; $0804C8A6 file: ide/lazbuild.lpr : 857; $0804BD68 file: ide/lazbuild.lpr : 938; $0804A48C file: ide/lazbuild.lpr : 426; $0804F361 file: ide/lazbuild.lpr : 1476; $08051AD7 file: ide/lazbuild.lpr : 1879; $00000000 |
|
/mnt/shares/ohjelmointi/32-bit/kielet/pascal/free_pascal/lazarus/projects_from_others/double_commander/git_double_commander/source/components/synunihighlighter/synuni.pas(10,37) Hint: (5023) Unit "SynUniRules" not used in SynUni (1008) 4023 lines compiled, 4.0 sec (1021) 4 warning(s) issued (1022) 77 hint(s) issued An unhandled exception occurred at $0877189C: EObjectCheck: Object reference is Nil $0877189C $087767EE $08777E0F $0806D882 $080A43A2 $0808C7C5 $F7F6F687 (1023) 9685 note(s) issued TExternalToolsConsole.HandleMesages: Calling CheckSynchronize! codetoolmanager.pas - finalization [TCodeToolManager.Destroy] A [TCodeToolManager.Destroy] B [TCodeToolManager.Destroy] C [TCodeToolManager.Destroy] D [TCodeToolManager.Destroy] E TProject.DoLoadStateFile Statefile not found: /mnt/shares/ohjelmointi/32-bit/kielet/pascal/free_pascal/lazarus/projects_from_others/double_commander/git_double_commander/source/units/i386-linux-gtk2-30301/.compiled Hint: (lazarus) Project needs building: No state file for Project >>>>>> [TCodeToolManager.LoadFile] /mnt/shares/ohjelmointi/32-bit/kielet/pascal/free_pascal/lazarus/projects_from_others/double_commander/git_double_commander/source/src/fmain.pas Update=True Revert=False An unhandled exception occurred at $0814EFEB: EAccessViolation: Access violation $0814EFEB $0814F1E6 $08154FD9 $08269C6B $0826A41C $0804D524 $0804C60F $0804A89F $08050AAE $08053A05 Stack trace $0877189C file: ide/exttools.pas : 475; $087767EE file: ide/exttools.pas : 1390; $08777E0F file: ide/exttools.pas : 1697; $0806D882 file: ../../../../../../fpc/source/git_source/rtl/inc/objpas.inc : 336; $080A43A2 file: ../../../../../../fpc/source/git_source/rtl/unix/tthread.inc : 137; $0808C7C5 file: ../../../../../../fpc/source/git_source/rtl/unix/cthreads.pp : 317; $F7F6F687 Stack trace $0814EFEB file: components/codetools/codecache.pas : 578; $0814F1E6 file: components/codetools/codecache.pas : 607; $08154FD9 file: components/codetools/codetoolmanager.pas : 1264; $08269C6B file: ide/buildmanager.pas : 1902; $0826A41C file: ide/buildmanager.pas : 1987; $0804D524 file: ide/lazbuild.lpr : 857; $0804C60F file: ide/lazbuild.lpr : 938; $0804A89F file: ide/lazbuild.lpr : 426; $08050AAE file: ide/lazbuild.lpr : 1476; $08053A05 file: ide/lazbuild.lpr : 1879; |
|
Apparently fixed. Please test. |
Date Modified | Username | Field | Change |
---|---|---|---|
2020-06-06 04:26 | Cyrax | New Issue | |
2020-06-06 04:26 | Cyrax | File Added: exttools.pas.diff | |
2020-06-06 08:53 | Alfred | Note Added: 0123253 | |
2020-06-06 10:04 | Mattias Gaertner | Assigned To | => Mattias Gaertner |
2020-06-06 10:04 | Mattias Gaertner | Status | new => resolved |
2020-06-06 10:04 | Mattias Gaertner | Resolution | open => fixed |
2020-06-06 10:04 | Mattias Gaertner | LazTarget | => - |
2020-06-06 10:04 | Mattias Gaertner | Widgetset | GTK 2 => GTK 2 |
2020-06-06 10:04 | Mattias Gaertner | Note Added: 0123254 | |
2020-06-06 10:29 | Alfred | Note Added: 0123256 | |
2020-06-06 10:33 | Alfred | Note Added: 0123257 | |
2020-06-06 10:33 | Alfred | File Added: lazbuild01.JPG | |
2020-06-06 11:00 | Juha Manninen | Relationship added | related to 0036318 |
2020-06-06 11:01 | Juha Manninen | Relationship added | related to 0036959 |
2020-06-07 07:37 | Cyrax | Status | resolved => assigned |
2020-06-07 07:37 | Cyrax | Resolution | fixed => reopened |
2020-06-07 07:37 | Cyrax | Note Added: 0123289 | |
2020-06-07 09:21 | Mattias Gaertner | Status | assigned => resolved |
2020-06-07 09:21 | Mattias Gaertner | Fixed in Revision | => 63321. |
2020-06-07 09:21 | Mattias Gaertner | Widgetset | GTK 2 => GTK 2 |
2020-06-07 10:46 | Alfred | Note Added: 0123293 | |
2020-06-07 11:46 | Alfred | Note Added: 0123298 | |
2020-06-08 06:41 | Cyrax | Status | resolved => assigned |
2020-06-08 06:41 | Cyrax | Note Added: 0123324 | |
2020-06-08 08:23 | Cyrax | Note Added: 0123327 | |
2020-06-08 11:04 | Juha Manninen | Note Added: 0123331 | |
2020-07-13 17:58 | Cyrax | Note Added: 0123987 | |
2020-07-13 18:05 | Cyrax | Note Edited: 0123987 | View Revisions |
2020-07-13 18:08 | Cyrax | Note Added: 0123988 | |
2020-07-13 18:08 | Cyrax | File Added: exttools-2.pas.diff | |
2020-07-13 18:17 | Cyrax | Note Added: 0123989 | |
2020-07-13 18:17 | Cyrax | File Added: ideexterntoolintf-2.pas.diff | |
2020-08-16 15:27 | Cyrax | Note Added: 0124923 | |
2020-08-16 21:19 | Cyrax | Note Added: 0124930 | |
2020-10-15 14:11 | Juha Manninen | Assigned To | Mattias Gaertner => Juha Manninen |
2020-10-15 14:11 | Juha Manninen | Status | assigned => resolved |
2020-10-15 14:11 | Juha Manninen | Fixed in Revision | 63321. => r63321, r64005 |
2020-10-15 14:11 | Juha Manninen | Widgetset | GTK 2 => GTK 2 |
2020-10-15 14:11 | Juha Manninen | Note Added: 0126332 |