View Issue Details

IDProjectCategoryView StatusLast Update
0035610LazarusLCLpublic2019-06-05 12:27
ReporterDeeAssigned Towp 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
PlatformDesktop-PC, NotebookOSWindows (64-bit)OS Version7, 10
Product Version2.0Product Buildr60307 
Target VersionFixed in Version 
Summary0035610: ERangeError in list based components (e.g. ListBox) with vertical scrollbars when there are at least 32779 items in it
DescriptionAn ERangeError occurs if I try to scroll down (e.g. in a ListBox) via mouse wheel to an item which is at least on the 32779th positon or higher. But this error occurs only if the IDE is compiled with debugging information (Tools -> Configure "Build Lazarus" -> Profile to Build -> Debug IDE).
Steps To Reproduce1. Recompile the Lazarus IDE as Debug IDE
2. place ListBox in any form
3. fill the ListBox with at least 32779 items (e.g. via a button and a loop)
4. scroll down in the ListBox at least to the 32779 item or higher with the mouse wheel
5. RunError 201 and ERangeError occurs
Additional InformationSeems to be an 64-bit problem. Also tested with 2.0.2 version and error occurs there, too.

Here is a link to a forum thread I created talking about the problem:

http://www.lazarusforum.de/viewtopic.php?f=18&t=12383

The user wp_xyz in this thread might have posted the solution.

SHA-1 checksum of demo program attachment: 095056e3ef393f356a0ad8f20196c5c9409e6312
TagsNo tags attached.
Fixed in Revision61321
LazTarget2.0.4
WidgetsetWin32/Win64
Attached Files
  • ListBox_ERangeError.zip (2,060 bytes)
  • rng.diff (2,328 bytes)
    Index: lcl/interfaces/win32/win32callback.inc
    ===================================================================
    --- lcl/interfaces/win32/win32callback.inc	(revision 61271)
    +++ lcl/interfaces/win32/win32callback.inc	(working copy)
    @@ -736,17 +736,22 @@
     procedure TWindowProcHelper.HandleScrollMessage(LMsg: integer);
     var
       ScrollInfo: TScrollInfo;
    +  NHigh, NLow: Word;
     begin
    +  // Cast WParam:PtrInt to LongInt to avoid range-check error
    +  NHigh := HIWORD(LongInt(WParam));
    +  NLow := LOWORD(LongInt(WParam));
    +
       with LMScroll do
       begin
         Msg := LMsg;
    -    ScrollCode := LOWORD(WParam);
    +    ScrollCode := NLow;
         SmallPos := 0;
         ScrollBar := HWND(LParam);
         Pos := 0;
       end;
     
    -  if not (LOWORD(WParam) in [SB_THUMBTRACK, SB_THUMBPOSITION])
    +  if not (NLow in [SB_THUMBTRACK, SB_THUMBPOSITION])
       then begin
         WindowInfo^.TrackValid := False;
         Exit;
    @@ -774,17 +779,17 @@
       // MWE.
     
       ScrollInfo.cbSize := SizeOf(ScrollInfo);
    -  if LOWORD(WParam) = SB_THUMBTRACK
    +  if NLow = SB_THUMBTRACK
       then begin
         ScrollInfo.fMask := SIF_TRACKPOS;
         // older windows versions may not support trackpos, so fill it with some default
         if WindowInfo^.TrackValid
    -    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
    -    else ScrollInfo.nTrackPos := HIWORD(WParam);
    +    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or NHigh
    +    else ScrollInfo.nTrackPos := NHigh;
       end
       else begin
         ScrollInfo.fMask := SIF_POS;
    -    ScrollInfo.nPos := HIWORD(WParam);
    +    ScrollInfo.nPos := NHigh;
       end;
     
       if LParam <> 0
    @@ -799,7 +804,7 @@
         else GetScrollInfo(Window, SB_VERT, ScrollInfo);
       end;
     
    -  if LOWORD(WParam) = SB_THUMBTRACK
    +  if NLow = SB_THUMBTRACK
       then begin
         LMScroll.Pos := ScrollInfo.nTrackPos;
         WindowInfo^.TrackPos := ScrollInfo.nTrackPos;
    @@ -807,8 +812,8 @@
       end
       else begin
         if WindowInfo^.TrackValid
    -    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
    -    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or HIWORD(WParam);
    +    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or NHigh
    +    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or NHigh;
       end;
     
       if LMScroll.Pos < High(LMScroll.SmallPos)
    
    rng.diff (2,328 bytes)
  • Listview_Demo.zip (2,136 bytes)
  • rng_alt.diff (2,229 bytes)
    Index: lcl/interfaces/win32/win32callback.inc
    ===================================================================
    --- lcl/interfaces/win32/win32callback.inc	(revision 61292)
    +++ lcl/interfaces/win32/win32callback.inc	(working copy)
    @@ -740,13 +740,13 @@
       with LMScroll do
       begin
         Msg := LMsg;
    -    ScrollCode := LOWORD(WParam);
    +    ScrollCode := LOWORD(Integer(WParam));
         SmallPos := 0;
         ScrollBar := HWND(LParam);
         Pos := 0;
       end;
     
    -  if not (LOWORD(WParam) in [SB_THUMBTRACK, SB_THUMBPOSITION])
    +  if not (LOWORD(Integer(WParam)) in [SB_THUMBTRACK, SB_THUMBPOSITION])
       then begin
         WindowInfo^.TrackValid := False;
         Exit;
    @@ -774,17 +774,17 @@
       // MWE.
     
       ScrollInfo.cbSize := SizeOf(ScrollInfo);
    -  if LOWORD(WParam) = SB_THUMBTRACK
    +  if LOWORD(Integer(WParam)) = SB_THUMBTRACK
       then begin
         ScrollInfo.fMask := SIF_TRACKPOS;
         // older windows versions may not support trackpos, so fill it with some default
         if WindowInfo^.TrackValid
    -    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
    -    else ScrollInfo.nTrackPos := HIWORD(WParam);
    +    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(Integer(WParam))
    +    else ScrollInfo.nTrackPos := HIWORD(Integer(WParam));
       end
       else begin
         ScrollInfo.fMask := SIF_POS;
    -    ScrollInfo.nPos := HIWORD(WParam);
    +    ScrollInfo.nPos := HIWORD(Integer(WParam));
       end;
     
       if LParam <> 0
    @@ -799,7 +799,7 @@
         else GetScrollInfo(Window, SB_VERT, ScrollInfo);
       end;
     
    -  if LOWORD(WParam) = SB_THUMBTRACK
    +  if LOWORD(Integer(WParam)) = SB_THUMBTRACK
       then begin
         LMScroll.Pos := ScrollInfo.nTrackPos;
         WindowInfo^.TrackPos := ScrollInfo.nTrackPos;
    @@ -807,8 +807,8 @@
       end
       else begin
         if WindowInfo^.TrackValid
    -    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
    -    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or HIWORD(WParam);
    +    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(Integer(WParam))
    +    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or HIWORD(Integer(WParam));
       end;
     
       if LMScroll.Pos < High(LMScroll.SmallPos)
    
    rng_alt.diff (2,229 bytes)

Activities

Dee

2019-05-21 20:19

reporter  

ListBox_ERangeError.zip (2,060 bytes)

Dee

2019-05-21 20:24

reporter   ~0116310

I have forgot to mention that the error happens in the file win32callback.inc in line 738.

Juha Manninen

2019-05-22 16:18

developer   ~0116334

If you know how to fix it, please create a patch.
 http://wiki.freepascal.org/Creating_A_Patch

Alexey Tor.

2019-05-22 16:57

reporter   ~0116341

fix is simple- added.

rng.diff (2,328 bytes)
Index: lcl/interfaces/win32/win32callback.inc
===================================================================
--- lcl/interfaces/win32/win32callback.inc	(revision 61271)
+++ lcl/interfaces/win32/win32callback.inc	(working copy)
@@ -736,17 +736,22 @@
 procedure TWindowProcHelper.HandleScrollMessage(LMsg: integer);
 var
   ScrollInfo: TScrollInfo;
+  NHigh, NLow: Word;
 begin
+  // Cast WParam:PtrInt to LongInt to avoid range-check error
+  NHigh := HIWORD(LongInt(WParam));
+  NLow := LOWORD(LongInt(WParam));
+
   with LMScroll do
   begin
     Msg := LMsg;
-    ScrollCode := LOWORD(WParam);
+    ScrollCode := NLow;
     SmallPos := 0;
     ScrollBar := HWND(LParam);
     Pos := 0;
   end;
 
-  if not (LOWORD(WParam) in [SB_THUMBTRACK, SB_THUMBPOSITION])
+  if not (NLow in [SB_THUMBTRACK, SB_THUMBPOSITION])
   then begin
     WindowInfo^.TrackValid := False;
     Exit;
@@ -774,17 +779,17 @@
   // MWE.
 
   ScrollInfo.cbSize := SizeOf(ScrollInfo);
-  if LOWORD(WParam) = SB_THUMBTRACK
+  if NLow = SB_THUMBTRACK
   then begin
     ScrollInfo.fMask := SIF_TRACKPOS;
     // older windows versions may not support trackpos, so fill it with some default
     if WindowInfo^.TrackValid
-    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
-    else ScrollInfo.nTrackPos := HIWORD(WParam);
+    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or NHigh
+    else ScrollInfo.nTrackPos := NHigh;
   end
   else begin
     ScrollInfo.fMask := SIF_POS;
-    ScrollInfo.nPos := HIWORD(WParam);
+    ScrollInfo.nPos := NHigh;
   end;
 
   if LParam <> 0
@@ -799,7 +804,7 @@
     else GetScrollInfo(Window, SB_VERT, ScrollInfo);
   end;
 
-  if LOWORD(WParam) = SB_THUMBTRACK
+  if NLow = SB_THUMBTRACK
   then begin
     LMScroll.Pos := ScrollInfo.nTrackPos;
     WindowInfo^.TrackPos := ScrollInfo.nTrackPos;
@@ -807,8 +812,8 @@
   end
   else begin
     if WindowInfo^.TrackValid
-    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
-    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or HIWORD(WParam);
+    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or NHigh
+    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or NHigh;
   end;
 
   if LMScroll.Pos < High(LMScroll.SmallPos)
rng.diff (2,328 bytes)

jamie philbrook

2019-05-22 22:52

reporter   ~0116351

Last edited: 2019-05-22 22:54

View 2 revisions

List boxes in windows have a Limit of 32,767 items. But they can use unlimited memory..

Here is an example of that information
https://msdn.microsoft.com/en-us/library/windows/desktop/bb761356(v=vs.85).aspx

I guess one should test it first.

Alexey Tor.

2019-05-22 23:38

reporter   ~0116353

jamie, limit is only on Win98/WinMe.

wp

2019-05-22 23:42

developer   ~0116354

No, it is no problem to fill the listbox with 100,000 and more items when range-checking is off. And the issue is not only restricted to TListbox, it also happens with TListView and probably others.

I am not sure whether Alexey's patch is correct. TWindowProcHelper.HandleScrollMessage appears to me as a rather elemental message handler, and I don't know whether trimming 64bit back to 32bit this way has any side-effects.

Dee

2019-05-24 00:53

reporter   ~0116385

"And the issue is not only restricted to TListbox, it also happens with TListView and probably others."

I can confirm that.

After applying the patch no error occurs. It seems that the patch is working but I do not know either, if it will have any side effects.
Attached is another demo but with a ListView generating 100.000.000 items. (original from the linked forum; translated into english)

Listview_Demo.zip (2,136 bytes)

Alexey Tor.

2019-05-25 12:17

reporter   ~0116407

@wp,
What patch does- it only adds LongInt() cast for WParam, where WParam is PtrInt (64-bit on Win64).
Win32 - patch has no effect (WParam is 32 bit)
Win64 - patch casts 64-bit WParam to LongInt, so LOWORD+HIWORD won't crash on 64-bit value if bits 32..63 are set.

Juha Manninen

2019-05-27 15:14

developer   ~0116434

wp, can you please look at this. My development OS is not Windows.

wp

2019-05-27 18:37

developer   ~0116437

No, sorry. I do develop in Windows, but I am not strong with the internals.

Michal Gawrycki

2019-05-27 22:48

reporter   ~0116438

I can confirm that this patch correctly solves the problem.
I can only suggest an alternative version of solution without introducing new local variables.
I have found more instances of LOWORD(WParam)/HIWORD (WParam) without casting to integer type that can cause a similar problems. I will prepare patch and open new issue.

rng_alt.diff (2,229 bytes)
Index: lcl/interfaces/win32/win32callback.inc
===================================================================
--- lcl/interfaces/win32/win32callback.inc	(revision 61292)
+++ lcl/interfaces/win32/win32callback.inc	(working copy)
@@ -740,13 +740,13 @@
   with LMScroll do
   begin
     Msg := LMsg;
-    ScrollCode := LOWORD(WParam);
+    ScrollCode := LOWORD(Integer(WParam));
     SmallPos := 0;
     ScrollBar := HWND(LParam);
     Pos := 0;
   end;
 
-  if not (LOWORD(WParam) in [SB_THUMBTRACK, SB_THUMBPOSITION])
+  if not (LOWORD(Integer(WParam)) in [SB_THUMBTRACK, SB_THUMBPOSITION])
   then begin
     WindowInfo^.TrackValid := False;
     Exit;
@@ -774,17 +774,17 @@
   // MWE.
 
   ScrollInfo.cbSize := SizeOf(ScrollInfo);
-  if LOWORD(WParam) = SB_THUMBTRACK
+  if LOWORD(Integer(WParam)) = SB_THUMBTRACK
   then begin
     ScrollInfo.fMask := SIF_TRACKPOS;
     // older windows versions may not support trackpos, so fill it with some default
     if WindowInfo^.TrackValid
-    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
-    else ScrollInfo.nTrackPos := HIWORD(WParam);
+    then ScrollInfo.nTrackPos := Integer(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(Integer(WParam))
+    else ScrollInfo.nTrackPos := HIWORD(Integer(WParam));
   end
   else begin
     ScrollInfo.fMask := SIF_POS;
-    ScrollInfo.nPos := HIWORD(WParam);
+    ScrollInfo.nPos := HIWORD(Integer(WParam));
   end;
 
   if LParam <> 0
@@ -799,7 +799,7 @@
     else GetScrollInfo(Window, SB_VERT, ScrollInfo);
   end;
 
-  if LOWORD(WParam) = SB_THUMBTRACK
+  if LOWORD(Integer(WParam)) = SB_THUMBTRACK
   then begin
     LMScroll.Pos := ScrollInfo.nTrackPos;
     WindowInfo^.TrackPos := ScrollInfo.nTrackPos;
@@ -807,8 +807,8 @@
   end
   else begin
     if WindowInfo^.TrackValid
-    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(WParam)
-    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or HIWORD(WParam);
+    then LMScroll.Pos := LongInt(WindowInfo^.TrackPos and $FFFF0000) or HIWORD(Integer(WParam))
+    else LMScroll.Pos := (ScrollInfo.nPos and $FFFF0000) or HIWORD(Integer(WParam));
   end;
 
   if LMScroll.Pos < High(LMScroll.SmallPos)
rng_alt.diff (2,229 bytes)

wp

2019-05-27 23:40

developer   ~0116440

Last edited: 2019-05-28 01:27

View 2 revisions

I am absolutely sure that this particular problem can be fixed this way. The question, however, is for me whether there are no side-effects. I don't have sufficient knowledge of the Windows API to tell how the ScrollMessage which obviously is a remnant of 32-bit code here should be handled correctly in 64-bit.

Michal Gawrycki

2019-05-28 13:37

reporter   ~0116445

Basically there is no difference between 32bit and 64bit api. Of course, except for pointers. So we still need to use the LOWORD / HIWORD function, which takes the 32bit integer as a parameter.

Dee

2019-06-05 01:58

reporter   ~0116571

Well then it is solved, I guess? What are we waiting for? I don't know the process of this.

wp

2019-06-05 12:26

developer   ~0116573

Last edited: 2019-06-05 12:27

View 2 revisions

> Basically there is no difference between 32bit and 64bit api. Of course, except for pointers.

Thanks, I saw this statement now also on some Microsoft site. Therefore, I pick up this issue again and resolve it. Thanks for the patch. Hower, I used "LongInt" instead of "Integer" to indicate that it must be 32 bit, "Integer", in general, almost can be anything.

Please test and close.

Issue History

Date Modified Username Field Change
2019-05-21 20:19 Dee New Issue
2019-05-21 20:19 Dee File Added: ListBox_ERangeError.zip
2019-05-21 20:24 Dee Note Added: 0116310
2019-05-22 16:18 Juha Manninen Note Added: 0116334
2019-05-22 16:57 Alexey Tor. File Added: rng.diff
2019-05-22 16:57 Alexey Tor. Note Added: 0116341
2019-05-22 22:52 jamie philbrook Note Added: 0116351
2019-05-22 22:54 jamie philbrook Note Edited: 0116351 View Revisions
2019-05-22 23:38 Alexey Tor. Note Added: 0116353
2019-05-22 23:42 wp Note Added: 0116354
2019-05-24 00:53 Dee File Added: Listview_Demo.zip
2019-05-24 00:53 Dee Note Added: 0116385
2019-05-25 12:17 Alexey Tor. Note Added: 0116407
2019-05-27 15:13 Juha Manninen Assigned To => wp
2019-05-27 15:13 Juha Manninen Status new => assigned
2019-05-27 15:14 Juha Manninen Note Added: 0116434
2019-05-27 18:36 wp Assigned To wp =>
2019-05-27 18:37 wp Note Added: 0116437
2019-05-27 22:48 Michal Gawrycki File Added: rng_alt.diff
2019-05-27 22:48 Michal Gawrycki Note Added: 0116438
2019-05-27 23:40 wp Note Added: 0116440
2019-05-28 01:27 wp Note Edited: 0116440 View Revisions
2019-05-28 11:13 wp Assigned To => wp
2019-05-28 11:13 wp Status assigned => new
2019-05-28 11:13 wp LazTarget => -
2019-05-28 11:20 wp Assigned To wp =>
2019-05-28 13:37 Michal Gawrycki Note Added: 0116445
2019-06-05 01:58 Dee Note Added: 0116571
2019-06-05 12:24 wp Assigned To => wp
2019-06-05 12:24 wp Status new => assigned
2019-06-05 12:26 wp Status assigned => resolved
2019-06-05 12:26 wp Resolution open => fixed
2019-06-05 12:26 wp Fixed in Revision => 61321
2019-06-05 12:26 wp LazTarget - => 2.0.4
2019-06-05 12:26 wp Widgetset Win32/Win64 => Win32/Win64
2019-06-05 12:26 wp Note Added: 0116573
2019-06-05 12:27 wp Note Edited: 0116573 View Revisions