View Issue Details

IDProjectCategoryView StatusLast Update
0037463LazarusLCLpublic2020-10-13 05:40
ReporterJulian Puhl Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
PlatformWin64OSWindows 
Product Version2.1 (SVN) 
Summary0037463: Double buffering seems not to be working on windows
DescriptionI noticed this when working with many components in a form. Attached is a test project and a screenshot which demonstrates the issue. If you scroll, you get a blurry mess. This seems not to happen with QT or GTK.
Steps To ReproduceScroll to get a blurry mess.
Additional InformationCompiler is FPC trunk win64.
TagsNo tags attached.
Fixed in Revision
LazTarget
WidgetsetWin32/Win64
Attached Files

Activities

Julian Puhl

2020-08-01 10:11

reporter  

TestDoubleBuffer.zip (107,389 bytes)
DoubleBufferError.jpg (61,487 bytes)   
DoubleBufferError.jpg (61,487 bytes)   

wp

2020-08-01 13:08

developer   ~0124458

I am not sure whether DoubleBuffered is the correct screw to turn in order to fix this issue. The problem is that when the scrollbars' hot-tracking is active there is an immense amount of redraw requests in the message queue. Writing to a bitmap first only adds another delay. I noticed in many cases, also with delphi, that DoubleBuffering makes the application slower.

You can avoid the drawing artifacts by turning off Tracking of both scrollbars. You can also add a client-aligned Scrollbox to the form into which the buttons are put. Interestingly, this combination results in much smoother scrolling. There is some time after the scrolling mouse button is released during which the application is not responsive. In both cases. (Move the mouse over the 'x' in the title bar and notice the delay until the 'x' becomes red (on Win 10).) there is also a noticable delay when the application is closed; this can be avoided when the AutoScroll property of Form or Scrollbar is set to false.

Julian Puhl

2020-08-01 13:54

reporter   ~0124459

I see. I did some testing before reporting this and for me it does not make a difference whether I use a Scrollbox, the scrollbars from the form, or enable/disable double buffering. Also if I set Autoscroll to false, I can't get the scrollbars to get working (tried a few method calls but it has no effect. Visible flag for each scrollbar is set to true).

Since I need the tracking (otherwise scrolling in an application is very annoying), would it be possible to reduce the number of messages in the queue (e.g. only sent it every 10ms)? Somehow QT and GTK do not have a problem with this. And I also never have seen this behaviour in any normal windows application.

Martin Friebe

2020-08-01 17:03

manager   ~0124461

Last edited: 2020-08-01 18:21

View 4 revisions

First of all, when testing: There is a different whether the child controls are TWinControl or TGraphicControl descendents.

If the scroll-container (form/scrollbox) is/were double-buffered, then this affects only items that the scroll-container draws. I.e. its own design and contained TGraphicControls.
Contained TWinContrals are (on Win, but probably most others too) never drawn by the scroll-container. They have there own OS-handle, and always draw themself.

So DoubleBuffer is not the full solution to "scroll mess".

----
What needs to happen (1st step, maybe that already happens) is: ScrollWindowEx (though if the scrolling is done by Windows, this is not something we can influence / but then it may be done correct already.)
If the scrolling is handled by the LCL, this call can tell the OS, that all the child controls should be moved. If everything is visible on screen, this should work, even without DoubleBuffering.


However this will send WM_Move messages...... And IIRC the LCL may do an invalidate of the control for which it receives them.

That is if the app receives WM_Move as result from a scroll, the Top/Left of the contained TWinControl are still at the old pos.
WM_Move causes a call to SetTop/SetLeft => and that will trigger various calls to invalidate for either the contained or the containing or both controls...... Those calls should not happen.

Those calls are in code affecting ALL widgetsets. => To fix the issue, they may need to be redirected into Widgetset code for each widgetset.....

---------------
So there is a good deal of analysis/debugging needed, to see where it goes wrong.


------------------------
EDIT:
Actually I am partly mistaken: WM_move does not call SetTop/SetLeft.

So not sure what causes it.
But even with double-buffer, any control that scrolls in, must have a paint event (even if it just copies the buffer). So if that paint event is delayed...

------------------------
EDIT:
I looked at your test.
On my System artefacts only happen for Buttons scrolling in. They must be painted once, even if double buffered.
If too many buttons scroll in, then the paint events do not happen fast enough.

I am not sure what slows them down...

One guess:
The form does get a paint event, on each pixel scroll,=> in case it needs to draw TGraphicControls. In your case there is nothing to be drawn.
But on each such event the form paint handler needs to loop the 500 buttons (check each for being a TGraphicControl, but none is) => I guess that could be what makes the paint event rather slow.

However, skipping those loops does not help

jamie philbrook

2020-08-01 23:38

reporter   ~0124473

Last edited: 2020-08-01 23:44

View 2 revisions

using 2.0.10 I can not open that project.. ?
why is that ?

 If even locks up the IDE and crashes it when I attempt to open it.

wp

2020-08-01 23:48

developer   ~0124476

Last edited: 2020-08-01 23:49

View 2 revisions

Always the same story: The Lazarus xml file formats were changed in trunk which makes projects and packages created by trunk unreadable by older versions, unless the "Maximize compatibility" boxes are checked in the project and package options. I am adding the project in a version for 2.0.x in which the "maximize" boxes were set.

Martin Friebe

2020-08-01 23:51

manager   ~0124477

If the project was saved in trunk, it may have a new format. / Remove the lpi/lps, open the lpr, and confirm "open as project" then create a new application type project when the IDE ask.
////

From what I can see, hooking WindProc, the Buttons are entirely drawn by the OS.
At least I did not see any WMPaint for them (only BN_UNPUSHED )

I currently can't test with other frameworks, if they could handle 500 Buttons.

If you use TSpeedButton (graphic control) scrolling should be fine (it is on my PC)

jamie philbrook

2020-08-02 00:29

reporter   ~0124479

ok, That looks like a windows GUI priority override.

Maybe this test should be done on a older OS..

On one of my i5 Windows 10-pro 64 bit EXE it actually LOCKS the form update for short times, there is no response from mouse of key input.
 
 So the window was hooked ? so how was this done btw using the SetWindowsHookEx ?

 I do believe the WM_PAINT message does arrive and maybe its in the Widget handler ?

 that being the case maybe a Rectangle test to see If it is in view and if not return the message has handled so windows does not attempt to paint a control needlessly .

BrunoK

2020-08-02 12:29

reporter   ~0124487

Win10.

What happens is that scrolling messages (mouse messages if I remember well) are processed before a WM_PAINT batch has totally finished to display a refreshed screen. The handling of the mouse message generates an Invalidate (Windows or Lazarus I don't know).
That induces Windows to terminate prematurely the WM_PAINT batch of Windows hence the striped appearance while scrolling and the settling at the end of mouse slider action.

I have never figured out what DoubleBuffering was useful for in Windows since the doublebuffer exists for the duration of a single WindowProc in \lcl\interfaces\win32\win32callback.inc Maybe a "professional" should review that.

Also I think I remember that QT5 does it differently but that is a 2 years old from memory observation.

WP : The effect can also be seen when dragging the Map in lamapviewer where it is aggravated by the multithreading nature of the application.

jamie philbrook

2020-08-02 14:34

reporter   ~0124492

I did some debugging last night and found that in the widget for the ButtonControl window procedure the WM_PAINT message is being delivered.
  
  I conducted a little experiment by returning the Procedure as HANDLED when ever the WM_PAINT and WM_ERASEBKGRD
was received so not to actually draw the buttons..
    
 That resulted in the forms background still drawing the outlines of the buttons and the same delay lag and smearing display exists ..

Also it seems windows knows enough to not paint or erase any controls that are not in the viewable area, that has been confirmed too by testing the controls boundaries of those that arrived at the Procedure.

 It looks like it could be painting issues related to the Parent of these controls and that would be the form, the TscrollingWinControl I would think ?

Martin Friebe

2020-08-02 15:47

manager   ~0124497

>> "So the window was hooked ? so how was this done btw using the SetWindowsHookEx ?"
Why ?

The LCL, when creating a control, passes a callback: WindowProc (in CreateParams). This is necessary, so the OS can sent updated to the LCL.
The LCL reacts to those messages, and passes them on the original handler of the OS>

For most WinControls, that includes that the OS sends WM_Paint.
- In case of a form, the WM_Paint would paint any TGraphicControls.
- In case of TButton, I did not see any WM_Paint message (Which would be in line, that you can not custom-paint buttons, or change the back-color)
  So as far as I can see button paint is handled by the OS, without telling the LCL.

In how for any action done in any other msg to WindProc affects this painting... I have no idea.

Martin Friebe

2020-08-02 15:54

manager   ~0124500

@BrunoK Any comments to the proposed changes?

Julian Puhl

2020-08-02 16:35

reporter   ~0124507

Last edited: 2020-08-02 16:39

View 4 revisions

It is nice to see this bug getting so much attention. I can attach a QT5.6.2 (32bit) build with all the dlls if it is allowed and could help (there seems to be no 64 bit support).

Btw. the drawing bug also appears with only two columns. I just exaggerated it for a better demonstration.

Sadly I do not have much experience with windows GUI handling, so I can't be of any help debugging this.

wp

2020-08-02 17:21

developer   ~0124509

In tried the demo program in a VM with Win7, and here it seems that scrolling occurs smoothly (although sometimes partially drawn buttons are visible here, too). This reminds me of that discussion about Windows 10 starting slowly after a particular update (https://bugs.freepascal.org/view.php?id=33705) in which you were involved BrunoK.

BrunoK

2020-08-02 18:33

reporter   ~0124511

WP : The difficult one was to get 0033923 accepted later (6 months) but I navigated today to approximately the same areas of the source.
If I have time and inspiration (an backups ...), I will try to retool win32callback.inc with a message tracker and see if anything surfaces.

I still think that WIN does come from another thread and initiates mouse messages that interfere with the running paint but without 'instrumenting' the code it is difficult to be to affirmative. Maybe base paint coordinates / offsets of the CustomForm canvas.

For the current glitch I have to correct my previous NEGATIVE comment about DoubleBuffered, it seems to work correctly for exactly the present test program.

Martin Friebe

2020-08-02 19:46

manager   ~0124513

I attached a small sample ("simple program") that creates a form with buttons, directly via win API. No LCL.
It does not remember the final scroll pos.
I obviously do not handle enough messages. But then, that means minimal slowdown.

It works up 150 or 200 buttons. But then things start going south.

The first issue is, that even if scrolling looks still fine, CPU load goes up AND stays UP several seconds after scrolling ended. (leaving the PC partly locked)
NativeScrollDemo.pas (3,026 bytes)   
program NativeScrollDemo;
{$mode objfpc}{$H+}

uses
  Classes, windows, LazLogger;

var
  ScrollInfo: TScrollInfo;
  WndHandle: HWND;
  lastpos: integer;


function WindowWndProc(Ahwnd: HWND; uMsg: UINT; wParam: WParam; lParam: LParam) : LRESULT; stdcall;
var
  p: Word;
begin
  Result := 0;
  Case uMsg  Of
    WM_DESTROY : PostQuitMessage(0);
    WM_VSCROLL: begin
      p := HIWORD(LongInt(WParam));

      ScrollWindowEx(WndHandle, 0, lastpos - p, nil, nil, 0, nil,
        SW_INVALIDATE or
        SW_ERASE or
        SW_SCROLLCHILDREN
      );
      lastpos := p;

      //ScrollInfo.cbSize := SizeOf(ScrollInfo);
      //ScrollInfo.fMask := SIF_ALL or SIF_DISABLENOSCROLL and not SIF_TRACKPOS;
      //ScrollInfo.nMin := 1;
      //ScrollInfo.nTrackPos := p;
      //ScrollInfo.nMax := 3000;
      //SetScrollInfo(WndHandle, SB_VERT, ScrollInfo, False);

      Result := 0;
    end;
    Else
      Result := Windows.DefWindowProc(ahwnd, uMsg, wParam, lParam);
  End;
end;



procedure ShowWindow;
Var
  A_Atom : TAtom = 0;
  WndClass : TWndClass;
  Msg: TMsg;
  ScreenWidth, ScreenHeight, MiddleX, MiddleY : LongInt;
  i, j: Integer;
Begin
  lastpos := 0;
  FillChar(WndClass, SizeOf(TWndClass), 0);

  ScreenWidth := GetSystemMetrics(SM_CXSCREEN);
  ScreenHeight := GetSystemMetrics(SM_CYSCREEN);

  MiddleX := (ScreenWidth - 500) Div 2;
  MiddleY := (ScreenHeight - 500) div 2;

  WndClass.lpszClassName:= 'HEAPTRACE_CLASS';
  WndClass.lpfnWndProc :=  @WindowWndProc;

  WndClass.hInstance := hInstance;
  WndClass.hbrBackground:= 1;
  WndClass.style := CS_HREDRAW or CS_VREDRAW;
  WndClass.hCursor := LoadCursor(0, IDC_ARROW);

  A_Atom := RegisterClass(WndClass);

  WndHandle := CreateWindow(
   WndClass.lpszClassName , // lpClassName, optional
   'Foo',
   WS_OVERLAPPEDWINDOW or WS_VISIBLE or WS_VSCROLL , // dwStyle
   MiddleX, // x
   MiddleY, // y
   500, // nWidth
   500, // nHeight
   0, // hWndParent
   0, // hMenu
   WndClass.hInstance, // hInstance
   nil  // lpParam
   );


  ScrollInfo.cbSize := SizeOf(ScrollInfo);
  ScrollInfo.fMask := SIF_ALL or SIF_DISABLENOSCROLL and not SIF_TRACKPOS;
  ScrollInfo.nMin := 1;
  ScrollInfo.nTrackPos := 0;
  ScrollInfo.nMax := 3000;
  SetScrollInfo(WndHandle, SB_VERT, ScrollInfo, False);


for i := 0 to 50 do
for j := 0 to 3 do begin
  CreateWindow(
   'BUTTON' , // lpClassName, optional
   'OK', // lpWindowName, optional
   WS_TABSTOP or WS_VISIBLE or WS_CHILD or BS_DEFPUSHBUTTON , // dwStyle
   20 + j * 60, // x
   40 + i * 60, // y
   50, // nWidth
   50, // nHeight
   WndHandle, // hWndParent
   0, // hMenu
   WndClass.hInstance, // hInstance
   nil  // lpParam
   );
end;

  BringWindowToTop(WndHandle);

  while GetMessage(Msg,0,0,0) do begin
//DbgOut(',');
    DispatchMessage(Msg);
  end;

  DestroyWindow(WndHandle);

  UnregisterClass(WndClass.lpszClassName, WndClass.hInstance);
end;


begin
  ShowWindow;
end.

NativeScrollDemo.pas (3,026 bytes)   

jamie philbrook

2020-08-03 02:34

reporter   ~0124521

I did some experimentation and what I did was during a scroll message I checked the message buffer for other scroll messages waiting and
if found extracted those (Removed) and use the newest one instead.
  I had to do a little math but the results were interesting...
there seems to be a flood of messages in the que that needs to be dealt with....

 Still testing..

BrunoK

2020-08-13 16:53

reporter   ~0124845

For whatever you can make use of, attachment of Martin Friebe's program that seems to be exempt of stall.

Seems WM_VSCROLL and WM_PAINT do not coexist well ...
NativeScrollDemo-2.pas (6,550 bytes)   
program NativeScrollDemo;

// https://bugs.freepascal.org/view.php?id=37463

{$mode objfpc}{$H+}

{.$DEFINE UseLogger}

uses
  SysUtils, Classes, Windows
  {$IFDEF UseLogger},
  LazLogger,
  WineMessages, gist_1_WM_Messages
  {$ENDIF UseLogger}
  ;

const
  PM_QS_INPUT = QS_INPUT shl 16; // Process mouse and keyboard messages.
  PM_QS_PAINT = QS_PAINT shl 16; // Process paint messages.
  { Process all posted messages, including timers and hotkeys. }
  PM_QS_POSTMESSAGE = (QS_POSTMESSAGE or QS_HOTKEY or QS_TIMER) shl 16;
  PM_QS_SENDMESSAGE = QS_SENDMESSAGE shl 16; // Process all sent messages.

var
  ScrollInfo: TScrollInfo;
  WndHandle: HWND;
  lastpos: integer;
  cPainting : boolean = false;

const
  cNbRows = 100;
  cNbColumns = 5;
  cMsgCounter : integer = -1;

{$IFDEF UseLogger},
function WineMsgStr(aMsg : UINT) : string;
var
  lWineStr : string;
begin
  Result := '';
  repeat
    lWineStr := GetWineWindMsgName(aMsg);
    if lWineStr<>'' then
      if Result <> '' then
        Result := Result + ',' + lWineStr
      else
        Result := lWineStr;
  until lWineStr = '';
end;
{$ENDIF UseLogger}

  function WindowWndProc(Ahwnd: HWND; uMsg: UINT; wParam: WParam;
    lParam: LParam): LRESULT; stdcall;

  var
    p: word;
    TextBuf : shortstring;
    TextLen : integer;
    lMsg: TMsg;
  label
    ExitProc;
  begin
    if uMsg = WM_GETTEXT then begin
      Result := Windows.DefWindowProc(ahwnd, uMsg, wParam, lParam);
      Exit;
    end;
    {$IFDEF UseLogger} DebugLnEnter;{$ENDIF UseLogger}
    if uMsg = WM_PAINT then
      cPainting := True;
    Result := 0;
    {$IFDEF UseLogger}
    Inc(cMsgCounter);
    if uMsg <> WM_NCMOUSEMOVE then begin
      DbgOut('WindowWndProc;');
      TextLen := GetWindowTextA(Ahwnd, @TextBuf[1], 255);
      SetLength(TextBuf, TextLen);
      DbgOut(TextBuf);
      DebugLn(';',IntToStr(cMsgCounter),';',IntToStr(Ahwnd),';',IntToStr(uMsg),';',
              IntToStr(wParam),';',HexStr(Pointer(lParam)),';',
              WineMsgStr(uMsg));
    end;
    {$ENDIF UseLogger}
    if cPainting and (uMsg = WM_VSCROLL) then begin
      cPainting := PeekMessage(lMsg, 0, 0, 0, PM_QS_PAINT or PM_NOREMOVE);
      if cPainting then begin
        {$IFDEF UseLogger} DebugLn('-> discarded');{$ENDIF UseLogger}
        goto
          ExitProc;
      end;
    end;
    case uMsg of
      WM_DESTROY: PostQuitMessage(0);
      WM_VSCROLL:
        begin
          p := HIWORD(longint(WParam));

          ScrollWindowEx(WndHandle, 0, lastpos - p, nil, nil, 0, nil,
            SW_INVALIDATE or SW_ERASE or SW_SCROLLCHILDREN
            );
          lastpos := p;

          //ScrollInfo.cbSize := SizeOf(ScrollInfo);
          //ScrollInfo.fMask := SIF_ALL or SIF_DISABLENOSCROLL and not SIF_TRACKPOS;
          //ScrollInfo.nMin := 1;
          //ScrollInfo.nTrackPos := p;
          //ScrollInfo.nMax := 3000;
          //SetScrollInfo(WndHandle, SB_VERT, ScrollInfo, False);

          Result := 0;
        end
      else
        Result := Windows.DefWindowProc(ahwnd, uMsg, wParam, lParam);
    end;
  ExitProc:
    {$IFDEF UseLogger} DebugLnExit; {$ENDIF UseLogger}

  end;


  procedure ShowWindow;
  var
    A_Atom: TAtom = 0;
    WndClass: TWndClass;
    Msg: TMsg;
    ScreenWidth, ScreenHeight, MiddleX, MiddleY: longint;
    i, j: integer;
    GMResult : BOOL;
    lBtnLabel : string;
    TextBuf : shortstring;
    TextLen : integer;
  begin
    lastpos := 0;
    FillChar(WndClass, SizeOf(TWndClass), 0);

    ScreenWidth := GetSystemMetrics(SM_CXSCREEN);
    ScreenHeight := GetSystemMetrics(SM_CYSCREEN);

    MiddleX := (ScreenWidth - 500) div 2;
    MiddleY := (ScreenHeight - 500) div 2;

    WndClass.lpszClassName := 'HEAPTRACE_CLASS';
    WndClass.lpfnWndProc := @WindowWndProc;

    WndClass.hInstance := hInstance;
    WndClass.hbrBackground := 1;
    WndClass.style := CS_HREDRAW or CS_VREDRAW;
    WndClass.hCursor := LoadCursor(0, IDC_ARROW);

    A_Atom := RegisterClass(WndClass);

    WndHandle := CreateWindow(WndClass.lpszClassName, // lpClassName, optional
      'Foo', WS_OVERLAPPEDWINDOW or WS_VISIBLE or WS_VSCROLL, // dwStyle
      MiddleX, // x
      MiddleY, // y
      500, // nWidth
      500, // nHeight
      0, // hWndParent
      0, // hMenu
      WndClass.hInstance, // hInstance
      nil  // lpParam
      );


    ScrollInfo.cbSize := SizeOf(ScrollInfo);
    ScrollInfo.fMask := SIF_ALL or SIF_DISABLENOSCROLL and not SIF_TRACKPOS;
    ScrollInfo.nMin := 0;
    ScrollInfo.nTrackPos := 0;
    ScrollInfo.nMax := cNbRows * 56; // Dont ask me why ~bk
    SetScrollInfo(WndHandle, SB_VERT, ScrollInfo, False);

    for i := 0 to cNbRows - 1 do
      for j := 0 to cNbColumns - 1 do begin
        lBtnLabel := Format('OK [%2d,%2d]', [i,j]);
        CreateWindow(
          'BUTTON', // lpClassName, optional
          @lBtnLabel[1],
          // 'OK ', // lpWindowName, optional
          WS_TABSTOP or WS_VISIBLE or WS_CHILD or BS_DEFPUSHBUTTON, // dwStyle
          20 + j * 90, // x
          40 + i * 60, // y
          80, // nWidth
          50, // nHeight
          WndHandle, // hWndParent
          0, // hMenu
          WndClass.hInstance, // hInstance
          nil  // lpParam
          );
      end;

    BringWindowToTop(WndHandle);

    repeat
      GMResult := GetMessage(Msg, 0, 0, 0);
      Inc(cMsgCounter);
      if not GMResult then begin
        WriteLn;
        break;
      end;
      {$IFDEF UseLogger}
      DbgOut('ShowWindow;');
      TextLen := GetWindowTextA(Msg.hwnd, @TextBuf[1], 255);
      SetLength(TextBuf, TextLen);
      DbgOut(TextBuf);
      DebugLn(';',IntToStr(cMsgCounter),';',IntToStr(Msg.hwnd),';',
              IntToStr(Msg.message),';',
              IntToStr(Msg.wParam),';',HexStr(Pointer(Msg.lParam)),';',
              WineMsgStr(Msg.message));
      {$ENDIF UseLogger}
      {
      if Msg.message = WM_PAINT then begin
        cPainting := True;
        repeat
          DispatchMessage(Msg);
        until not PeekMessage(Msg, 0, 0, 0, PM_QS_PAINT or PM_REMOVE);
        cPainting := False;
      end
      else
      }
      DispatchMessage(Msg);
      cPainting := False;
    until False;

    DestroyWindow(WndHandle);

    UnregisterClass(WndClass.lpszClassName, WndClass.hInstance);
  end;


begin
  ShowWindow;
  Write('Enter to quit > ');ReadLn;
end.






NativeScrollDemo-2.pas (6,550 bytes)   

Julian Puhl

2020-10-10 15:28

reporter   ~0126216

Last edited: 2020-10-10 16:04

View 2 revisions

How easily can this be implemented in a normal lazarus application? I compared it to my example program and to me it looks like it solves the problem.

Edit: I did some more testing and if I add way more rows (like e.g. 400 more) it starts to lock up my dwm.exe (windows 7) for some time when scrolling. It still is a lot better than my example program behaves, as this many buttons is quite unrealistic.

jamie philbrook

2020-10-10 16:57

reporter   ~0126217

I believe I experimented with this already and by simply purging the message que of old scroll messages and keeping only the newest one in the que it works fine.

  Basically performing a peakMessage and keeping only the newest in the que seems to work.

 but that is just my hack of the day and I have done that many times over the years with congested apps.

Julian Puhl

2020-10-10 21:45

reporter   ~0126227

So I finally had some time to look into this issue in detail. If you compare it to the example from Microsoft for bitmap scrolling you can see that they call UpdateWindow(hwnd) after ScrollWindowEx. I tried this in Martin's program and it fixes it. Attached is a patch for Lazarus which fixes all applications build with it. My test program runs fine after this patch and the program we had the problem in the first place now also is fixed.
win32wscontrols.pp.patch (567 bytes)   
Index: lcl/interfaces/win32/win32wscontrols.pp
===================================================================
--- lcl/interfaces/win32/win32wscontrols.pp	(revision 63988)
+++ lcl/interfaces/win32/win32wscontrols.pp	(working copy)
@@ -591,8 +591,11 @@
   DeltaX, DeltaY: integer);
 begin
   if AWinControl.HandleAllocated then
+  begin
     ScrollWindowEx(AWinControl.Handle, DeltaX, DeltaY, nil, nil, 0, nil,
       SW_INVALIDATE or SW_ERASE or SW_SCROLLCHILDREN);
+    UpdateWindow(AWinControl.Handle);
+  end;
 end;
 
 { TWin32WSDragImageListResolution }
win32wscontrols.pp.patch (567 bytes)   

Erik Rößiger

2020-10-10 23:45

reporter   ~0126229

I can second that, the initial issues when scrolling are now gone with this patch and there are no more artifacts and it rather looks as smooth as users would expect it. Tested on a Win10 machine.

jamie philbrook

2020-10-11 04:11

reporter   ~0126233

Does not work for me..

Although its a little improvement with the painting, it still locks up and forces me to wait for the app to release so it becomes responsive again..

and this is on Win10 i5 12G memory PC in 64 bit mode.

I am sure my I7 would do much better but face it, not everyone has a I7.

Julian Puhl

2020-10-11 10:35

reporter   ~0126234

Last edited: 2020-10-11 13:07

View 2 revisions

I think the locking up is a separate issue from Martin's test program. I still get that if I add many more rows, but there are no rendering issues anymore. I do not get that with my example. Did you try mine with the patched lazarus?

jamie philbrook

2020-10-11 14:33

reporter   ~0126236

it was late last night so I didn't get to finish the debugging but I tracked the issue down to the LCL WMSCROLL... message handling, that is where it gets congested during the SetSize or SetPosition, I can't remember now, its too early here.

 I tried your code of course, it does not do much in the way of improvements and I used your test app of course..,

 The fix you offered just offers more to the confusing and little change here.

 Anyways, I'll look later in the LCL code where the actual problem is. It behaves like there is a Application.PRocessMessages or the like being executed and allowing re-entry of code which can not be allowed.

Julian Puhl

2020-10-11 15:58

reporter   ~0126239

I was suspecting "The function sends a WM_PAINT message directly to the window procedure of the specified window, bypassing the application queue." (quoted from MS UpdateWindow documentation) might help with that issue, that is why I tested it. I only can tell you that our application plus my test project now do not have any issues on Windows 7 and 10 on the machines we tested it. Maybe there is another issue with very slow processors. And of course there also still might be a better solution.

Martin Friebe

2020-10-11 16:39

manager   ~0126240

There are 2 issues here.

The first issue is the partial paint. That can be fixed by forcing paint events. Of course doing so, will slow down the scrolling.

This may be related to
https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-scrollwindowex
<quote>
If the SW_SCROLLCHILDREN flag is specified, the system does not properly update the screen if part of a child window is scrolled. The part of the scrolled child window that lies outside the source rectangle is not erased and is not properly redrawn in its new destination.
</quote>

The first scroll would invalidate parts of the scrolled window (scrollbox or whatever). If there is NO paint then the next Scroll would only scroll the valid area. But child controls can be partly in that area. So that is indeed documented.
Calling "UpdateWindow" (or "RedrawWindow") will fix this. There may be other (maybe more efficient ways) but this is the easiest to implement).

The issue is, that those paint messages are skipped for a reason (AFAIK / not 100% sure):
If the next scroll message arrives before there was time to paint then it makes sense to first scroll again, and then do ONE accumulated paint.
1) maybe some of the unpainted area gets affected by the scroll, and no longer needs painting or requires different content to be painted (so squeezing the paint in, would paint something that would be thrown away anyway)
2) The accumulated paint may be faster than 2 individual paint / E.g., half a textline was scrolled, still needs a full draw of the text, and then clip it to the half line => done twice for each scroll / But after the 2nd scroll the text needs to be done just once.

The problem is, that the child controls are not properly invalidated, and therefore not painted in the accumulated paint.

 *** => So rather than forcing a paint immediately, one might try to add invalidation for those child controls, that may have been affected. <= ***
Determining, which controls to invalidate may however be quite some work.....
(And of course I have not tested this, I may be wrong entirely)

-----------------------------------------------------------------------
The second issue:

In my original code, set the count of buttons (the 2 nested loops) to

for i := 0 to 500 do
for j := 0 to 10 do begin

After you scrolled (grab the scroll bar knob, and move it up and down (without releasing the mouse button) for maybe 5 to 10 seconds)
your entire desktop will be frozen for a minute or two.

One CPU core will run at 100% (but your app is idle). The windows kernel is doing some obscure work.

This freezing is not fixed by adding UpdateWindow.

jamie philbrook

2020-10-11 18:32

reporter   ~0126241

I inserted this in the ScrollBy widget for a test after the scrollWindowEx call in the widget..

AWinControl.Reapint.

No more locking up for me and the repaint isn't that bad. it does fully repaint with maybe a slight delay but it gets fully repainted while scrolling.
I can offer up an actual Patch file if that is desirable .

class procedure TWin32WSWinControl.ScrollBy(const AWinControl: TWinControl;
  DeltaX, DeltaY: integer);
begin
   If AWinControl.HandleAllocated Then
    ScrollWindowEx(AWinControl.Handle, DeltaX, DeltaY, nil, nil, 0, nil,
     SW_INVALIDATE or SW_ERASE or SW_SCROLLCHILDREN);
   AWinControl.Repaint; // Added this JP.
end;

Martin Friebe

2020-10-11 20:07

manager   ~0126243

Interestingly enough in the LCL code, forcing the repaint makes the entire scrolling faster.
Not sure why under the LCL this does improve speed (it is counter expectations....)

On an I7 8700X the locking up does not happen with the 50 rows from the example (well, if I scroll a hell lot,then it does).
But increase this to 300 rows, and it happens regardless of the "Repaint"

jamie philbrook

2020-10-11 20:46

reporter   ~0126244

Windows puts a hold on apps that has too many paint messages in the que and if you don't get them purged or at least doing something like
calling the BeginPaint and EndPaint, the GDI becomes non responsive because it is put at the lowest of the priority list. Non client controls normally will still behave because those are painted Via the OS., like the Scrollbars etc.

a TButton is a GUI handle on its own so that is a lot of handles that needs updates.

Maybe a Clipping region can be applied but can't be sure if that would stop the request of painting controls that are not visible

jamie philbrook

2020-10-11 21:17

reporter   ~0126246

This combination seems to be smoother with the test app supplied here.
With UpdateWindow and the forced paint.. see below..
class procedure TWin32WSWinControl.ScrollBy(const AWinControl: TWinControl;
  DeltaX, DeltaY: integer);
begin
   If AWinControl.HandleAllocated Then
   Begin
    ScrollWindowEx(AWinControl.Handle, DeltaX, DeltaY, nil, nil, 0, nil,
     SW_INVALIDATE or SW_ERASE or SW_SCROLLCHILDREN);
     UpdateWindow(AWinControl.Handle); //added Jp
     AWinControl.Repaint; //added jp
   End;
end;

Martin Friebe

2020-10-12 21:29

manager   ~0126267

I have to do some more tests myself.

I currently consider something like

class procedure TWin32WSWinControl.ScrollBy(const AWinControl: TWinControl;
  DeltaX, DeltaY: integer);
begin
  if AWinControl.HandleAllocated then begin
    ScrollWindowEx(AWinControl.Handle, DeltaX, DeltaY, nil, nil, 0, nil,
      SW_INVALIDATE or SW_ERASE or SW_SCROLLCHILDREN);
    if AWinControl.ControlCount > 0 then
      UpdateWindow(AWinControl.Handle);
  end;
end;

After all, the issue only exists if there are child controls.
And the doc on msdn - at least to my understanding - describes it the same way.

--
Also Repaint is overkill.
Repaint translates to InvalidateRect(ClientRect); UpdateWindow(Handle);
So repaint would do a lot of extra painting.

On Win10 64 bit, adding the UpdateWindow seems to (unexpectedly) even improve timings (not just painting).
I have to see how this works on other versions of windows.

--------------
As for the desktop hang after the scrolling -> that remains the same.

jamie philbrook

2020-10-12 22:12

reporter   ~0126271

Last edited: 2020-10-13 05:40

View 3 revisions

I've read the ms docs over and over. done this for years ;) windows 10 does not behave the in the same manner as the older oses do. it has a lot to do with themes and double buffering..

 also MS has limited the number of congested non response of GUI messages to prevent stall out for other parts of the system so it's easy now to see this behavior..

  in my old Delphi, when ever there is a scrollby all the controls are updated immediately in the same call block and there is obviously a reason for that.

  so I prefer to keep the calls of updatewindow and repaint because it prevents lockups and repaints fast on the scrolls.

 otherwise there Is a delay set by windows of when it decides to send a paint message.

Edit :
  This is pretty sad, I just did 2500 controls with D3 in windows 2000 and other than starting up a little slow, the controls scrolled perfectly with very little artifacts while smooth scrolling and finished perfectly, no delays or lock ups.

Edit2:
 for the time being, I've concluded this must be a windows bug or intentional slow down because I've found lots of references about random self scrolling forms and also found references after updates of forms locking up just like this in windows 10 with a large count of controls on it.
 i moved the test app to my windows 2000 machine which only has 500k mem on it, its old. using Lazarus 2.0.0 , it compiled and ran the app fine with 2000+ controls on it.

i'll dig more to see if there is a way to stop windows 10 from doing this but i don't hold much hope.

Issue History

Date Modified Username Field Change
2020-08-01 10:11 Julian Puhl New Issue
2020-08-01 10:11 Julian Puhl File Added: TestDoubleBuffer.zip
2020-08-01 10:11 Julian Puhl File Added: DoubleBufferError.jpg
2020-08-01 13:08 wp Note Added: 0124458
2020-08-01 13:54 Julian Puhl Note Added: 0124459
2020-08-01 17:03 Martin Friebe Note Added: 0124461
2020-08-01 17:12 Martin Friebe Note Edited: 0124461 View Revisions
2020-08-01 17:38 Martin Friebe Note Edited: 0124461 View Revisions
2020-08-01 18:21 Martin Friebe Note Edited: 0124461 View Revisions
2020-08-01 23:38 jamie philbrook Note Added: 0124473
2020-08-01 23:44 jamie philbrook Note Edited: 0124473 View Revisions
2020-08-01 23:48 wp Note Added: 0124476
2020-08-01 23:48 wp File Added: TestDoubleBuffer_v20.zip
2020-08-01 23:49 wp Note Edited: 0124476 View Revisions
2020-08-01 23:51 Martin Friebe Note Added: 0124477
2020-08-02 00:29 jamie philbrook Note Added: 0124479
2020-08-02 12:29 BrunoK Note Added: 0124487
2020-08-02 14:34 jamie philbrook Note Added: 0124492
2020-08-02 15:47 Martin Friebe Note Added: 0124497
2020-08-02 15:54 Martin Friebe Note Added: 0124500
2020-08-02 16:35 Julian Puhl Note Added: 0124507
2020-08-02 16:38 Julian Puhl Note Edited: 0124507 View Revisions
2020-08-02 16:39 Julian Puhl Note Edited: 0124507 View Revisions
2020-08-02 16:39 Julian Puhl Note Edited: 0124507 View Revisions
2020-08-02 17:21 wp Note Added: 0124509
2020-08-02 18:33 BrunoK Note Added: 0124511
2020-08-02 19:46 Martin Friebe Note Added: 0124513
2020-08-02 19:46 Martin Friebe File Added: NativeScrollDemo.pas
2020-08-03 02:34 jamie philbrook Note Added: 0124521
2020-08-13 16:53 BrunoK Note Added: 0124845
2020-08-13 16:53 BrunoK File Added: NativeScrollDemo-2.pas
2020-10-10 15:28 Julian Puhl Note Added: 0126216
2020-10-10 16:04 Julian Puhl Note Edited: 0126216 View Revisions
2020-10-10 16:57 jamie philbrook Note Added: 0126217
2020-10-10 21:45 Julian Puhl Note Added: 0126227
2020-10-10 21:45 Julian Puhl File Added: win32wscontrols.pp.patch
2020-10-10 23:45 Erik Rößiger Note Added: 0126229
2020-10-11 04:11 jamie philbrook Note Added: 0126233
2020-10-11 10:35 Julian Puhl Note Added: 0126234
2020-10-11 13:07 Julian Puhl Note Edited: 0126234 View Revisions
2020-10-11 14:33 jamie philbrook Note Added: 0126236
2020-10-11 15:58 Julian Puhl Note Added: 0126239
2020-10-11 16:39 Martin Friebe Note Added: 0126240
2020-10-11 18:32 jamie philbrook Note Added: 0126241
2020-10-11 20:07 Martin Friebe Note Added: 0126243
2020-10-11 20:46 jamie philbrook Note Added: 0126244
2020-10-11 21:17 jamie philbrook Note Added: 0126246
2020-10-12 21:29 Martin Friebe Note Added: 0126267
2020-10-12 22:12 jamie philbrook Note Added: 0126271
2020-10-13 01:01 jamie philbrook Note Edited: 0126271 View Revisions
2020-10-13 05:40 jamie philbrook Note Edited: 0126271 View Revisions