View Issue Details

IDProjectCategoryView StatusLast Update
0037463LazarusLCLpublic2020-08-03 02:34
ReporterJulian Puhl Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
PlatformWin64OSWindows 
Product Version2.1 (SVN) 
Summary0037463: Double buffering seems not to be working on windows
DescriptionI noticed this when working with many components in a form. Attached is a test project and a screenshot which demonstrates the issue. If you scroll, you get a blurry mess. This seems not to happen with QT or GTK.
Steps To ReproduceScroll to get a blurry mess.
Additional InformationCompiler is FPC trunk win64.
TagsNo tags attached.
Fixed in Revision
LazTarget
WidgetsetWin32/Win64
Attached Files

Activities

Julian Puhl

2020-08-01 10:11

reporter  

TestDoubleBuffer.zip (107,389 bytes)
DoubleBufferError.jpg (61,487 bytes)   
DoubleBufferError.jpg (61,487 bytes)   

wp

2020-08-01 13:08

developer   ~0124458

I am not sure whether DoubleBuffered is the correct screw to turn in order to fix this issue. The problem is that when the scrollbars' hot-tracking is active there is an immense amount of redraw requests in the message queue. Writing to a bitmap first only adds another delay. I noticed in many cases, also with delphi, that DoubleBuffering makes the application slower.

You can avoid the drawing artifacts by turning off Tracking of both scrollbars. You can also add a client-aligned Scrollbox to the form into which the buttons are put. Interestingly, this combination results in much smoother scrolling. There is some time after the scrolling mouse button is released during which the application is not responsive. In both cases. (Move the mouse over the 'x' in the title bar and notice the delay until the 'x' becomes red (on Win 10).) there is also a noticable delay when the application is closed; this can be avoided when the AutoScroll property of Form or Scrollbar is set to false.

Julian Puhl

2020-08-01 13:54

reporter   ~0124459

I see. I did some testing before reporting this and for me it does not make a difference whether I use a Scrollbox, the scrollbars from the form, or enable/disable double buffering. Also if I set Autoscroll to false, I can't get the scrollbars to get working (tried a few method calls but it has no effect. Visible flag for each scrollbar is set to true).

Since I need the tracking (otherwise scrolling in an application is very annoying), would it be possible to reduce the number of messages in the queue (e.g. only sent it every 10ms)? Somehow QT and GTK do not have a problem with this. And I also never have seen this behaviour in any normal windows application.

Martin Friebe

2020-08-01 17:03

manager   ~0124461

Last edited: 2020-08-01 18:21

View 4 revisions

First of all, when testing: There is a different whether the child controls are TWinControl or TGraphicControl descendents.

If the scroll-container (form/scrollbox) is/were double-buffered, then this affects only items that the scroll-container draws. I.e. its own design and contained TGraphicControls.
Contained TWinContrals are (on Win, but probably most others too) never drawn by the scroll-container. They have there own OS-handle, and always draw themself.

So DoubleBuffer is not the full solution to "scroll mess".

----
What needs to happen (1st step, maybe that already happens) is: ScrollWindowEx (though if the scrolling is done by Windows, this is not something we can influence / but then it may be done correct already.)
If the scrolling is handled by the LCL, this call can tell the OS, that all the child controls should be moved. If everything is visible on screen, this should work, even without DoubleBuffering.


However this will send WM_Move messages...... And IIRC the LCL may do an invalidate of the control for which it receives them.

That is if the app receives WM_Move as result from a scroll, the Top/Left of the contained TWinControl are still at the old pos.
WM_Move causes a call to SetTop/SetLeft => and that will trigger various calls to invalidate for either the contained or the containing or both controls...... Those calls should not happen.

Those calls are in code affecting ALL widgetsets. => To fix the issue, they may need to be redirected into Widgetset code for each widgetset.....

---------------
So there is a good deal of analysis/debugging needed, to see where it goes wrong.


------------------------
EDIT:
Actually I am partly mistaken: WM_move does not call SetTop/SetLeft.

So not sure what causes it.
But even with double-buffer, any control that scrolls in, must have a paint event (even if it just copies the buffer). So if that paint event is delayed...

------------------------
EDIT:
I looked at your test.
On my System artefacts only happen for Buttons scrolling in. They must be painted once, even if double buffered.
If too many buttons scroll in, then the paint events do not happen fast enough.

I am not sure what slows them down...

One guess:
The form does get a paint event, on each pixel scroll,=> in case it needs to draw TGraphicControls. In your case there is nothing to be drawn.
But on each such event the form paint handler needs to loop the 500 buttons (check each for being a TGraphicControl, but none is) => I guess that could be what makes the paint event rather slow.

However, skipping those loops does not help

jamie philbrook

2020-08-01 23:38

reporter   ~0124473

Last edited: 2020-08-01 23:44

View 2 revisions

using 2.0.10 I can not open that project.. ?
why is that ?

 If even locks up the IDE and crashes it when I attempt to open it.

wp

2020-08-01 23:48

developer   ~0124476

Last edited: 2020-08-01 23:49

View 2 revisions

Always the same story: The Lazarus xml file formats were changed in trunk which makes projects and packages created by trunk unreadable by older versions, unless the "Maximize compatibility" boxes are checked in the project and package options. I am adding the project in a version for 2.0.x in which the "maximize" boxes were set.

Martin Friebe

2020-08-01 23:51

manager   ~0124477

If the project was saved in trunk, it may have a new format. / Remove the lpi/lps, open the lpr, and confirm "open as project" then create a new application type project when the IDE ask.
////

From what I can see, hooking WindProc, the Buttons are entirely drawn by the OS.
At least I did not see any WMPaint for them (only BN_UNPUSHED )

I currently can't test with other frameworks, if they could handle 500 Buttons.

If you use TSpeedButton (graphic control) scrolling should be fine (it is on my PC)

jamie philbrook

2020-08-02 00:29

reporter   ~0124479

ok, That looks like a windows GUI priority override.

Maybe this test should be done on a older OS..

On one of my i5 Windows 10-pro 64 bit EXE it actually LOCKS the form update for short times, there is no response from mouse of key input.
 
 So the window was hooked ? so how was this done btw using the SetWindowsHookEx ?

 I do believe the WM_PAINT message does arrive and maybe its in the Widget handler ?

 that being the case maybe a Rectangle test to see If it is in view and if not return the message has handled so windows does not attempt to paint a control needlessly .

BrunoK

2020-08-02 12:29

reporter   ~0124487

Win10.

What happens is that scrolling messages (mouse messages if I remember well) are processed before a WM_PAINT batch has totally finished to display a refreshed screen. The handling of the mouse message generates an Invalidate (Windows or Lazarus I don't know).
That induces Windows to terminate prematurely the WM_PAINT batch of Windows hence the striped appearance while scrolling and the settling at the end of mouse slider action.

I have never figured out what DoubleBuffering was useful for in Windows since the doublebuffer exists for the duration of a single WindowProc in \lcl\interfaces\win32\win32callback.inc Maybe a "professional" should review that.

Also I think I remember that QT5 does it differently but that is a 2 years old from memory observation.

WP : The effect can also be seen when dragging the Map in lamapviewer where it is aggravated by the multithreading nature of the application.

jamie philbrook

2020-08-02 14:34

reporter   ~0124492

I did some debugging last night and found that in the widget for the ButtonControl window procedure the WM_PAINT message is being delivered.
  
  I conducted a little experiment by returning the Procedure as HANDLED when ever the WM_PAINT and WM_ERASEBKGRD
was received so not to actually draw the buttons..
    
 That resulted in the forms background still drawing the outlines of the buttons and the same delay lag and smearing display exists ..

Also it seems windows knows enough to not paint or erase any controls that are not in the viewable area, that has been confirmed too by testing the controls boundaries of those that arrived at the Procedure.

 It looks like it could be painting issues related to the Parent of these controls and that would be the form, the TscrollingWinControl I would think ?

Martin Friebe

2020-08-02 15:47

manager   ~0124497

>> "So the window was hooked ? so how was this done btw using the SetWindowsHookEx ?"
Why ?

The LCL, when creating a control, passes a callback: WindowProc (in CreateParams). This is necessary, so the OS can sent updated to the LCL.
The LCL reacts to those messages, and passes them on the original handler of the OS>

For most WinControls, that includes that the OS sends WM_Paint.
- In case of a form, the WM_Paint would paint any TGraphicControls.
- In case of TButton, I did not see any WM_Paint message (Which would be in line, that you can not custom-paint buttons, or change the back-color)
  So as far as I can see button paint is handled by the OS, without telling the LCL.

In how for any action done in any other msg to WindProc affects this painting... I have no idea.

Martin Friebe

2020-08-02 15:54

manager   ~0124500

@BrunoK Any comments to the proposed changes?

Julian Puhl

2020-08-02 16:35

reporter   ~0124507

Last edited: 2020-08-02 16:39

View 4 revisions

It is nice to see this bug getting so much attention. I can attach a QT5.6.2 (32bit) build with all the dlls if it is allowed and could help (there seems to be no 64 bit support).

Btw. the drawing bug also appears with only two columns. I just exaggerated it for a better demonstration.

Sadly I do not have much experience with windows GUI handling, so I can't be of any help debugging this.

wp

2020-08-02 17:21

developer   ~0124509

In tried the demo program in a VM with Win7, and here it seems that scrolling occurs smoothly (although sometimes partially drawn buttons are visible here, too). This reminds me of that discussion about Windows 10 starting slowly after a particular update (https://bugs.freepascal.org/view.php?id=33705) in which you were involved BrunoK.

BrunoK

2020-08-02 18:33

reporter   ~0124511

WP : The difficult one was to get 0033923 accepted later (6 months) but I navigated today to approximately the same areas of the source.
If I have time and inspiration (an backups ...), I will try to retool win32callback.inc with a message tracker and see if anything surfaces.

I still think that WIN does come from another thread and initiates mouse messages that interfere with the running paint but without 'instrumenting' the code it is difficult to be to affirmative. Maybe base paint coordinates / offsets of the CustomForm canvas.

For the current glitch I have to correct my previous NEGATIVE comment about DoubleBuffered, it seems to work correctly for exactly the present test program.

Martin Friebe

2020-08-02 19:46

manager   ~0124513

I attached a small sample ("simple program") that creates a form with buttons, directly via win API. No LCL.
It does not remember the final scroll pos.
I obviously do not handle enough messages. But then, that means minimal slowdown.

It works up 150 or 200 buttons. But then things start going south.

The first issue is, that even if scrolling looks still fine, CPU load goes up AND stays UP several seconds after scrolling ended. (leaving the PC partly locked)
NativeScrollDemo.pas (3,026 bytes)   
program NativeScrollDemo;
{$mode objfpc}{$H+}

uses
  Classes, windows, LazLogger;

var
  ScrollInfo: TScrollInfo;
  WndHandle: HWND;
  lastpos: integer;


function WindowWndProc(Ahwnd: HWND; uMsg: UINT; wParam: WParam; lParam: LParam) : LRESULT; stdcall;
var
  p: Word;
begin
  Result := 0;
  Case uMsg  Of
    WM_DESTROY : PostQuitMessage(0);
    WM_VSCROLL: begin
      p := HIWORD(LongInt(WParam));

      ScrollWindowEx(WndHandle, 0, lastpos - p, nil, nil, 0, nil,
        SW_INVALIDATE or
        SW_ERASE or
        SW_SCROLLCHILDREN
      );
      lastpos := p;

      //ScrollInfo.cbSize := SizeOf(ScrollInfo);
      //ScrollInfo.fMask := SIF_ALL or SIF_DISABLENOSCROLL and not SIF_TRACKPOS;
      //ScrollInfo.nMin := 1;
      //ScrollInfo.nTrackPos := p;
      //ScrollInfo.nMax := 3000;
      //SetScrollInfo(WndHandle, SB_VERT, ScrollInfo, False);

      Result := 0;
    end;
    Else
      Result := Windows.DefWindowProc(ahwnd, uMsg, wParam, lParam);
  End;
end;



procedure ShowWindow;
Var
  A_Atom : TAtom = 0;
  WndClass : TWndClass;
  Msg: TMsg;
  ScreenWidth, ScreenHeight, MiddleX, MiddleY : LongInt;
  i, j: Integer;
Begin
  lastpos := 0;
  FillChar(WndClass, SizeOf(TWndClass), 0);

  ScreenWidth := GetSystemMetrics(SM_CXSCREEN);
  ScreenHeight := GetSystemMetrics(SM_CYSCREEN);

  MiddleX := (ScreenWidth - 500) Div 2;
  MiddleY := (ScreenHeight - 500) div 2;

  WndClass.lpszClassName:= 'HEAPTRACE_CLASS';
  WndClass.lpfnWndProc :=  @WindowWndProc;

  WndClass.hInstance := hInstance;
  WndClass.hbrBackground:= 1;
  WndClass.style := CS_HREDRAW or CS_VREDRAW;
  WndClass.hCursor := LoadCursor(0, IDC_ARROW);

  A_Atom := RegisterClass(WndClass);

  WndHandle := CreateWindow(
   WndClass.lpszClassName , // lpClassName, optional
   'Foo',
   WS_OVERLAPPEDWINDOW or WS_VISIBLE or WS_VSCROLL , // dwStyle
   MiddleX, // x
   MiddleY, // y
   500, // nWidth
   500, // nHeight
   0, // hWndParent
   0, // hMenu
   WndClass.hInstance, // hInstance
   nil  // lpParam
   );


  ScrollInfo.cbSize := SizeOf(ScrollInfo);
  ScrollInfo.fMask := SIF_ALL or SIF_DISABLENOSCROLL and not SIF_TRACKPOS;
  ScrollInfo.nMin := 1;
  ScrollInfo.nTrackPos := 0;
  ScrollInfo.nMax := 3000;
  SetScrollInfo(WndHandle, SB_VERT, ScrollInfo, False);


for i := 0 to 50 do
for j := 0 to 3 do begin
  CreateWindow(
   'BUTTON' , // lpClassName, optional
   'OK', // lpWindowName, optional
   WS_TABSTOP or WS_VISIBLE or WS_CHILD or BS_DEFPUSHBUTTON , // dwStyle
   20 + j * 60, // x
   40 + i * 60, // y
   50, // nWidth
   50, // nHeight
   WndHandle, // hWndParent
   0, // hMenu
   WndClass.hInstance, // hInstance
   nil  // lpParam
   );
end;

  BringWindowToTop(WndHandle);

  while GetMessage(Msg,0,0,0) do begin
//DbgOut(',');
    DispatchMessage(Msg);
  end;

  DestroyWindow(WndHandle);

  UnregisterClass(WndClass.lpszClassName, WndClass.hInstance);
end;


begin
  ShowWindow;
end.

NativeScrollDemo.pas (3,026 bytes)   

jamie philbrook

2020-08-03 02:34

reporter   ~0124521

I did some experimentation and what I did was during a scroll message I checked the message buffer for other scroll messages waiting and
if found extracted those (Removed) and use the newest one instead.
  I had to do a little math but the results were interesting...
there seems to be a flood of messages in the que that needs to be dealt with....

 Still testing..

Issue History

Date Modified Username Field Change
2020-08-01 10:11 Julian Puhl New Issue
2020-08-01 10:11 Julian Puhl File Added: TestDoubleBuffer.zip
2020-08-01 10:11 Julian Puhl File Added: DoubleBufferError.jpg
2020-08-01 13:08 wp Note Added: 0124458
2020-08-01 13:54 Julian Puhl Note Added: 0124459
2020-08-01 17:03 Martin Friebe Note Added: 0124461
2020-08-01 17:12 Martin Friebe Note Edited: 0124461 View Revisions
2020-08-01 17:38 Martin Friebe Note Edited: 0124461 View Revisions
2020-08-01 18:21 Martin Friebe Note Edited: 0124461 View Revisions
2020-08-01 23:38 jamie philbrook Note Added: 0124473
2020-08-01 23:44 jamie philbrook Note Edited: 0124473 View Revisions
2020-08-01 23:48 wp Note Added: 0124476
2020-08-01 23:48 wp File Added: TestDoubleBuffer_v20.zip
2020-08-01 23:49 wp Note Edited: 0124476 View Revisions
2020-08-01 23:51 Martin Friebe Note Added: 0124477
2020-08-02 00:29 jamie philbrook Note Added: 0124479
2020-08-02 12:29 BrunoK Note Added: 0124487
2020-08-02 14:34 jamie philbrook Note Added: 0124492
2020-08-02 15:47 Martin Friebe Note Added: 0124497
2020-08-02 15:54 Martin Friebe Note Added: 0124500
2020-08-02 16:35 Julian Puhl Note Added: 0124507
2020-08-02 16:38 Julian Puhl Note Edited: 0124507 View Revisions
2020-08-02 16:39 Julian Puhl Note Edited: 0124507 View Revisions
2020-08-02 16:39 Julian Puhl Note Edited: 0124507 View Revisions
2020-08-02 17:21 wp Note Added: 0124509
2020-08-02 18:33 BrunoK Note Added: 0124511
2020-08-02 19:46 Martin Friebe Note Added: 0124513
2020-08-02 19:46 Martin Friebe File Added: NativeScrollDemo.pas
2020-08-03 02:34 jamie philbrook Note Added: 0124521