All Projects FPC fpcprojects fpprofilerfpGUI Lazarus Packages PatchesLazarus CCR Mantis pas2js

View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0019812LazarusLCLpublic2011-07-26 08:272014-09-15 22:36
ReporterReinhard Berger Assigned ToZeljan Rikalo  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
Product Version0.9.30.1 (SVN) 
Summary0019812: Selecting items in listview is extremly slow
DescriptionWhen selecting entrys (either from code or manualy by keyboard) in a listview and the listview holds alot of entrys (say for example 2.000 or more) then time for selecting is extremy long (e.g. tested with 14.000 entrys it takes 1-2 hours !!!)

When compared with WIN32 there it takes only 1-2 secounds.

This happens on QT and GTK2 as well.

Note : I already call listview.items.BeginUpdate and Listview.items.EndUpdate
TagsListView
Fixed in Revision43244
LazTarget1.4
WidgetsetGTK 2, QT
Attached Files

Relationships

related to 0019094 resolvedZeljan Rikalo Event on SelectItem not triggered when ListView.Items[x].Selected is called 
related to 0020631 closedZeljan Rikalo Listview Items not selected in GTK2 by using BeginUpdate and EndUpdate 

Activities

Mike Thompson

2011-07-27 05:24

developer   ~0050193

Last edited: 2011-08-08 04:00

 Items.BeginUpdate and Items.EndUpdate are useful when adding multiple entries. I'm sure they don't apply when changing selection. <CORRECTION> BeginUpdate/EndUpdate should work with changing selection - it just doesn't in GTK2 at the moment due to other selection issues...</CORRECTION>

I'm unsure why you're trying to select 14 thousand different entries in less than 2 seconds. That kind of UI feedback isn't useful for the end user. OTOH If you're just enumerating over the list items, then you don't need to call select. OTOH2 if you mean you're trying to select a single item in a list view with 14,000 entries in it, then on my PC (GTK2, Ubuntu, Laz 0.9.30) I can change selection in a list view with 100,000 entries in it (tested with mouse, keyboard and code) without noticing any delay at all.

Can you post the code for a sample project and I'll have a look at it?

Reinhard Berger

2011-07-30 18:27

reporter   ~0050289

Well you can use the same demo code as for the issue 0019094

just change in "procedure TForm1.FillClick(Sender: TObject)"

For loop:=0 to 2 into : For Loop:=0 to 2.000

2.000 is enough for testing, otherwise you'll wait a long time.

Messure the time needed with 2.000 files and messure the time then next when you comment out the line : "ListItem_Source.Selected := True"

Speed is improved 100% under GTK2 / QT

Under Win32 selecting a amount of 2.000 files takes just a half secound, under QT/GK2 it takes 40 sec.

If you then do more then 2.000 this goes to hours of waiting time just for selecting the items.

First i tought it is maybe the X-Server, but compared with dolphin and 20.000 files it just takes in dolphin also only a few secounds and not hours as in lazarus QT.

My personal feeling is also somehow the time needed to draw things in QT has been much slower now as in previous version of lazarus.

For example draw a progressbar during file reading slows down reading of file 20-30%.

If you set progressbar to marquee it reads the same file much faster.

Again this does not happen under Win32 and as far as i can remember me also not in 0.9.28.

I'll test this today a bit later under 0.9.28 and see whats happen there.

Reinhard Berger

2011-07-31 14:19

reporter   ~0050304

tested now the following code under 0.27 and 0.9.31 under QT since GTK2 does not allow items to be selected currently under 0.9.31.

ListView1.Items.BeginUpdate;
  For i:=0 to 2000 do
  Begin
      alistitem:=Listview1.Items.Add;
      alistitem.Selected:=True;
      alistitem.caption:='test-'+IntToStr(i);
  end;
  ListView1.Items.EndUpdate;


This code takes about 10 sec under 0.9.27 and about 1 min under 0.9.31
compoared to win32 where it takes maximal 1 sec or less.

Reinhard Berger

2011-07-31 15:59

reporter   ~0050310

Additional Note :
Tested 0.9.27 with GTK2 :

Result is like the same as Win32.

The 2.0000 Items got selected within 1 sec.

QT needs with 0.9.27 around 10 sec.

Mike Thompson

2011-08-06 03:17

developer   ~0050525

Running the following code on a ListView with MultiSelect := True.

  For i := 0 To ListView.Items.Count-1
    ListView.Items[i].Selected := True;

TListView (GTK2) in vsReport or vsList is extremely quick. With the OnSelect disconnected, I've just selected 5000 records in around 100msec.

However, in vsIcon or vsSmallIcon it's extremely slow, and by that I mean 17 seconds on my PC.

With an OnSelect handler connected, this took minutes to run, and that was with the ListView in either mode. My OnSelect handler added a single line to a TMemo and updated the status bar. Look for the forums for the mechanism to disconnect the OnSelect (search for ListView SelectAll)

I'll see what optimizations can be implemented, but I suspect the practical solution here may be the implementation of a ListView.SelectAll. I've confirmed both Win32 and GTK2 support trivial mechanisms for selecting and unselecting all items that are blindingly fast. I'm only guessing, but there may be similar API calls on other widget sets.

Mike Thompson

2011-08-06 06:42

developer   ~0050528

Reinhard - I simply cannot reproduce the extreme slowdowns you've reported. There are four projects at the issue you pointed to, which is it you want me to look at?

Reinhard Berger

2011-08-07 11:01

reporter   ~0050580

Last edited: 2011-08-07 11:41

 I've uploaded a litte demo program so that you can see it what i mean.

Also i include some binarys QT/GTK2/WIn32. (All 32 bit)

Test result for GTK2 is missing, since in GTK2 the selection of items isnt working, thereforce GTK2 is much more faster as QT.

Slecting 2.000 items in listview style VsReport : Win32 (Vbox) less 1 sec / QT : 11-12 secounds / WIN32 (Wine 1.3.18) 6 secounds / GTK2 1 secound (no selected items)

Also added binary compiled under 0.9.28.2 GTK2.

The 2.000 entrys got selected there in less then 1 sec - same as under Win32


Selecting 2.000 items in listview style VsIcon / VsIcon : Win32 (Vbox) less 1 sec / QT : 2-3 secounds / Win32 (Wine 1.3.18) 6 secounds / GTK2 1 secound (no selected items)

Also QT/GTK2 seems to do display the items in mode VsList diffrent as under Win32 (1 row instead of 2 )

2011-08-07 11:02

 

ListViewGTK2-QT.zip (94,132 bytes)

2011-08-07 11:06

 

listview -GTK2 -VsReport -0.9.31.tar.gz (1,606,398 bytes)

2011-08-07 11:07

 

listview -QT -VsReport-0.9.31.tar.gz (1,048,311 bytes)

2011-08-07 11:08

 

listview -Win32-VsReport -0.9.31.exe.tar.gz (733,164 bytes)

2011-08-07 11:38

 

listview GTK2 -VsReport.0.9.28.2.tar.gz (1,139,189 bytes)

Mike Thompson

2011-08-08 03:53

developer   ~0050600

Thanks - downloading now.
I've played with vsList. With GTK2 there is simply no control that will reproduce the same behaviour (multiple columns) as in the Win32 control, so all the original programmers have done is use the same control as vsReport and remove the columns. Best they could do....

Zeljan Rikalo

2011-10-30 08:56

developer   ~0053621

Last edited: 2011-10-30 09:23

 hm... with your example + setted up as vsReport this is result (5000 items).
1.Gtk2: 73 msec
2.Qt: 44501 msec

ugly. I'll try to optimize it a bit , but do not forget that pure qt app shows huge slowdown when dealing with itemviews. I've already mentioned that few years ago on qt lists that itemview design is bad, so any list (QListWidget, QTreeWidget, QTableView) is *very* slow when dealing with more of 1000 items.
because of this I'm using virtualtreeview in my apps, and I'm using TListView maybe in 5-6 forms for some small amount of data (max 50 items).

Update: without Selected:=True qt returns 5000 items in 67 msec and gtk2 in 70 msec .... I'll try to optimize signaling.

Update2: No way.This is it. Qt sends selectionChanged on each selection change via LCL. So when signaling is disabled it takes 32 seconds instead of 44.
So, pure qt app can load 5000 items faster than gtk2 (67 vs 70 msec), but problem is when you set selection at each of items when loading.
You said that you tested with dolphin, but it's innacurate measurement because:
1.Dolphin uses kdelibs, which have their own QAbstractItemView implementation which is faster than original one.
2.You load files into Dolphin (by eg. changing directory), but you didn't and you can't change selection of that files ?

2011-10-30 10:38

 

pureqtapp.tar.gz (131,044 bytes)

Zeljan Rikalo

2011-10-30 10:41

developer   ~0053623

@Reinhard, I've uploaded pureqtapp example (it uses only qt4 and libqt4pas),
 so you can see that it's qt design problem.
 Try to comment code which does selection inside loop and you'll see that loading
 is fast, but selecting each item while adding is PITA.
 I won't fix anything regarding this problem under qt. If you need things faster, then prepare an optimization patch which won't kill current correct selection behaviour (to be lcl compatibile).

Reinhard Berger

2011-10-31 13:36

reporter   ~0053680

maybe there is a command as "Mike" described before that selects all items in a listview/treeview at once without sending a signal for each item.


I know that when listview.item.selected:=True is commented out loading goes extremly fast, but unfortunaly i need to select the items.

It doesnt matter currently if you select each item during adding, or later when the items are filled already.

The time of selecting the items is always the same.

I'll try virtual Treeview and see if it deals better with selecting, but i can't think that since selecting a item will always be a lcl (QT / GTK2 ) call even if they programm the element of its own, or does they draw the selection of thier own ?

Zeljan Rikalo

2011-10-31 15:17

developer   ~0053685

No there's no command in LCL which selects all items without signal processing.
IMO, it would be nice feature, but not only that, also ListView.Clear should be in WS ,so no iteration for each item handle of ws, just clear in one hit.
VirtualTree does this with speed of light :) You cannot compare it with listviews from gtk and especially from qt which is slow.
I didn't understand your last sentence about selections.
1.When you ask lcl to select an item it calls WS which selects that item and sends signals.In case of listviews signals are always passed ,so OnSelectItem will trigger by mouse lick or by calling Item.Selected := True;
2.ListViews are native controls and they're painted by widgetset unless you use OwnerData.

Zeljan Rikalo

2012-02-09 14:32

developer   ~0056687

Update on my comments: To get OwnerData work fast we must make changes in all widgetsets so WS handles only with num of visible items (so de-facto ws knows only for visible items and they're allocated as items). Paint method of ws should do the trick and selectAll and such stuff will be very fast.
This can be done after 1.0 is released.

Zeljan Rikalo

2013-10-09 22:11

developer   ~0070708

I'll implement SelectAll and ClearSelection (which are in Delphi TCustomListView).
http://docwiki.embarcadero.com/VCL/XE/en/API:ComCtrls.TCustomListView.SelectAll
Seem that we need it to be delphi compatibile, and yes, under Qt selectAll() works at least twice faster than selecting item by item. Gtk2 supports selectall and I guess that's true for win32 too.
I'm only afraid of reaching limit in ws.

Zeljan Rikalo

2013-10-09 23:16

developer   ~0070711

Yes, adding SelectAll to WS passes VIRTUAL_VMT_COUNT and lcl crashes ... so I'll try to discuss with developers about increasing VIRTUAL_VMT_COUNT and eventual colateral damage.

Zeljan Rikalo

2013-10-14 09:36

developer   ~0070809

Implemented TCustomListView.SelectAll and TCustomListView.ClearSelection.
Now qt is fastest, then win32 and then gtk2.
Please test and close if ok.

Issue History

Date Modified Username Field Change
2011-07-26 08:27 Reinhard Berger New Issue
2011-07-26 08:27 Reinhard Berger Widgetset => GTK 2, QT
2011-07-27 05:24 Mike Thompson Note Added: 0050193
2011-07-27 05:28 Mike Thompson Note Edited: 0050193
2011-07-27 05:30 Mike Thompson Note Edited: 0050193
2011-07-30 18:27 Reinhard Berger Note Added: 0050289
2011-07-31 14:19 Reinhard Berger Note Added: 0050304
2011-07-31 15:59 Reinhard Berger Note Added: 0050310
2011-08-06 03:17 Mike Thompson Note Added: 0050525
2011-08-06 06:42 Mike Thompson Note Added: 0050528
2011-08-07 11:01 Reinhard Berger Note Added: 0050580
2011-08-07 11:02 Reinhard Berger File Added: ListViewGTK2-QT.zip
2011-08-07 11:06 Reinhard Berger File Added: listview -GTK2 -VsReport -0.9.31.tar.gz
2011-08-07 11:07 Reinhard Berger File Added: listview -QT -VsReport-0.9.31.tar.gz
2011-08-07 11:08 Reinhard Berger File Added: listview -Win32-VsReport -0.9.31.exe.tar.gz
2011-08-07 11:15 Reinhard Berger Note Edited: 0050580
2011-08-07 11:38 Reinhard Berger File Added: listview GTK2 -VsReport.0.9.28.2.tar.gz
2011-08-07 11:41 Reinhard Berger Note Edited: 0050580
2011-08-07 14:39 Maxim Ganetsky LazTarget => -
2011-08-07 14:39 Maxim Ganetsky Summary Slelecting items in listview is extremly slow => Selecting items in listview is extremly slow
2011-08-08 03:53 Mike Thompson Note Added: 0050600
2011-08-08 04:00 Mike Thompson Note Edited: 0050193
2011-10-06 13:46 Vincent Snijders Relationship added related to 0019094
2011-10-06 13:47 Vincent Snijders LazTarget - => 1.2
2011-10-06 13:47 Vincent Snijders Status new => acknowledged
2011-10-06 13:47 Vincent Snijders Target Version => 1.2.0
2011-10-30 08:56 Zeljan Rikalo Note Added: 0053621
2011-10-30 09:04 Zeljan Rikalo Note Edited: 0053621
2011-10-30 09:23 Zeljan Rikalo Note Edited: 0053621
2011-10-30 10:38 Zeljan Rikalo File Added: pureqtapp.tar.gz
2011-10-30 10:41 Zeljan Rikalo Note Added: 0053623
2011-10-30 10:42 Zeljan Rikalo LazTarget 1.2 => post 1.2
2011-10-31 13:36 Reinhard Berger Note Added: 0053680
2011-10-31 14:49 Vincent Snijders Target Version 1.2.0 =>
2011-10-31 15:17 Zeljan Rikalo Note Added: 0053685
2011-11-19 20:32 Zeljan Rikalo Relationship added related to 0020631
2012-02-09 13:29 Zeljan Rikalo Tag Attached: ListView
2012-02-09 14:32 Zeljan Rikalo Note Added: 0056687
2013-09-03 12:07 Martin Friebe LazTarget post 1.2 => 1.4
2013-10-09 22:11 Zeljan Rikalo Note Added: 0070708
2013-10-09 22:11 Zeljan Rikalo Assigned To => Zeljan Rikalo
2013-10-09 22:11 Zeljan Rikalo Status acknowledged => feedback
2013-10-09 23:16 Zeljan Rikalo Note Added: 0070711
2013-10-14 09:36 Zeljan Rikalo Fixed in Revision => 43244
2013-10-14 09:36 Zeljan Rikalo Note Added: 0070809
2013-10-14 09:36 Zeljan Rikalo Status feedback => resolved
2013-10-14 09:36 Zeljan Rikalo Resolution open => fixed