View Issue Details

IDProjectCategoryView StatusLast Update
0035356LazarusTAChartpublic2019-06-14 12:27
ReporterMarcin WiazowskiAssigned Towp 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
Product Version2.1 (SVN)Product Build60918 
Target VersionFixed in Version 
Summary0035356: TAChart: unfortunate sorting limitation in TCustomChartSource
DescriptionTCustomChartSource introduces only a dummy sorting implementation:

  function TCustomChartSource.IsSorted: Boolean;
  begin
    Result := false;
  end;

so sorting is not defined in any way in TCustomChartSource - but IsSorted() method can be overridden in TCustomChartSource descendants.



There can exist potentially unlimited number of sorting algorithms - in particular by X, by Y, by Color (which can hold in fact any integer information), by Text, by using also XList and/or YList.


But, unfortunately, TCustomChartSource blindly assumes, that "sorting" is equal to "sorting by X". So it's not currently possible to inherit from TCustomChartSource to implement any other sorting algorithm.

So I'm attaching a tiny patch, that introduces an "IsSortedByX" protected method in TCustomChartSource. By default, "IsSortedByX" just calls "IsSorted", so everything works as usual.

But "IsSortedByX" can by overridden in descendant classes to return False, if some other (than sorting by X) algorithm is used. Thanks to that, TCustomChartSource still works properly, and any needed sorting algorithm can be implemented in descendant classes.
TagsNo tags attached.
Fixed in Revision60972, 61189, 61190, 61211, 61248, 61251. 61252
LazTarget-
WidgetsetWin32/Win64
Attached Files
  • patch.diff (1,997 bytes)
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 60918)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -207,6 +207,7 @@
         function GetHasErrorBars(Which: Integer): Boolean;
         function GetItem(AIndex: Integer): PChartDataItem; virtual; abstract;
         procedure InvalidateCaches;
    +    function IsSortedByX: Boolean; virtual;
         procedure SetXCount(AValue: Cardinal); virtual; abstract;
         procedure SetYCount(AValue: Cardinal); virtual; abstract;
         property XErrorBarData: TChartErrorBarData index 0 read GetErrorBarData
    @@ -1011,7 +1012,7 @@
     
     // ALB -> leftmost item where X >= AXMin, or Count if no such item
     // ALB -> rightmost item where X <= AXMax, or -1 if no such item
    -// If the source is sorted, performs binary search. Otherwise, skips NaNs.
    +// If the source is sorted by X, performs binary search. Otherwise, skips NaNs.
     procedure TCustomChartSource.FindBounds(
       AXMin, AXMax: Double; out ALB, AUB: Integer);
     
    @@ -1041,7 +1042,7 @@
     
     begin
       EnsureOrder(AXMin, AXMax);
    -  if IsSorted then begin
    +  if IsSortedByX then begin
         ALB := FindLB(AXMin, 0, Count - 1);
         AUB := FindUB(AXMax, 0, Count - 1);
       end
    @@ -1267,8 +1268,13 @@
         (AYIndex > -1);
     end;
     
    -function TCustomChartSource.IsSorted: Boolean;
    +function TCustomChartSource.IsSortedByX: Boolean; inline;
     begin
    +  Result := IsSorted; // By default, we assume that IsSorted means IsSortedByX
    +end;
    +
    +function TCustomChartSource.IsSorted: Boolean; inline;
    +begin
       Result := false;
     end;
     
    @@ -1421,7 +1427,7 @@
         cnt += 1;
       end;
     
    -  if not IsSorted and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
    +  if not IsSortedByX and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
         SortValuesInRange(AValues, start, cnt - 1);
         if aipUseMinLength in AParams.FIntervals.Options then
           cnt := EnsureMinLength(start, cnt - 1);
    
    patch.diff (1,997 bytes)
  • Test.zip (3,441 bytes)
  • Test.png (16,567 bytes)
    Test.png (16,567 bytes)
  • patch_ver2.diff (8,462 bytes)
    Index: components/tachart/tacustomseries.pas
    ===================================================================
    --- components/tachart/tacustomseries.pas	(revision 60943)
    +++ components/tachart/tacustomseries.pas	(working copy)
    @@ -1940,15 +1940,15 @@
       {FLoBound and FUpBound fields may be outdated here (if axis' range has been
        changed after the last series' painting). FLoBound and FUpBound will be fully
        updated later, in a PrepareGraphPoints() call. But we need them now. If data
    -   source is sorted, obtaining FLoBound and FUpBound is very fast (binary search) -
    -   so we call FindExtentInterval() with True as the second parameter. If data
    -   source is not sorted, obtaining FLoBound and FUpBound requires enumerating all
    -   the data points to see, if they are in the current chart's viewport. But this
    -   is exactly what we are going to do in the loop below, so obtaining true FLoBound
    -   and FUpBound values makes no sense in this case - so we call FindExtentInterval()
    -   with False as the second parameter, thus setting FLoBound to 0 and FUpBound to
    -   Count-1}
    -  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
    +   source is sorted by X, obtaining FLoBound and FUpBound is very fast (binary
    +   search) - so we call FindExtentInterval() with True as the second parameter.
    +   If data source is not sorted by X, obtaining FLoBound and FUpBound requires
    +   enumerating all the data points to see, if they are in the current chart's
    +   viewport. But this is exactly what we are going to do in the loop below, so
    +   obtaining true FLoBound and FUpBound values makes no sense in this case - so
    +   we call FindExtentInterval() with False as the second parameter, thus setting
    +   FLoBound to 0 and FUpBound to Count-1}
    +  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByX);
     
       with Extent do
         center := AxisToGraphY((a.y + b.y) * 0.5);
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 60943)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -246,6 +246,8 @@
         function HasYErrorBars: Boolean;
         function IsXErrorIndex(AXIndex: Integer): Boolean;
         function IsYErrorIndex(AYIndex: Integer): Boolean;
    +    function IsSortedByX: Boolean; virtual; // must return true if -
    +      // and only if - source is sorted by X in the ascending order
         function IsSorted: Boolean; virtual;
         procedure ValuesInRange(
           AParams: TValuesInRangeParams; var AValues: TChartValueTextArray); virtual;
    @@ -1011,7 +1013,7 @@
     
     // ALB -> leftmost item where X >= AXMin, or Count if no such item
     // ALB -> rightmost item where X <= AXMax, or -1 if no such item
    -// If the source is sorted, performs binary search. Otherwise, skips NaNs.
    +// If the source is sorted by X, performs binary search. Otherwise, skips NaNs.
     procedure TCustomChartSource.FindBounds(
       AXMin, AXMax: Double; out ALB, AUB: Integer);
     
    @@ -1041,7 +1043,7 @@
     
     begin
       EnsureOrder(AXMin, AXMax);
    -  if IsSorted then begin
    +  if IsSortedByX then begin
         ALB := FindLB(AXMin, 0, Count - 1);
         AUB := FindUB(AXMax, 0, Count - 1);
       end
    @@ -1267,8 +1269,13 @@
         (AYIndex > -1);
     end;
     
    -function TCustomChartSource.IsSorted: Boolean;
    +function TCustomChartSource.IsSortedByX: Boolean; inline;
     begin
    +  Result := IsSorted; // By default, we assume that IsSorted means IsSortedByX
    +end;
    +
    +function TCustomChartSource.IsSorted: Boolean; inline;
    +begin
       Result := false;
     end;
     
    @@ -1421,7 +1428,7 @@
         cnt += 1;
       end;
     
    -  if not IsSorted and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
    +  if not IsSortedByX and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
         SortValuesInRange(AValues, start, cnt - 1);
         if aipUseMinLength in AParams.FIntervals.Options then
           cnt := EnsureMinLength(start, cnt - 1);
    Index: components/tachart/tamultiseries.pas
    ===================================================================
    --- components/tachart/tamultiseries.pas	(revision 60943)
    +++ components/tachart/tamultiseries.pas	(working copy)
    @@ -879,7 +879,7 @@
       if Count = 0 then exit;
       if not RequestValidChartScaling then exit;
     
    -  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
    +  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByX);
       with Extent do
         center := AxisToGraphY((a.y + b.y) * 0.5);
       UpdateLabelDirectionReferenceLevel(0, 0, center);
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 60943)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -23,9 +23,7 @@
     
       TListChartSource = class(TCustomChartSource)
       private
    -    FData: TFPList;
         FDataPoints: TStrings;
    -    FSorted: Boolean;
         FXCountMin: Cardinal;
         FYCountMin: Cardinal;
         procedure AddAt(
    @@ -36,6 +34,8 @@
         procedure SetSorted(AValue: Boolean);
         procedure UpdateCachesAfterAdd(AX, AY: Double);
       protected
    +    FData: TFPList;
    +    FSorted: Boolean;
         function GetCount: Integer; override;
         function GetItem(AIndex: Integer): PChartDataItem; override;
         procedure Loaded; override;
    @@ -70,7 +70,7 @@
         procedure SetYList(AIndex: Integer; const AYList: array of Double);
         procedure SetYValue(AIndex: Integer; AValue: Double);
     
    -    procedure Sort;
    +    procedure Sort; virtual;
       published
         property DataPoints: TStrings read FDataPoints write SetDataPoints;
         property Sorted: Boolean read FSorted write SetSorted default false;
    @@ -229,6 +229,7 @@
         constructor Create(AOwner: TComponent); override;
         destructor Destroy; override;
     
    +    function IsSortedByX: Boolean; override;
         function IsSorted: Boolean; override;
       published
         property AccumulationDirection: TChartAccumulationDirection
    @@ -490,7 +491,7 @@
       AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
     begin
       Result := FData.Count;
    -  if Sorted then
    +  if IsSortedByX then
         // Keep data points ordered by X coordinate.
         // Note that this leads to O(N^2) time except
         // for the case of adding already ordered points.
    @@ -603,7 +604,18 @@
             SetXList(FData.Count - 1, XList);
             SetYList(FData.Count - 1, YList);
           end;
    -    if Sorted and not ASource.IsSorted then Sort;
    +    if IsSorted then
    +      if IsSortedByX and ASource.IsSortedByX then
    +        // both Self and ASource are sorted by X in the ascending order,
    +        // so there is nothing more to do
    +      else
    +      if (ClassInfo = ASource.ClassInfo) and ASource.IsSorted then
    +        // both Self and ASource are sorted by some custom algorithm -
    +        // but both of them are objects of the exactly same class,
    +        // so both use the exactly same sorting algorithm - so there
    +        // is nothing more to do
    +      else
    +        Sort;
       finally
         EndUpdate;
       end;
    @@ -670,7 +682,7 @@
       Result := PChartDataItem(FData.Items[AIndex]);
     end;
     
    -function TListChartSource.IsSorted: Boolean;
    +function TListChartSource.IsSorted: Boolean; inline;
     begin
       Result := Sorted;
     end;
    @@ -766,7 +778,7 @@
       end;
     
     begin
    -  if Sorted then
    +  if IsSortedByX then
         if IsNan(AValue) then
           raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
       oldX := Item[AIndex]^.X;
    @@ -774,7 +786,7 @@
       if IsEquivalent(oldX, AValue) then exit;
       Item[AIndex]^.X := AValue;
       UpdateExtent;
    -  if Sorted then begin
    +  if IsSortedByX then begin
         if AValue > oldX then
           while (Result < Count - 1) and (Item[Result + 1]^.X < AValue) do
             Inc(Result)
    @@ -1035,7 +1047,7 @@
       Result := @FCurItem;
     end;
     
    -function TRandomChartSource.IsSorted: Boolean;
    +function TRandomChartSource.IsSorted: Boolean; inline;
     begin
       Result := not RandomX;
     end;
    @@ -1142,7 +1154,7 @@
       Result := @FItem;
     end;
     
    -function TUserDefinedChartSource.IsSorted: Boolean;
    +function TUserDefinedChartSource.IsSorted: Boolean; inline;
     begin
       Result := Sorted;
     end;
    @@ -1435,6 +1447,14 @@
       Result := AccumulationMethod in [camDerivative, camSmoothDerivative];
     end;
     
    +function TCalculatedChartSource.IsSortedByX: Boolean;
    +begin
    +  if Origin <> nil then
    +    Result := Origin.IsSortedByX
    +  else
    +    Result := false;
    +end;
    +
     function TCalculatedChartSource.IsSorted: Boolean;
     begin
       if Origin <> nil then
    
    patch_ver2.diff (8,462 bytes)
  • Test-wp.zip (3,227 bytes)
  • patch_ver3.diff (13,836 bytes)
    Index: components/tachart/tachartstrconsts.pas
    ===================================================================
    --- components/tachart/tachartstrconsts.pas	(revision 60963)
    +++ components/tachart/tachartstrconsts.pas	(working copy)
    @@ -78,12 +78,12 @@
       rsSourceNotEditable = 'Editable chart source required';
       rsSourceCountError = '%0:s requires a chart source with at least %1:d %2:s value(s) per data point.';
       rsSourceCountError2 = 'This %0:s instance must have at least %1:d %2:s value(s) per data point.';
    +  rsSourceSortError = 'Selected sorting parameters are not supported by %s.';
       rsListSourceStringFormatError = 'The data value count in the %0:s.DataPoints '+
         'string "%1:s" differs from what is expected from XCount and YCount.';
       rsListSourceNumericError = 'The %0:s.DataPoints string "%1:s" is not a valid number.';
       rsListSourceColorError = 'The %0:s.DataPoints string "%1:s" is not an integer.';
     
    -
       // Transformations
       tasAxisTransformsEditorTitle = 'Edit axis transformations';
       rsAutoScale = 'Auto scale';
    Index: components/tachart/tacustomseries.pas
    ===================================================================
    --- components/tachart/tacustomseries.pas	(revision 60963)
    +++ components/tachart/tacustomseries.pas	(working copy)
    @@ -1940,15 +1940,15 @@
       {FLoBound and FUpBound fields may be outdated here (if axis' range has been
        changed after the last series' painting). FLoBound and FUpBound will be fully
        updated later, in a PrepareGraphPoints() call. But we need them now. If data
    -   source is sorted, obtaining FLoBound and FUpBound is very fast (binary search) -
    -   so we call FindExtentInterval() with True as the second parameter. If data
    -   source is not sorted, obtaining FLoBound and FUpBound requires enumerating all
    -   the data points to see, if they are in the current chart's viewport. But this
    -   is exactly what we are going to do in the loop below, so obtaining true FLoBound
    -   and FUpBound values makes no sense in this case - so we call FindExtentInterval()
    -   with False as the second parameter, thus setting FLoBound to 0 and FUpBound to
    -   Count-1}
    -  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
    +   source is sorted by X in the ascending order, obtaining FLoBound and FUpBound
    +   is very fast (binary search) - so we call FindExtentInterval() with True as
    +   the second parameter. Otherwise, obtaining FLoBound and FUpBound requires
    +   enumerating all the data points to see, if they are in the current chart's
    +   viewport. But this is exactly what we are going to do in the loop below, so
    +   obtaining true FLoBound and FUpBound values makes no sense in this case - so
    +   we call FindExtentInterval() with False as the second parameter, thus setting
    +   FLoBound to 0 and FUpBound to Count-1}
    +  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByXAsc);
     
       with Extent do
         center := AxisToGraphY((a.y + b.y) * 0.5);
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 60963)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -71,9 +71,10 @@
     type
       EBufferError = class(EChartError);
       EEditableSourceRequired = class(EChartError);
    +  EListSourceStringError = class(EChartError);
    +  ESortError = class(EChartError);
       EXCountError = class(EChartError);
       EYCountError = class(EChartError);
    -  EListSourceStringError = class(EChartError);
     
       TChartValueText = record
         FText: String;
    @@ -175,9 +176,11 @@
         property IndexPlus: Integer index 0 read GetIndex write SetIndex default -1;
         property ValueMinus: Double index 1 read GetValue write SetValue stored IsErrorBarValueStored;
         property ValuePlus: Double index 0 read GetValue write SetValue stored IsErrorBarValueStored;
    -
       end;
     
    +  TChartSortBy = (sbX, sbY, sbColor, sbText, sbCustom);
    +  TChartSortDir = (sdAscending, sdDescending);
    +
       TCustomChartSource = class(TBasicChartSource)
       strict private
         FErrorBarData: array[0..1] of TChartErrorBarData;
    @@ -197,6 +200,9 @@
         FYListExtentIsValid: Boolean;
         FValuesTotal: Double;
         FValuesTotalIsValid: Boolean;
    +    FSortBy: TChartSortBy;
    +    FSortDir: TChartSortDir;
    +    FSortIndex: Cardinal;
         FXCount: Cardinal;
         FYCount: Cardinal;
         function CalcExtentXYList(UseXList: Boolean): TDoubleRect;
    @@ -207,6 +213,9 @@
         function GetHasErrorBars(Which: Integer): Boolean;
         function GetItem(AIndex: Integer): PChartDataItem; virtual; abstract;
         procedure InvalidateCaches;
    +    procedure SetSortBy(AValue: TChartSortBy); virtual;
    +    procedure SetSortDir(AValue: TChartSortDir); virtual;
    +    procedure SetSortIndex(AValue: Cardinal); virtual;
         procedure SetXCount(AValue: Cardinal); virtual; abstract;
         procedure SetYCount(AValue: Cardinal); virtual; abstract;
         property XErrorBarData: TChartErrorBarData index 0 read GetErrorBarData
    @@ -247,6 +256,7 @@
         function IsXErrorIndex(AXIndex: Integer): Boolean;
         function IsYErrorIndex(AYIndex: Integer): Boolean;
         function IsSorted: Boolean; virtual;
    +    function IsSortedByXAsc: Boolean;
         procedure ValuesInRange(
           AParams: TValuesInRangeParams; var AValues: TChartValueTextArray); virtual;
         function ValuesTotal: Double; virtual;
    @@ -255,6 +265,9 @@
     
         property Count: Integer read GetCount;
         property Item[AIndex: Integer]: PChartDataItem read GetItem; default;
    +    property SortBy: TChartSortBy read FSortBy write SetSortBy default sbX;
    +    property SortDir: TChartSortDir read FSortDir write SetSortDir default sdAscending;
    +    property SortIndex: Cardinal read FSortIndex write SetSortIndex default 0;
         property XCount: Cardinal read FXCount write SetXCount default 1;
         property YCount: Cardinal read FYCount write SetYCount default 1;
       end;
    @@ -288,7 +301,7 @@
     implementation
     
     uses
    -  Math, StrUtils, SysUtils, TAMath;
    +  Math, StrUtils, SysUtils, TAMath, TAChartStrConsts;
     
     function CompareChartValueTextPtr(AItem1, AItem2: Pointer): Integer;
     begin
    @@ -895,8 +908,6 @@
       end;
     end;
     
    -
    -
     class procedure TCustomChartSource.CheckFormat(const AFormat: String);
     begin
       Format(AFormat, [0.0, 0.0, '', 0.0, 0.0]);
    @@ -907,6 +918,9 @@
       i: Integer;
     begin
       inherited Create(AOwner);
    +  FSortBy := sbX;
    +  FSortDir := sdAscending;
    +  FSortIndex := 0;
       FXCount := 1;
       FYCount := 1;
       for i:=Low(FErrorBarData) to High(FErrorBarData) do begin
    @@ -1011,7 +1025,8 @@
     
     // ALB -> leftmost item where X >= AXMin, or Count if no such item
     // ALB -> rightmost item where X <= AXMax, or -1 if no such item
    -// If the source is sorted, performs binary search. Otherwise, skips NaNs.
    +// If the source is sorted by X in the ascending order, performs
    +// binary search. Otherwise, skips NaNs.
     procedure TCustomChartSource.FindBounds(
       AXMin, AXMax: Double; out ALB, AUB: Integer);
     
    @@ -1041,7 +1056,7 @@
     
     begin
       EnsureOrder(AXMin, AXMax);
    -  if IsSorted then begin
    +  if IsSortedByXAsc then begin
         ALB := FindLB(AXMin, 0, Count - 1);
         AUB := FindUB(AXMax, 0, Count - 1);
       end
    @@ -1267,11 +1282,34 @@
         (AYIndex > -1);
     end;
     
    -function TCustomChartSource.IsSorted: Boolean;
    +function TCustomChartSource.IsSorted: Boolean; inline;
     begin
       Result := false;
     end;
     
    +function TCustomChartSource.IsSortedByXAsc: Boolean;
    +begin
    +  Result := IsSorted and (FSortBy = sbX) and (FSortDir = sdAscending) and (FSortIndex = 0);
    +end;
    +
    +procedure TCustomChartSource.SetSortBy(AValue: TChartSortBy);
    +begin
    +  if FSortBy <> AValue then
    +    raise ESortError.CreateFmt(rsSourceSortError, [ClassName]);
    +end;
    +
    +procedure TCustomChartSource.SetSortDir(AValue: TChartSortDir);
    +begin
    +  if FSortDir <> AValue then
    +    raise ESortError.CreateFmt(rsSourceSortError, [ClassName]);
    +end;
    +
    +procedure TCustomChartSource.SetSortIndex(AValue: Cardinal);
    +begin
    +  if FSortIndex <> AValue then
    +    raise ESortError.CreateFmt(rsSourceSortError, [ClassName]);
    +end;
    +
     procedure TCustomChartSource.SetErrorBarData(AIndex: Integer;
       AValue: TChartErrorBarData);
     begin
    @@ -1421,7 +1459,7 @@
         cnt += 1;
       end;
     
    -  if not IsSorted and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
    +  if not IsSortedByXAsc and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
         SortValuesInRange(AValues, start, cnt - 1);
         if aipUseMinLength in AParams.FIntervals.Options then
           cnt := EnsureMinLength(start, cnt - 1);
    Index: components/tachart/tamultiseries.pas
    ===================================================================
    --- components/tachart/tamultiseries.pas	(revision 60963)
    +++ components/tachart/tamultiseries.pas	(working copy)
    @@ -879,7 +879,7 @@
       if Count = 0 then exit;
       if not RequestValidChartScaling then exit;
     
    -  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
    +  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByXAsc);
       with Extent do
         center := AxisToGraphY((a.y + b.y) * 0.5);
       UpdateLabelDirectionReferenceLevel(0, 0, center);
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 60963)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -23,9 +23,7 @@
     
       TListChartSource = class(TCustomChartSource)
       private
    -    FData: TFPList;
         FDataPoints: TStrings;
    -    FSorted: Boolean;
         FXCountMin: Cardinal;
         FYCountMin: Cardinal;
         procedure AddAt(
    @@ -36,6 +34,8 @@
         procedure SetSorted(AValue: Boolean);
         procedure UpdateCachesAfterAdd(AX, AY: Double);
       protected
    +    FData: TFPList;
    +    FSorted: Boolean;
         function GetCount: Integer; override;
         function GetItem(AIndex: Integer): PChartDataItem; override;
         procedure Loaded; override;
    @@ -70,7 +70,7 @@
         procedure SetYList(AIndex: Integer; const AYList: array of Double);
         procedure SetYValue(AIndex: Integer; AValue: Double);
     
    -    procedure Sort;
    +    procedure Sort; virtual;
       published
         property DataPoints: TStrings read FDataPoints write SetDataPoints;
         property Sorted: Boolean read FSorted write SetSorted default false;
    @@ -203,6 +203,7 @@
         FOriginYCount: Cardinal;
         FPercentage: Boolean;
         FReorderYList: String;
    +    FSorted: Boolean;
         FYOrder: array of Integer;
     
         procedure CalcAccumulation(AIndex: Integer);
    @@ -490,7 +491,7 @@
       AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
     begin
       Result := FData.Count;
    -  if Sorted then
    +  if IsSortedByXAsc then
         // Keep data points ordered by X coordinate.
         // Note that this leads to O(N^2) time except
         // for the case of adding already ordered points.
    @@ -603,7 +604,22 @@
             SetXList(FData.Count - 1, XList);
             SetYList(FData.Count - 1, YList);
           end;
    -    if Sorted and not ASource.IsSorted then Sort;
    +
    +    if IsSorted then begin
    +      if
    +        ASource.IsSorted and (SortBy = ASource.SortBy) and
    +        (SortDir = ASource.SortDir) and (SortIndex = ASource.SortIndex) and
    +        (
    +          (SortBy <> sbCustom) or // data is already sorted as needed -
    +            // so there is nothing more to do
    +          (ClassInfo = ASource.ClassInfo) // both Self and ASource are
    +            // sorted by some custom algorithm - but both of them are
    +            // objects of the exactly same class, so they use the
    +            // exactly same algorithm - so there is nothing more to do
    +        ) then
    +          exit;
    +      Sort;
    +    end;
       finally
         EndUpdate;
       end;
    @@ -670,9 +686,9 @@
       Result := PChartDataItem(FData.Items[AIndex]);
     end;
     
    -function TListChartSource.IsSorted: Boolean;
    +function TListChartSource.IsSorted: Boolean; inline;
     begin
    -  Result := Sorted;
    +  Result := FSorted;
     end;
     
     function TListChartSource.NewItem: PChartDataItem;
    @@ -766,7 +782,7 @@
       end;
     
     begin
    -  if Sorted then
    +  if IsSortedByXAsc then
         if IsNan(AValue) then
           raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
       oldX := Item[AIndex]^.X;
    @@ -774,7 +790,7 @@
       if IsEquivalent(oldX, AValue) then exit;
       Item[AIndex]^.X := AValue;
       UpdateExtent;
    -  if Sorted then begin
    +  if IsSortedByXAsc then begin
         if AValue > oldX then
           while (Result < Count - 1) and (Item[Result + 1]^.X < AValue) do
             Inc(Result)
    @@ -1035,7 +1051,7 @@
       Result := @FCurItem;
     end;
     
    -function TRandomChartSource.IsSorted: Boolean;
    +function TRandomChartSource.IsSorted: Boolean; inline;
     begin
       Result := not RandomX;
     end;
    @@ -1142,9 +1158,9 @@
       Result := @FItem;
     end;
     
    -function TUserDefinedChartSource.IsSorted: Boolean;
    +function TUserDefinedChartSource.IsSorted: Boolean; inline;
     begin
    -  Result := Sorted;
    +  Result := FSorted;
     end;
     
     procedure TUserDefinedChartSource.Reset;
    @@ -1350,6 +1366,22 @@
     
     procedure TCalculatedChartSource.Changed(ASender: TObject);
     begin
    +  if FOrigin <> nil then begin
    +    FSortBy := Origin.SortBy;
    +    FSortDir := Origin.SortDir;
    +    FSortIndex := Origin.SortIndex;
    +    // We recalculate Y values, so we can't guarantee, that transformed
    +    // data is still sorted by Y or by Origin's custom algorithm
    +    FSorted := (FSortBy in [sbX, sbColor, sbText]) and Origin.IsSorted;
    +    FXCount := Origin.XCount;
    +  end else begin
    +    FSortBy := sbX;
    +    FSortDir := sdAscending;
    +    FSortIndex := 0;
    +    FSorted := false;
    +    FXCount := 0;
    +  end;
    +
       if
         (FOrigin <> nil) and (ASender = FOrigin) and
         (FOrigin.YCount <> FOriginYCount)
    @@ -1437,10 +1469,7 @@
     
     function TCalculatedChartSource.IsSorted: Boolean;
     begin
    -  if Origin <> nil then
    -    Result := Origin.IsSorted
    -  else
    -    Result := false;
    +  Result := FSorted;
     end;
     
     procedure TCalculatedChartSource.RangeAround(
    
    patch_ver3.diff (13,836 bytes)
  • BubbleSortTest.zip (3,572 bytes)
  • patch_new_update.diff (4,705 bytes)
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 60976)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -27,6 +27,7 @@
       private
         FOnCompare: TChartSortCompare;
         procedure SetSorted(AValue: Boolean);
    +    procedure SetOnCompare(AValue: TChartSortCompare);
       protected
         FData: TFPList;
         FSorted: Boolean;
    @@ -36,7 +37,7 @@
         procedure SetSortDir(AValue: TChartSortDir); override;
         procedure SetSortIndex(AValue: Cardinal); override;
         property Sorted: Boolean read FSorted write SetSorted default false;
    -    property OnCompare: TChartSortCompare read FOnCompare write FOnCompare;
    +    property OnCompare: TChartSortCompare read FOnCompare write SetOnCompare;
       public
         function IsSorted: Boolean; override;
         procedure Sort;
    @@ -528,23 +529,23 @@
     
       procedure QuickSort(L, R: Longint);
       var
    -    I, J : Longint;
    -    P, Q : Pointer;
    +    I, J: Longint;
    +    P, Q: Pointer;
       begin
        repeat
          I := L;
          J := R;
    -     P := FData[(L + R) div 2];
    +     P := FData.List^[(L + R) div 2];
          repeat
    -       while ACompare(P, FData[i]) > 0 do
    +       while ACompare(P, FData.List^[I]) > 0 do
              I := I + 1;
    -       while ACompare(P, FData[J]) < 0 do
    +       while ACompare(P, FData.List^[J]) < 0 do
              J := J - 1;
            If I <= J then
            begin
    -         Q := FData[I];
    -         FData[I] := FData[J];
    -         FData[J] := Q;
    +         Q := FData.List^[I];
    +         FData.List^[I] := FData.List^[J];
    +         FData.List^[J] := Q;
              I := I + 1;
              J := J - 1;
            end;
    @@ -578,7 +579,7 @@
     begin
       if FSortBy = AValue then exit;
       FSortBy := AValue;
    -  Sort;
    +  if Sorted then Sort;
     end;
     
     procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
    @@ -585,7 +586,7 @@
     begin
       if FSorted = AValue then exit;
       FSorted := AValue;
    -  Sort;
    +  if Sorted then Sort;
     end;
     
     procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
    @@ -592,7 +593,7 @@
     begin
       if FSortDir = AValue then exit;
       FSortDir := AValue;
    -  Sort;
    +  if Sorted then Sort;
     end;
     
     procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
    @@ -599,15 +600,27 @@
     begin
       if FSortIndex = AValue then exit;
       FSortIndex := AValue;
    -  Sort;
    +  if Sorted then Sort;
     end;
     
    +procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
    +begin
    +  if FOnCompare = AValue then exit;
    +  FOnCompare := AValue;
    +  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
    +end;
    +
     procedure TCustomSortedChartSource.Sort;
     begin
    +  if csLoading in ComponentState then exit;
       if (FSortBy = sbCustom) then begin
    -    if Assigned(FOnCompare) then ExecSort(FOnCompare);
    -  end else
    +    if not Assigned(FOnCompare) then exit;
    +    ExecSort(FOnCompare);
    +  end else begin
    +    if (FSortBy = sbX) and (FSortIndex > 0) and (FSortIndex >= FXCount) then exit;
    +    if (FSortBy = sbY) and (FSortIndex > 0) and (FSortIndex >= FYCount) then exit;
         ExecSort(@DefaultCompare);
    +  end;
       Notify;
     end;
     
    @@ -767,17 +780,27 @@
     end;
     
     function TListChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
    +const
    +  sInvalidCall = 'Invalid call to %s.DefaultCompare';
     var
       item1: PChartDataItem absolute AItem1;
       item2: PChartDataItem absolute AItem2;
     begin
    - case FSortBy of
    -   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
    -   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
    -   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
    -   sbText: Result := CompareText(item1^.Text, item2^.Text);
    -   sbCustom: Result := FOnCompare(AItem1, AItem2);
    - end;
    +  case FSortBy of
    +    sbX: begin
    +           if (FSortIndex > 0) and (FSortIndex >= FXCount) then
    +             ESortError.CreateFmt(sInvalidCall, [ClassName]);
    +           Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
    +         end;
    +    sbY: begin
    +           if (FSortIndex > 0) and (FSortIndex >= FYCount) then
    +             ESortError.CreateFmt(sInvalidCall, [ClassName]);
    +           Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
    +         end;
    +    sbColor: Result := CompareValue(item1^.Color, item2^.Color);
    +    sbText: Result := CompareText(item1^.Text, item2^.Text);
    +    else raise ESortError.CreateFmt(sInvalidCall, [ClassName]);
    +  end;
      if FSortDir = sdDescending then Result := -Result;
     end;
     
    @@ -845,7 +868,7 @@
       BeginUpdate;
       try
         FDataPoints.Assign(AValue);
    -    if IsSorted then Sort;
    +    if Sorted then Sort;
       finally
         EndUpdate;
       end;
    
    patch_new_update.diff (4,705 bytes)
  • SpeedTest.zip (2,536 bytes)
  • exchange_test.diff (3,605 bytes)
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 61162)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -219,6 +219,7 @@
         FAccumulationDirection: TChartAccumulationDirection;
         FAccumulationMethod: TChartAccumulationMethod;
         FAccumulationRange: Cardinal;
    +    FExchangeXY: Boolean;
         FHistory: TChartSourceBuffer;
         FIndex: Integer;
         FItem: TChartDataItem;
    @@ -241,6 +242,7 @@
         procedure SetAccumulationDirection(AValue: TChartAccumulationDirection);
         procedure SetAccumulationMethod(AValue: TChartAccumulationMethod);
         procedure SetAccumulationRange(AValue: Cardinal);
    +    procedure SetExchangeXY(AValue: Boolean);
         procedure SetOrigin(AValue: TCustomChartSource);
         procedure SetPercentage(AValue: Boolean);
         procedure SetReorderYList(const AValue: String);
    @@ -263,6 +265,8 @@
         property AccumulationRange: Cardinal
           read FAccumulationRange write SetAccumulationRange default 2;
     
    +    property ExchangeXY: Boolean
    +      read FExchangeXY write SetExchangeXY default false;
         property Origin: TCustomChartSource read FOrigin write SetOrigin;
         property Percentage: Boolean
           read FPercentage write SetPercentage default false;
    @@ -1491,13 +1495,20 @@
     procedure TCalculatedChartSource.Changed(ASender: TObject);
     begin
       if FOrigin <> nil then begin
    -    FSortBy := TCustomChartSourceAccess(Origin).SortBy;
    -    FSortDir := TCustomChartSourceAccess(Origin).SortDir;
    -    FSortIndex := TCustomChartSourceAccess(Origin).SortIndex;
    +    FSortBy := TCustomChartSourceAccess(FOrigin).SortBy;
    +
    +    if FExchangeXY then
    +      case FSortBy of
    +        sbX : FSortBy := sbY;
    +        sbY : FSortBy := sbX;
    +      end;
    +
    +    FSortDir := TCustomChartSourceAccess(FOrigin).SortDir;
    +    FSortIndex := TCustomChartSourceAccess(FOrigin).SortIndex;
         // We recalculate Y values, so we can't guarantee, that transformed
         // data is still sorted by Y or by Origin's custom algorithm
    -    FSorted := (FSortBy in [sbX, sbColor, sbText]) and Origin.IsSorted;
    -    FXCount := Origin.XCount;
    +    FSorted := (FSortBy in [sbX, sbColor, sbText]) and FOrigin.IsSorted;
    +    FXCount := IfThen(not FExchangeXY, FOrigin.XCount, FOrigin.YCount);
         // FYCount is set below, in the UpdateYOrder() call
       end else begin
         FSortBy := sbX;
    @@ -1510,7 +1521,7 @@
     
       if
         (FOrigin <> nil) and (ASender = FOrigin) and
    -    (FOrigin.YCount <> FOriginYCount)
    +    (IfThen(not FExchangeXY, FOrigin.YCount, FOrigin.XCount) <> FOriginYCount)
       then begin
         UpdateYOrder;
         exit;
    @@ -1559,10 +1570,21 @@
       end;
     
     var
    +  d: double;
       t: TDoubleDynArray;
       i: Integer;
     begin
       FItem := Origin[AIndex]^;
    +
    +  if FExchangeXY then begin
    +    d := FItem.X;
    +    FItem.X := FItem.Y;
    +    FItem.Y := d;
    +    t := FItem.XList;
    +    FItem.XList := FItem.YList;
    +    FItem.YList := t;
    +  end;
    +
       if Length(FYOrder) > 0 then begin
         SetLength(t, High(FYOrder));
         for i := 1 to High(FYOrder) do
    @@ -1639,6 +1661,13 @@
       Changed(nil);
     end;
     
    +procedure TCalculatedChartSource.SetExchangeXY(AValue: Boolean);
    +begin
    +  if FExchangeXY = AValue then exit;
    +  FExchangeXY := AValue;
    +  UpdateYOrder;
    +end;
    +
     procedure TCalculatedChartSource.SetOrigin(AValue: TCustomChartSource);
     begin
       if AValue = Self then
    @@ -1692,7 +1721,7 @@
         exit;
       end;
     
    -  FOriginYCount := FOrigin.YCount;
    +  FOriginYCount := IfThen(not FExchangeXY, FOrigin.YCount, FOrigin.XCount);
       if FOriginYCount = 0 then
         FYOrder := nil
       else
    
    exchange_test.diff (3,605 bytes)
  • phase1.diff (23,495 bytes)
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 61181)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -93,11 +93,11 @@
         YList: TDoubleDynArray;
         function GetX(AIndex: Integer): Double;
         function GetY(AIndex: Integer): Double;
    -    procedure SetX(AIndex: Integer; AValue: Double);
    -    procedure SetX(AValue: Double);
    -    procedure SetY(AIndex: Integer; AValue: Double);
    -    procedure SetY(AValue: Double);
    -    procedure MultiplyY(ACoeff: Double);
    +    procedure SetX(AIndex: Integer; const AValue: Double);
    +    procedure SetX(const AValue: Double);
    +    procedure SetY(AIndex: Integer; const AValue: Double);
    +    procedure SetY(const AValue: Double);
    +    procedure MultiplyY(const ACoeff: Double);
         function Point: TDoublePoint; inline;
       end;
       PChartDataItem = ^TChartDataItem;
    @@ -165,7 +165,7 @@
         function IsErrorBarValueStored(AIndex: Integer): Boolean;
         procedure SetKind(AValue: TChartErrorBarKind);
         procedure SetIndex(AIndex, AValue: Integer);
    -    procedure SetValue(AIndex: Integer; AValue: Double);
    +    procedure SetValue(AIndex: Integer; const AValue: Double);
       public
         constructor Create;
         procedure Assign(ASource: TPersistent); override;
    @@ -245,7 +245,7 @@
         function FormatItem(
           const AFormat: String; AIndex, AYIndex: Integer): String; inline;
         function FormatItemXYText(
    -      const AFormat: String; AX, AY: Double; AText: String): String;
    +      const AFormat: String; const AX, AY: Double; const AText: String): String;
         function GetEnumerator: TCustomChartSourceEnumerator;
         function GetXErrorBarLimits(APointIndex: Integer;
           out AUpperLimit, ALowerLimit: Double): Boolean;
    @@ -273,8 +273,35 @@
         property YCount: Cardinal read FYCount write SetYCount default 1;
       end;
     
    -  { TChartSourceBuffer }
    +  TChartSortCompare = function(AItem1, AItem2: Pointer): Integer of object;
     
    +  TCustomSortedChartSource = class(TCustomChartSource)
    +  private
    +    FOnCompare: TChartSortCompare;
    +    procedure SetOnCompare(AValue: TChartSortCompare);
    +    procedure SetSorted(AValue: Boolean);
    +  protected
    +    FCompareProc: TChartSortCompare;
    +    FData: TFPList;
    +    FSorted: Boolean;
    +    function DefaultCompare(AItem1, AItem2: Pointer): Integer; virtual;
    +    function DoCompare(AItem1, AItem2: Pointer): Integer; virtual;
    +    procedure ExecSort(ACompare: TChartSortCompare); virtual;
    +    function GetCount: Integer; override;
    +    function GetItem(AIndex: Integer): PChartDataItem; override;
    +    procedure SetSortBy(AValue: TChartSortBy); override;
    +    procedure SetSortDir(AValue: TChartSortDir); override;
    +    procedure SetSortIndex(AValue: Cardinal); override;
    +    property Sorted: Boolean read FSorted write SetSorted default false;
    +    property OnCompare: TChartSortCompare read FOnCompare write SetOnCompare;
    +  public
    +    constructor Create(AOwner: TComponent); override;
    +    destructor Destroy; override;
    +  public
    +    function IsSorted: Boolean; override;
    +    procedure Sort;
    +  end;
    +
       TChartSourceBuffer = class
       strict private
         FBuf: array of TChartDataItem;
    @@ -352,7 +379,7 @@
     
     procedure TValuesInRangeParams.RoundToImage(var AValue: Double);
     
    -  function A2I(AX: Double): Integer; inline;
    +  function A2I(const AX: Double): Integer; inline;
       begin
         Result := FGraphToImage(FAxisToGraph(AX));
       end;
    @@ -504,7 +531,7 @@
         Result := YList[AIndex - 1];
     end;
     
    -procedure TChartDataItem.MultiplyY(ACoeff: Double);
    +procedure TChartDataItem.MultiplyY(const ACoeff: Double);
     var
       i: Integer;
     begin
    @@ -519,7 +546,7 @@
       Result.Y := Y;
     end;
     
    -procedure TChartDataItem.SetX(AValue: Double);
    +procedure TChartDataItem.SetX(const AValue: Double);
     var
       i: Integer;
     begin
    @@ -528,7 +555,7 @@
         XList[i] := AValue;
     end;
     
    -procedure TChartDataItem.SetX(AIndex: Integer; AValue: Double);
    +procedure TChartDataItem.SetX(AIndex: Integer; const AValue: Double);
     begin
       if AIndex = 0 then
         X := AValue
    @@ -536,7 +563,7 @@
         XList[AIndex - 1] := AValue;
     end;
     
    -procedure TChartDataItem.SetY(AValue: Double);
    +procedure TChartDataItem.SetY(const AValue: Double);
     var
       i: Integer;
     begin
    @@ -545,7 +572,7 @@
         YList[i] := AValue;
     end;
     
    -procedure TChartDataItem.SetY(AIndex: Integer; AValue: Double);
    +procedure TChartDataItem.SetY(AIndex: Integer; const AValue: Double);
     begin
       if AIndex = 0 then
         Y := AValue
    @@ -787,7 +814,7 @@
       Changed;
     end;
     
    -procedure TChartErrorBarData.SetValue(AIndex: Integer; AValue: Double);
    +procedure TChartErrorBarData.SetValue(AIndex: Integer; const AValue: Double);
     begin
       if FValue[AIndex] = AValue then exit;
       FValue[AIndex] := AValue;
    @@ -1049,7 +1076,7 @@
     procedure TCustomChartSource.FindBounds(
       AXMin, AXMax: Double; out ALB, AUB: Integer);
     
    -  function FindLB(X: Double; L, R: Integer): Integer;
    +  function FindLB(const X: Double; L, R: Integer): Integer;
       begin
         while L <= R do begin
           Result := (R - L) div 2 + L;
    @@ -1061,7 +1088,7 @@
         Result := L;
       end;
     
    -  function FindUB(X: Double; L, R: Integer): Integer;
    +  function FindUB(const X: Double; L, R: Integer): Integer;
       begin
         while L <= R do begin
           Result := (R - L) div 2 + L;
    @@ -1107,11 +1134,11 @@
       const AFormat: String; AIndex, AYIndex: Integer): String;
     begin
       with Item[AIndex]^ do
    -    Result := FormatItemXYText(AFormat, IfThen(XCount > 0, X, double(AIndex)), GetY(AYIndex), Text);
    +    Result := FormatItemXYText(AFormat, IfThen(XCount > 0, X, Double(AIndex)), GetY(AYIndex), Text);
     end;
     
     function TCustomChartSource.FormatItemXYText(
    -  const AFormat: String; AX, AY: Double; AText: String): String;
    +  const AFormat: String; const AX, AY: Double; const AText: String): String;
     const
       TO_PERCENT = 100;
     var
    @@ -1406,7 +1433,7 @@
     var
       prevImagePos: Integer = MaxInt;
     
    -  function IsTooClose(AValue: Double): Boolean;
    +  function IsTooClose(const AValue: Double): Boolean;
       var
         imagePos: Integer;
       begin
    @@ -1538,5 +1565,164 @@
       Result := 0.0;
     end;
     
    +
    +{ TCustomSortedChartSource }
    +
    +constructor TCustomSortedChartSource.Create(AOwner: TComponent);
    +begin
    +  inherited Create(AOwner);
    +  FData := TFPList.Create;
    +end;
    +
    +destructor TCustomSortedChartSource.Destroy;
    +begin
    +  FreeAndNil(FData);
    +  inherited;
    +end;
    +
    +function CompareFloat(const x1, x2: Double): Integer;
    +begin
    +  if IsNaN(x1) and IsNaN(x2) then
    +    Result := 0
    +  else if IsNaN(x1) then
    +    Result := +1
    +  else if IsNaN(x2) then
    +    Result := -1
    +  else
    +    Result := CompareValue(x1, x2);
    +end;
    +
    +function TCustomSortedChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
    +var
    +  item1: PChartDataItem absolute AItem1;
    +  item2: PChartDataItem absolute AItem2;
    +begin
    + case FSortBy of
    +   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
    +   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
    +   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
    +   sbText: Result := CompareText(item1^.Text, item2^.Text);
    +   sbCustom: Result := FOnCompare(AItem1, AItem2);
    + end;
    + if FSortDir = sdDescending then Result := -Result;
    +end;
    +
    +function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
    +begin
    +  Result := FCompareProc(AItem1, AItem2);
    +end;
    +
    +{ Built-in sorting algorithm of the ChartSource, a standard QuickSort.
    +  Copied from the classes unit because the compare function must be a method. }
    +procedure TCustomSortedChartSource.ExecSort(ACompare: TChartSortCompare);
    +
    +  procedure QuickSort(L, R: Longint);
    +  var
    +    I, J: Longint;
    +    P, Q: Pointer;
    +  begin
    +   repeat
    +     I := L;
    +     J := R;
    +     P := FData.List^[(L + R) div 2];
    +     repeat
    +       while ACompare(P, FData.List^[I]) > 0 do
    +         I := I + 1;
    +       while ACompare(P, FData.List^[J]) < 0 do
    +         J := J - 1;
    +       If I <= J then
    +       begin
    +         Q := FData.List^[I];
    +         FData.List^[I] := FData.List^[J];
    +         FData.List^[J] := Q;
    +         I := I + 1;
    +         J := J - 1;
    +       end;
    +     until I > J;
    +     if J - L < R - I then
    +     begin
    +       if L < J then
    +         QuickSort(L, J);
    +       L := I;
    +     end
    +     else
    +     begin
    +       if I < R then
    +         QuickSort(I, R);
    +       R := J;
    +     end;
    +   until L >= R;
    +  end;
    +
    +begin
    +  if FData.Count < 2 then exit;
    +  QuickSort(0, FData.Count-1);
    +end;
    +
    +function TCustomSortedChartSource.GetCount: Integer;
    +begin
    +  Result := FData.Count;
    +end;
    +
    +function TCustomSortedChartSource.GetItem(AIndex: Integer): PChartDataItem;
    +begin
    +  Result := PChartDataItem(FData.Items[AIndex]);
    +end;
    +
    +function TCustomSortedChartSource.IsSorted: Boolean;
    +begin
    +  Result := FSorted;
    +end;
    +
    +procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
    +begin
    +  if FOnCompare = AValue then exit;
    +  FOnCompare := AValue;
    +  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
    +end;
    +
    +procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
    +begin
    +  if FSorted = AValue then exit;
    +  FSorted := AValue;
    +  if Sorted then Sort else Notify;
    +end;
    +
    +procedure TCustomSortedChartSource.SetSortBy(AValue: TChartSortBy);
    +begin
    +  if FSortBy = AValue then exit;
    +  FSortBy := AValue;
    +  if Sorted then Sort;
    +end;
    +
    +procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
    +begin
    +  if FSortDir = AValue then exit;
    +  FSortDir := AValue;
    +  if Sorted then Sort;
    +end;
    +
    +procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
    +begin
    +  if FSortIndex = AValue then exit;
    +  FSortIndex := AValue;
    +  if Sorted then Sort;
    +end;
    +
    +procedure TCustomSortedChartSource.Sort;
    +begin
    +  if csLoading in ComponentState then exit;
    +  if (FSortBy = sbCustom) then begin
    +    if not Assigned(FOnCompare) then exit;
    +    FCompareProc := FOnCompare;
    +  end else begin
    +    if (FSortBy = sbX) and (FSortIndex <> 0) and (FSortIndex >= FXCount) then exit;
    +    if (FSortBy = sbY) and (FSortIndex <> 0) and (FSortIndex >= FYCount) then exit;
    +    FCompareProc := @DefaultCompare;
    +  end;
    +  ExecSort(@DoCompare);
    +  Notify;
    +end;
    +
     end.
     
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 61181)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -19,33 +19,9 @@
       Classes, Types, TAChartUtils, TACustomSource;
     
     type
    -  TChartSortCompare = function(AItem1, AItem2: Pointer): Integer of object;
     
    -  { TCustomSortedChartSource }
    +  { TListChartSource }
     
    -  TCustomSortedChartSource = class(TCustomChartSource)
    -  private
    -    FOnCompare: TChartSortCompare;
    -    procedure SetSorted(AValue: Boolean);
    -    procedure SetOnCompare(AValue: TChartSortCompare);
    -  protected
    -    FCompareProc: TChartSortCompare;
    -    FData: TFPList;
    -    FSorted: Boolean;
    -    function DefaultCompare(AItem1, AItem2: Pointer): Integer; virtual; abstract;
    -    function DoCompare(AItem1, AItem2: Pointer): Integer; virtual;
    -    procedure ExecSort(ACompare: TChartSortCompare); virtual;
    -    procedure SetSortBy(AValue: TChartSortBy); override;
    -    procedure SetSortDir(AValue: TChartSortDir); override;
    -    procedure SetSortIndex(AValue: Cardinal); override;
    -    property Sorted: Boolean read FSorted write SetSorted default false;
    -    property OnCompare: TChartSortCompare read FOnCompare write SetOnCompare;
    -  public
    -    function IsSorted: Boolean; override;
    -    procedure Sort;
    -  end;
    -
    -  { TListChartSource }
       TListChartSource = class(TCustomSortedChartSource)
       private
         FDataPoints: TStrings;
    @@ -56,11 +32,8 @@
         procedure ClearCaches;
         function NewItem: PChartDataItem;
         procedure SetDataPoints(AValue: TStrings);
    -    procedure UpdateCachesAfterAdd(AX, AY: Double);
    +    procedure UpdateCachesAfterAdd(const AX, AY: Double);
       protected
    -    function DefaultCompare(AItem1, AItem2: Pointer): Integer; override;
    -    function GetCount: Integer; override;
    -    function GetItem(AIndex: Integer): PChartDataItem; override;
         procedure Loaded; override;
         procedure SetXCount(AValue: Cardinal); override;
         procedure SetYCount(AValue: Cardinal); override;
    @@ -74,22 +47,22 @@
         destructor Destroy; override;
       public
         function Add(
    -      AX, AY: Double; const ALabel: String = '';
    +      const AX, AY: Double; const ALabel: String = '';
           AColor: TChartColor = clTAColor): Integer;
    -    function AddXListYList(const AX, AY: array of Double; ALabel: String = '';
    +    function AddXListYList(const AX, AY: array of Double; const ALabel: String = '';
           AColor: TChartColor = clTAColor): Integer;
         function AddXYList(
    -      AX: Double; const AY: array of Double; const ALabel: String = '';
    +      const AX: Double; const AY: array of Double; const ALabel: String = '';
           AColor: TChartColor = clTAColor): Integer;
         procedure Clear;
         procedure CopyFrom(ASource: TCustomChartSource);
         procedure Delete(AIndex: Integer);
         procedure SetColor(AIndex: Integer; AColor: TChartColor);
    -    procedure SetText(AIndex: Integer; AValue: String);
    -    function SetXValue(AIndex: Integer; AValue: Double): Integer;
    +    procedure SetText(AIndex: Integer; const AValue: String);
         procedure SetXList(AIndex: Integer; const AXList: array of Double);
    +    function SetXValue(AIndex: Integer; const AValue: Double): Integer;
         procedure SetYList(AIndex: Integer; const AYList: array of Double);
    -    procedure SetYValue(AIndex: Integer; AValue: Double);
    +    procedure SetYValue(AIndex: Integer; const AValue: Double);
       published
         property DataPoints: TStrings read FDataPoints write SetDataPoints;
         property XCount;
    @@ -141,10 +114,10 @@
         procedure SetPointsNumber(AValue: Integer);
         procedure SetRandomX(AValue: Boolean);
         procedure SetRandSeed(AValue: Integer);
    -    procedure SetXMax(AValue: Double);
    -    procedure SetXMin(AValue: Double);
    -    procedure SetYMax(AValue: Double);
    -    procedure SetYMin(AValue: Double);
    +    procedure SetXMax(const AValue: Double);
    +    procedure SetXMin(const AValue: Double);
    +    procedure SetYMax(const AValue: Double);
    +    procedure SetYMin(const AValue: Double);
         procedure SetYNanPercent(AValue: TPercent);
       protected
         procedure ChangeErrorBars(Sender: TObject); override;
    @@ -285,7 +258,7 @@
       strict private
         FSource: TListChartSource;
         FLoadingCache: TStringList;
    -    procedure Parse(AString: String; ADataItem: PChartDataItem);
    +    procedure Parse(const AString: String; ADataItem: PChartDataItem);
       private
         procedure LoadingFinished;
       protected
    @@ -301,18 +274,6 @@
         procedure Insert(Index: Integer; const S: String); override;
       end;
     
    -function CompareFloat(x1, x2: Double): Integer;
    -begin
    -  if IsNaN(x1) and IsNaN(x2) then
    -    Result := 0
    -  else if IsNaN(x1) then
    -    Result := +1
    -  else if IsNaN(x2) then
    -    Result := -1
    -  else
    -    Result := CompareValue(x1, x2);
    -end;
    -
     procedure Register;
     begin
       RegisterComponents(
    @@ -352,7 +313,7 @@
     
     function TListChartSourceStrings.Get(Index: Integer): String;
     
    -  function NumberStr(AValue: Double): String;
    +  function NumberStr(const AValue: Double): String;
       begin
         if IsNaN(AValue) then
           Result := '|'
    @@ -421,7 +382,7 @@
     end;
     
     procedure TListChartSourceStrings.Parse(
    -  AString: String; ADataItem: PChartDataItem);
    +  const AString: String; ADataItem: PChartDataItem);
     var
       p: Integer = 0;
       parts: TStrings;
    @@ -523,120 +484,10 @@
     end;
     
     
    -{ TCustomSortedChartSource }
    -
    -function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
    -begin
    -  Result := FCompareProc(AItem1, AItem2);
    -end;
    -
    -{ Built-in sorting algorithm of the ChartSource, a standard QuickSort.
    -  Copied from the classes unit because the compare function must be a method. }
    -procedure TCustomSortedChartSource.ExecSort(ACompare: TChartSortCompare);
    -
    -  procedure QuickSort(L, R: Longint);
    -  var
    -    I, J: Longint;
    -    P, Q: Pointer;
    -  begin
    -   repeat
    -     I := L;
    -     J := R;
    -     P := FData.List^[(L + R) div 2];
    -     repeat
    -       while ACompare(P, FData.List^[I]) > 0 do
    -         I := I + 1;
    -       while ACompare(P, FData.List^[J]) < 0 do
    -         J := J - 1;
    -       If I <= J then
    -       begin
    -         Q := FData.List^[I];
    -         FData.List^[I] := FData.List^[J];
    -         FData.List^[J] := Q;
    -         I := I + 1;
    -         J := J - 1;
    -       end;
    -     until I > J;
    -     if J - L < R - I then
    -     begin
    -       if L < J then
    -         QuickSort(L, J);
    -       L := I;
    -     end
    -     else
    -     begin
    -       if I < R then
    -         QuickSort(I, R);
    -       R := J;
    -     end;
    -   until L >= R;
    -  end;
    -
    -begin
    -  if FData.Count < 2 then exit;
    -  QuickSort(0, FData.Count-1);
    -end;
    -
    -function TCustomSortedChartSource.IsSorted: Boolean;
    -begin
    -  Result := FSorted;
    -end;
    -
    -procedure TCustomSortedChartSource.SetSortBy(AValue: TChartSortBy);
    -begin
    -  if FSortBy = AValue then exit;
    -  FSortBy := AValue;
    -  if Sorted then Sort;
    -end;
    -
    -procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
    -begin
    -  if FSorted = AValue then exit;
    -  FSorted := AValue;
    -  if Sorted then Sort else Notify;
    -end;
    -
    -procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
    -begin
    -  if FSortDir = AValue then exit;
    -  FSortDir := AValue;
    -  if Sorted then Sort;
    -end;
    -
    -procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
    -begin
    -  if FSortIndex = AValue then exit;
    -  FSortIndex := AValue;
    -  if Sorted then Sort;
    -end;
    -
    -procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
    -begin
    -  if FOnCompare = AValue then exit;
    -  FOnCompare := AValue;
    -  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
    -end;
    -
    -procedure TCustomSortedChartSource.Sort;
    -begin
    -  if csLoading in ComponentState then exit;
    -  if (FSortBy = sbCustom) then begin
    -    if not Assigned(FOnCompare) then exit;
    -    FCompareProc := FOnCompare;
    -  end else begin
    -    if (FSortBy = sbX) and (FSortIndex <> 0) and (FSortIndex >= FXCount) then exit;
    -    if (FSortBy = sbY) and (FSortIndex <> 0) and (FSortIndex >= FYCount) then exit;
    -    FCompareProc := @DefaultCompare;
    -  end;
    -  ExecSort(@DoCompare);
    -  Notify;
    -end;
    -
    -
     { TListChartSource }
     
     function TListChartSource.Add(
    -  AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
    +  const AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
     begin
       Result := FData.Count;
       if IsSortedByXAsc then
    @@ -665,7 +516,7 @@
     end;
     
     function TListChartSource.AddXListYList(const AX, AY: array of Double;
    -  ALabel: String = ''; AColor: TChartColor = clTAColor): Integer;
    +  const ALabel: String; AColor: TChartColor): Integer;
     begin
       if Length(AX) = 0 then
         raise EXListEmptyError.Create('AddXListYList: XList is empty');
    @@ -688,7 +539,7 @@
     end;
     
     function TListChartSource.AddXYList(
    -  AX: Double; const AY: array of Double;
    +  const AX: Double; const AY: array of Double;
       const ALabel: String; AColor: TChartColor): Integer;
     begin
       if Length(AY) = 0 then
    @@ -711,7 +562,7 @@
     var
       i: Integer;
     begin
    -  for i := 0 to FData.Count - 1 do
    +  for i := 0 to Count - 1 do
         Dispose(Item[i]);
       FData.Clear;
       ClearCaches;
    @@ -775,7 +626,6 @@
     constructor TListChartSource.Create(AOwner: TComponent);
     begin
       inherited Create(AOwner);
    -  FData := TFPList.Create;
       FDataPoints := TListChartSourceStrings.Create(Self);
       ClearCaches;
     end;
    @@ -791,21 +641,6 @@
         FYCount := FYCountMin;
     end;
     
    -function TListChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
    -var
    -  item1: PChartDataItem absolute AItem1;
    -  item2: PChartDataItem absolute AItem2;
    -begin
    - case FSortBy of
    -   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
    -   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
    -   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
    -   sbText: Result := CompareText(item1^.Text, item2^.Text);
    -   sbCustom: Result := FOnCompare(AItem1, AItem2);
    - end;
    - if FSortDir = sdDescending then Result := -Result;
    -end;
    -
     procedure TListChartSource.Delete(AIndex: Integer);
     begin
       // Optimization
    @@ -834,20 +669,9 @@
     begin
       Clear;
       FreeAndNil(FDataPoints);
    -  FreeAndNil(FData);
       inherited;
     end;
     
    -function TListChartSource.GetCount: Integer;
    -begin
    -  Result := FData.Count;
    -end;
    -
    -function TListChartSource.GetItem(AIndex: Integer): PChartDataItem;
    -begin
    -  Result := PChartDataItem(FData.Items[AIndex]);
    -end;
    -
     function TListChartSource.NewItem: PChartDataItem;
     begin
       New(Result);
    @@ -876,7 +700,7 @@
       end;
     end;
     
    -procedure TListChartSource.SetText(AIndex: Integer; AValue: String);
    +procedure TListChartSource.SetText(AIndex: Integer; const AValue: String);
     begin
       with Item[AIndex]^ do begin
         if Text = AValue then exit;
    @@ -912,7 +736,7 @@
       Notify;
     end;
     
    -function TListChartSource.SetXValue(AIndex: Integer; AValue: Double): Integer;
    +function TListChartSource.SetXValue(AIndex: Integer; const AValue: Double): Integer;
     var
       oldX: Double;
     
    @@ -935,10 +759,12 @@
       if IsSortedByXAsc then
         if IsNan(AValue) then
           raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
    -  oldX := Item[AIndex]^.X;
       Result := AIndex;
    -  if IsEquivalent(oldX, AValue) then exit;
    -  Item[AIndex]^.X := AValue;
    +  with Item[AIndex]^ do begin
    +    if IsEquivalent(X, AValue) then exit; // IsEquivalent() can compare also NaNs
    +    oldX := X;
    +    X := AValue;
    +  end;
       UpdateExtent;
       if IsSortedByXAsc then begin
         if AValue > oldX then
    @@ -981,7 +807,7 @@
       Notify;
     end;
     
    -procedure TListChartSource.SetYValue(AIndex: Integer; AValue: Double);
    +procedure TListChartSource.SetYValue(AIndex: Integer; const AValue: Double);
     var
       oldY: Double;
     
    @@ -1001,9 +827,11 @@
       end;
     
     begin
    -  oldY := Item[AIndex]^.Y;
    -  if IsEquivalent(oldY, AValue) then exit;
    -  Item[AIndex]^.Y := AValue;
    +  with Item[AIndex]^ do begin
    +    if IsEquivalent(Y, AValue) then exit; // IsEquivalent() can compare also NaNs
    +    oldY := Y;
    +    Y := AValue;
    +  end;
       if FValuesTotalIsValid then
         FValuesTotal += NumberOr(AValue) - NumberOr(oldY);
       UpdateExtent;
    @@ -1010,7 +838,7 @@
       Notify;
     end;
     
    -procedure TListChartSource.UpdateCachesAfterAdd(AX, AY: Double);
    +procedure TListChartSource.UpdateCachesAfterAdd(const AX, AY: Double);
     begin
       if IsUpdating then exit; // Optimization
       if FBasicExtentIsValid then begin
    @@ -1216,7 +1044,7 @@
       Reset;
     end;
     
    -procedure TRandomChartSource.SetXMax(AValue: Double);
    +procedure TRandomChartSource.SetXMax(const AValue: Double);
     begin
       if FXMax = AValue then exit;
       FXMax := AValue;
    @@ -1223,7 +1051,7 @@
       Reset;
     end;
     
    -procedure TRandomChartSource.SetXMin(AValue: Double);
    +procedure TRandomChartSource.SetXMin(const AValue: Double);
     begin
       if FXMin = AValue then exit;
       FXMin := AValue;
    @@ -1237,7 +1065,7 @@
       Reset;
     end;
     
    -procedure TRandomChartSource.SetYMax(AValue: Double);
    +procedure TRandomChartSource.SetYMax(const AValue: Double);
     begin
       if FYMax = AValue then exit;
       FYMax := AValue;
    @@ -1245,7 +1073,7 @@
       Notify;
     end;
     
    -procedure TRandomChartSource.SetYMin(AValue: Double);
    +procedure TRandomChartSource.SetYMin(const AValue: Double);
     begin
       if FYMin = AValue then exit;
       FYMin := AValue;
    
    phase1.diff (23,495 bytes)
  • const_rec.png (25,093 bytes)
    const_rec.png (25,093 bytes)
  • const_str.png (38,842 bytes)
    const_str.png (38,842 bytes)
  • phase2.diff (3,019 bytes)
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 61203)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -1675,7 +1675,16 @@
     
     function TCustomSortedChartSource.IsSorted: Boolean;
     begin
    -  Result := FSorted;
    +  case FSortBy of
    +    sbX:
    +      Result := FSorted and ((FSortIndex = 0) or (FSortIndex < FXCount));
    +    sbY:
    +      Result := FSorted and ((FSortIndex = 0) or (FSortIndex < FYCount));
    +    sbColor, sbText:
    +      Result := FSorted;
    +    sbCustom:
    +      Result := FSorted and Assigned(FOnCompare);
    +  end;
     end;
     
     procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
    @@ -1682,7 +1691,7 @@
     begin
       if FOnCompare = AValue then exit;
       FOnCompare := AValue;
    -  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
    +  if IsSorted then Sort else Notify;
     end;
     
     procedure TCustomSortedChartSource.SetSortBy(AValue: TChartSortBy);
    @@ -1689,7 +1698,7 @@
     begin
       if FSortBy = AValue then exit;
       FSortBy := AValue;
    -  if Sorted then Sort;
    +  if IsSorted then Sort else Notify;
     end;
     
     procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
    @@ -1696,7 +1705,7 @@
     begin
       if FSortDir = AValue then exit;
       FSortDir := AValue;
    -  if Sorted then Sort;
    +  if IsSorted then Sort else Notify;
     end;
     
     procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
    @@ -1703,7 +1712,7 @@
     begin
       if FSorted = AValue then exit;
       FSorted := AValue;
    -  if Sorted then Sort else Notify;
    +  if IsSorted then Sort else Notify;
     end;
     
     procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
    @@ -1710,20 +1719,28 @@
     begin
       if FSortIndex = AValue then exit;
       FSortIndex := AValue;
    -  if Sorted then Sort;
    +  if IsSorted then Sort else Notify;
     end;
     
     procedure TCustomSortedChartSource.Sort;
    +var
    +  SaveSorted: Boolean;
     begin
       if csLoading in ComponentState then exit;
    -  if (FSortBy = sbCustom) then begin
    -    if not Assigned(FOnCompare) then exit;
    -    FCompareProc := FOnCompare;
    -  end else begin
    -    if (FSortBy = sbX) and (FSortIndex <> 0) and (FSortIndex >= FXCount) then exit;
    -    if (FSortBy = sbY) and (FSortIndex <> 0) and (FSortIndex >= FYCount) then exit;
    +
    +  // Avoid useless sorting and notification
    +  SaveSorted := FSorted;
    +  try
    +    FSorted := true;
    +    if not IsSorted then exit;
    +  finally
    +    FSorted := SaveSorted;
    +  end;
    +
    +  if FSortBy = sbCustom then
    +    FCompareProc := FOnCompare
    +  else
         FCompareProc := @DefaultCompare;
    -  end;
       ExecSort(@DoCompare);
       Notify;
     end;
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 61203)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -695,7 +695,7 @@
       BeginUpdate;
       try
         FDataPoints.Assign(AValue);
    -    if Sorted then Sort;
    +    if IsSorted then Sort;
       finally
         EndUpdate;
       end;
    
    phase2.diff (3,019 bytes)
  • phase3.diff (18,275 bytes)
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 61227)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -212,6 +212,7 @@
           out AUpperDelta, ALowerDelta: Double): Boolean;
         function GetHasErrorBars(Which: Integer): Boolean;
         function GetItem(AIndex: Integer): PChartDataItem; virtual; abstract;
    +    function HasSameSorting(ASource: TCustomChartSource): Boolean; virtual;
         procedure InvalidateCaches;
         procedure SetSortBy(AValue: TChartSortBy); virtual;
         procedure SetSortDir(AValue: TChartSortDir); virtual;
    @@ -283,14 +284,16 @@
         procedure SetOnCompare(AValue: TChartSortCompare);
         procedure SetSorted(AValue: Boolean);
       protected
    -    FCompareProc: TChartSortCompare;
         FData: TFPList;
         FSorted: Boolean;
    -    function DefaultCompare(AItem1, AItem2: Pointer): Integer; virtual;
         function DoCompare(AItem1, AItem2: Pointer): Integer; virtual;
    -    procedure ExecSort(ACompare: TChartSortCompare); virtual;
    +    procedure DoSort; virtual;
         function GetCount: Integer; override;
         function GetItem(AIndex: Integer): PChartDataItem; override;
    +    function ItemAdd(AItem: PChartDataItem): Integer;
    +    procedure ItemInsert(AIndex: Integer; AItem: PChartDataItem);
    +    function ItemFind(AItem: PChartDataItem; L: Integer = 0; R: Integer = High(Integer)): Integer;
    +    function ItemModified(AIndex: Integer): Integer;
         procedure SetSortBy(AValue: TChartSortBy); override;
         procedure SetSortDir(AValue: TChartSortDir); override;
         procedure SetSortIndex(AValue: Cardinal); override;
    @@ -1298,6 +1301,20 @@
         end;
     end;
     
    +function TCustomChartSource.HasSameSorting(ASource: TCustomChartSource): Boolean;
    +begin
    +  case SortBy of
    +    sbX, sbY:
    +      Result := ASource.IsSorted and (ASource.SortBy = SortBy) and
    +                (ASource.SortDir = SortDir) and (ASource.SortIndex = SortIndex);
    +    sbColor, sbText:
    +      Result := ASource.IsSorted and (ASource.SortBy = SortBy) and
    +                (ASource.SortDir = SortDir);
    +    sbCustom:
    +      Result := false;
    +  end;
    +end;
    +
     function TCustomChartSource.HasXErrorBars: Boolean;
     begin
       Result := GetHasErrorBars(0);
    @@ -1596,71 +1613,129 @@
         Result := CompareValue(x1, x2);
     end;
     
    -function TCustomSortedChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
    +function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
     var
       item1: PChartDataItem absolute AItem1;
       item2: PChartDataItem absolute AItem2;
    +  d1, d2: Double;
     begin
    - case FSortBy of
    -   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
    -   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
    -   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
    -   sbText: Result := CompareText(item1^.Text, item2^.Text);
    -   sbCustom: Result := FOnCompare(AItem1, AItem2);
    - end;
    - if FSortDir = sdDescending then Result := -Result;
    +  case FSortBy of
    +    sbX:
    +      if FSortIndex = 0 then
    +        Result := CompareFloat(item1^.X, item2^.X)
    +      else
    +      if FSortIndex < FXCount then begin
    +        if FSortIndex <= Cardinal(Length(item1^.XList)) then
    +          d1 := item1^.XList[FSortIndex - 1]
    +        else
    +          d1 := SafeNan;
    +        if FSortIndex <= Cardinal(Length(item2^.XList)) then
    +          d2 := item2^.XList[FSortIndex - 1]
    +        else
    +          d2 := SafeNan;
    +        Result := CompareFloat(d1, d2);
    +      end else
    +        Result := 0;
    +    sbY:
    +      if FSortIndex = 0 then
    +        Result := CompareFloat(item1^.Y, item2^.Y)
    +      else
    +      if FSortIndex < FYCount then begin
    +        if FSortIndex <= Cardinal(Length(item1^.YList)) then
    +          d1 := item1^.YList[FSortIndex - 1]
    +        else
    +          d1 := SafeNan;
    +        if FSortIndex <= Cardinal(Length(item2^.YList)) then
    +          d2 := item2^.YList[FSortIndex - 1]
    +        else
    +          d2 := SafeNan;
    +        Result := CompareFloat(d1, d2);
    +      end else
    +        Result := 0;
    +    sbColor:
    +      Result := CompareValue(item1^.Color, item2^.Color);
    +    sbText:
    +      Result := CompareText(item1^.Text, item2^.Text);
    +    sbCustom:
    +      if Assigned(FOnCompare) then
    +        Result := FOnCompare(AItem1, AItem2)
    +      else
    +        Result := 0;
    +  end;
    +  if FSortDir = sdDescending then Result := -Result;
     end;
     
    -function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
    -begin
    -  Result := FCompareProc(AItem1, AItem2);
    -end;
    +{ Built-in sorting algorithm of the ChartSource - a QuickSort algorithm, copied
    +  from the Classes unit and modified. Modifications are:
    +  - uses a DoCompare() virtual method for comparisons,
    +  - does NOT exchange equal items - this would have some side effect here: let's
    +    consider sorting by X, in the ascending order, for the following data points:
    +      X=3, Text='ccc'
    +      X=2, Text='bbb 1'
    +      X=2, Text='bbb 2'
    +      X=2, Text='bbb 3'
    +      X=1, Text='aaa'
     
    -{ Built-in sorting algorithm of the ChartSource, a standard QuickSort.
    -  Copied from the classes unit because the compare function must be a method. }
    -procedure TCustomSortedChartSource.ExecSort(ACompare: TChartSortCompare);
    +    after sorting, data would be (note the reversed 'bbb' order):
    +      X=1, Text='aaa'
    +      X=2, Text='bbb 3'
    +      X=2, Text='bbb 2'
    +      X=2, Text='bbb 1'
    +      X=3, Text='ccc'
     
    +    after sorting AGAIN, data would be (note the original 'bbb' order):
    +      X=1, Text='aaa'
    +      X=2, Text='bbb 1'
    +      X=2, Text='bbb 2'
    +      X=2, Text='bbb 3'
    +      X=3, Text='ccc'
    +}
    +procedure TCustomSortedChartSource.DoSort;
    +
       procedure QuickSort(L, R: Longint);
       var
         I, J: Longint;
         P, Q: Pointer;
       begin
    -   repeat
    -     I := L;
    -     J := R;
    -     P := FData.List^[(L + R) div 2];
    -     repeat
    -       while ACompare(P, FData.List^[I]) > 0 do
    -         I := I + 1;
    -       while ACompare(P, FData.List^[J]) < 0 do
    -         J := J - 1;
    -       If I <= J then
    -       begin
    -         Q := FData.List^[I];
    -         FData.List^[I] := FData.List^[J];
    -         FData.List^[J] := Q;
    -         I := I + 1;
    -         J := J - 1;
    -       end;
    -     until I > J;
    -     if J - L < R - I then
    -     begin
    -       if L < J then
    -         QuickSort(L, J);
    -       L := I;
    -     end
    -     else
    -     begin
    -       if I < R then
    -         QuickSort(I, R);
    -       R := J;
    -     end;
    -   until L >= R;
    +    repeat
    +      I := L;
    +      J := R;
    +      P := FData.List^[(L + R) div 2];
    +      repeat
    +        while DoCompare(P, FData.List^[I]) > 0 do
    +          I := I + 1;
    +        while DoCompare(P, FData.List^[J]) < 0 do
    +          J := J - 1;
    +        if I <= J then
    +        begin
    +          // do NOT exchange equal items
    +          if DoCompare(FData.List^[I], FData.List^[J]) <> 0 then begin
    +            Q := FData.List^[I];
    +            FData.List^[I] := FData.List^[J];
    +            FData.List^[J] := Q;
    +          end;
    +          I := I + 1;
    +          J := J - 1;
    +        end;
    +      until I > J;
    +      if J - L < R - I then
    +      begin
    +        if L < J then
    +          QuickSort(L, J);
    +        L := I;
    +      end
    +      else
    +      begin
    +        if I < R then
    +          QuickSort(I, R);
    +        R := J;
    +      end;
    +    until L >= R;
       end;
     
     begin
       if FData.Count < 2 then exit;
    -  QuickSort(0, FData.Count-1);
    +  QuickSort(0, FData.Count - 1);
     end;
     
     function TCustomSortedChartSource.GetCount: Integer;
    @@ -1673,6 +1748,78 @@
       Result := PChartDataItem(FData.Items[AIndex]);
     end;
     
    +function TCustomSortedChartSource.ItemAdd(AItem: PChartDataItem): Integer;
    +begin
    +  if IsSorted then begin
    +    Result := ItemFind(AItem);
    +    FData.Insert(Result, AItem);
    +  end else
    +    Result := FData.Add(AItem);
    +end;
    +
    +procedure TCustomSortedChartSource.ItemInsert(AIndex: Integer; AItem: PChartDataItem);
    +begin
    +  if IsSorted then
    +    if AIndex <> ItemFind(AItem) then
    +      raise ESortError.CreateFmt('%0:s.ItemInsert cannot insert data at the requested '+
    +        'position, because source is sorted', [ClassName]);
    +  FData.Insert(AIndex, AItem);
    +end;
    +
    +function TCustomSortedChartSource.ItemFind(AItem: PChartDataItem; L: Integer = 0; R: Integer = High(Integer)): Integer;
    +var
    +  I: Integer;
    +begin
    +  if not IsSorted then
    +    raise ESortError.CreateFmt('%0:s.ItemFind can be called only for sorted source', [ClassName]);
    +
    +  if R >= FData.Count then
    +    R := FData.Count - 1;
    +
    +  // special optimization for adding sorted data at the end
    +  if R >= 0 then
    +    if DoCompare(FData.List^[R], AItem) <= 0 then
    +      exit(R + 1);
    +
    +  // use binary search
    +  if L < 0 then
    +    L := 0;
    +  while L <= R do
    +  begin
    +    I := L + (R - L) div 2;
    +    if DoCompare(FData.List^[I], AItem) <= 0 then
    +      L := I + 1
    +    else
    +      R := I - 1;
    +  end;
    +  Result := L;
    +end;
    +
    +function TCustomSortedChartSource.ItemModified(AIndex: Integer): Integer;
    +begin
    +  Result := AIndex;
    +  if IsSorted then begin
    +    if FData.Count < 2 then exit;
    +    if (AIndex < 0) or (AIndex >= FData.Count) then exit;
    +
    +    if AIndex > 0 then
    +      if DoCompare(FData.List^[AIndex - 1], FData.List^[AIndex]) > 0 then begin
    +        Result := ItemFind(FData.List^[AIndex], 0, AIndex - 1);
    +        // no Dec(Result) here, as it is below
    +        FData.Move(AIndex, Result);
    +        exit; // optimization: the item cannot be unsorted from both sides
    +              // simultaneously, so we can exit now
    +      end;
    +
    +    if AIndex < FData.Count - 1 then
    +      if DoCompare(FData.List^[AIndex], FData.List^[AIndex + 1]) > 0 then begin
    +        Result := ItemFind(FData.List^[AIndex], AIndex + 1, FData.Count - 1);
    +        Dec(Result);
    +        FData.Move(AIndex, Result);
    +      end;
    +  end;
    +end;
    +
     function TCustomSortedChartSource.IsSorted: Boolean;
     begin
       case FSortBy of
    @@ -1737,11 +1884,7 @@
         FSorted := SaveSorted;
       end;
     
    -  if FSortBy = sbCustom then
    -    FCompareProc := FOnCompare
    -  else
    -    FCompareProc := @DefaultCompare;
    -  ExecSort(@DoCompare);
    +  DoSort;
       Notify;
     end;
     
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 61227)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -27,8 +27,6 @@
         FDataPoints: TStrings;
         FXCountMin: Cardinal;
         FYCountMin: Cardinal;
    -    procedure AddAt(
    -      APos: Integer; const AX, AY: Double; const ALabel: String; AColor: TChartColor);
         procedure ClearCaches;
         function NewItem: PChartDataItem;
         procedure SetDataPoints(const AValue: TStrings);
    @@ -57,12 +55,12 @@
         procedure Clear;
         procedure CopyFrom(ASource: TCustomChartSource);
         procedure Delete(AIndex: Integer);
    -    procedure SetColor(AIndex: Integer; AColor: TChartColor);
    -    procedure SetText(AIndex: Integer; const AValue: String);
    -    procedure SetXList(AIndex: Integer; const AXList: array of Double);
    +    function SetColor(AIndex: Integer; AColor: TChartColor): Integer;
    +    function SetText(AIndex: Integer; const AValue: String): Integer;
    +    function SetXList(AIndex: Integer; const AXList: array of Double): Integer;
         function SetXValue(AIndex: Integer; const AValue: Double): Integer;
    -    procedure SetYList(AIndex: Integer; const AYList: array of Double);
    -    procedure SetYValue(AIndex: Integer; const AValue: Double);
    +    function SetYList(AIndex: Integer; const AYList: array of Double): Integer;
    +    function SetYValue(AIndex: Integer; const AValue: Double): Integer;
       published
         property DataPoints: TStrings read FDataPoints write SetDataPoints;
         property XCount;
    @@ -373,7 +371,7 @@
       item := FSource.NewItem;
       try
         Parse(S, item);
    -    FSource.FData.Insert(Index, item);
    +    FSource.ItemInsert(Index, item);
       except
         Dispose(item);
         raise;
    @@ -489,30 +487,20 @@
     function TListChartSource.Add(
       const AX, AY: Double; const ALabel: String = '';
       const AColor: TChartColor = clTAColor): Integer;
    -begin
    -  Result := FData.Count;
    -  if IsSortedByXAsc then
    -    // Keep data points ordered by X coordinate.
    -    // Note that this leads to O(N^2) time except
    -    // for the case of adding already ordered points.
    -    // So, is the user wants to add many (>10000) points to a graph,
    -    // he should pre-sort them to avoid performance penalty.
    -    while (Result > 0) and (Item[Result - 1]^.X > AX) do
    -      Dec(Result);
    -  AddAt(Result, AX, AY, ALabel, AColor);
    -end;
    -
    -procedure TListChartSource.AddAt(
    -  APos: Integer; const AX, AY: Double; const ALabel: String; AColor: TChartColor);
     var
       pcd: PChartDataItem;
     begin
       pcd := NewItem;
    -  pcd^.X := AX;
    -  pcd^.Y := AY;
    -  pcd^.Color := AColor;
    -  pcd^.Text := ALabel;
    -  FData.Insert(APos, pcd);
    +  try
    +    pcd^.X := AX;
    +    pcd^.Y := AY;
    +    pcd^.Color := AColor;
    +    pcd^.Text := ALabel;
    +    Result := ItemAdd(pcd);
    +  except
    +    Dispose(pcd);
    +    raise;
    +  end;
       UpdateCachesAfterAdd(AX, AY);
     end;
     
    @@ -530,9 +518,9 @@
       try
         Result := Add(AX[0], AY[0], ALabel, AColor);
         if Length(AX) > 1 then
    -      SetXList(Result, AX[1..High(AX)]);
    +      Result := SetXList(Result, AX[1..High(AX)]);
         if Length(AY) > 1 then
    -      SetYList(Result, AY[1..High(AY)]);
    +      Result := SetYList(Result, AY[1..High(AY)]);
       finally
         Dec(FUpdateCount);
       end;
    @@ -552,7 +540,7 @@
       try
         Result := Add(AX, AY[0], ALabel, AColor);
         if Length(AY) > 1 then
    -      SetYList(Result, AY[1..High(AY)]);
    +      Result := SetYList(Result, AY[1..High(AY)]);
       finally
         Dec(FUpdateCount);
       end;
    @@ -591,6 +579,7 @@
     procedure TListChartSource.CopyFrom(ASource: TCustomChartSource);
     var
       i: Integer;
    +  pcd: PChartDataItem;
     begin
       if ASource.XCount < FXCountMin then
         raise EXCountError.CreateFmt(rsSourceCountError2, [ClassName, FXCountMin, 'x']);
    @@ -602,23 +591,23 @@
         Clear;
         XCount := ASource.XCount;
         YCount := ASource.YCount;
    -    for i := 0 to ASource.Count - 1 do
    -      with ASource[i]^ do begin
    -        AddAt(FData.Count, X, Y, Text, Color);
    -        SetXList(FData.Count - 1, XList);
    -        SetYList(FData.Count - 1, YList);
    +    FData.Capacity := ASource.Count;
    +
    +    pcd := nil;
    +    try // optimization: don't execute try..except..end in a loop
    +      for i := 0 to ASource.Count - 1 do begin
    +        pcd := NewItem;
    +        pcd^ := ASource[i]^;
    +        FData.Add(pcd); // don't use ItemAdd() here
    +        pcd := nil;
           end;
    +    except
    +      if pcd <> nil then
    +        Dispose(pcd);
    +      raise;
    +    end;
     
    -    if IsSorted then begin
    -      if ASource.IsSorted and
    -        (SortBy = TCustomChartSourceAccess(ASource).SortBy) and
    -        (SortDir = TCustomChartSourceAccess(ASource).SortDir) and
    -        (SortIndex = TCustomChartSourceAccess(ASource).SortIndex) and
    -        (SortBy <> sbCustom)
    -      then
    -        exit;
    -      Sort;
    -    end;
    +    if IsSorted and (not HasSameSorting(ASource)) then Sort;
       finally
         EndUpdate;
       end;
    @@ -680,12 +669,13 @@
       if YCount > 1 then SetLength(Result^.YList, YCount - 1);
     end;
     
    -procedure TListChartSource.SetColor(AIndex: Integer; AColor: TChartColor);
    +function TListChartSource.SetColor(AIndex: Integer; AColor: TChartColor): Integer;
     begin
       with Item[AIndex]^ do begin
    -    if Color = AColor then exit;
    +    if Color = AColor then exit(AIndex);
         Color := AColor;
       end;
    +  Result := ItemModified(AIndex);
       Notify;
     end;
     
    @@ -701,12 +691,13 @@
       end;
     end;
     
    -procedure TListChartSource.SetText(AIndex: Integer; const AValue: String);
    +function TListChartSource.SetText(AIndex: Integer; const AValue: String): Integer;
     begin
       with Item[AIndex]^ do begin
    -    if Text = AValue then exit;
    +    if Text = AValue then exit(AIndex);
         Text := AValue;
       end;
    +  Result := ItemModified(AIndex);
       Notify;
     end;
     
    @@ -725,8 +716,8 @@
       Notify;
     end;
     
    -procedure TListChartSource.SetXList(
    -  AIndex: Integer; const AXList: array of Double);
    +function TListChartSource.SetXList(
    +  AIndex: Integer; const AXList: array of Double): Integer;
     var
       i: Integer;
     begin
    @@ -734,6 +725,7 @@
         for i := 0 to Min(High(AXList), High(XList)) do
           XList[i] := AXList[i];
       FXListExtentIsValid := false;
    +  Result := ItemModified(AIndex);
       Notify;
     end;
     
    @@ -757,26 +749,13 @@
       end;
     
     begin
    -  if IsSortedByXAsc then
    -    if IsNan(AValue) then
    -      raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
    -  Result := AIndex;
       with Item[AIndex]^ do begin
    -    if IsEquivalent(X, AValue) then exit; // IsEquivalent() can compare also NaNs
    +    if IsEquivalent(X, AValue) then exit(AIndex); // IsEquivalent() can compare also NaNs
         oldX := X;
         X := AValue;
       end;
       UpdateExtent;
    -  if IsSortedByXAsc then begin
    -    if AValue > oldX then
    -      while (Result < Count - 1) and (Item[Result + 1]^.X < AValue) do
    -        Inc(Result)
    -    else
    -      while (Result > 0) and (Item[Result - 1]^.X > AValue) do
    -        Dec(Result);
    -    if Result <> AIndex then
    -      FData.Move(AIndex, Result);
    -  end;
    +  Result := ItemModified(AIndex);
       Notify;
     end;
     
    @@ -795,8 +774,8 @@
       Notify;
     end;
     
    -procedure TListChartSource.SetYList(
    -  AIndex: Integer; const AYList: array of Double);
    +function TListChartSource.SetYList(
    +  AIndex: Integer; const AYList: array of Double): Integer;
     var
       i: Integer;
     begin
    @@ -805,10 +784,11 @@
           YList[i] := AYList[i];
       FCumulativeExtentIsValid := false;
       FYListExtentIsValid := false;
    +  Result := ItemModified(AIndex);
       Notify;
     end;
     
    -procedure TListChartSource.SetYValue(AIndex: Integer; const AValue: Double);
    +function TListChartSource.SetYValue(AIndex: Integer; const AValue: Double): Integer;
     var
       oldY: Double;
     
    @@ -829,7 +809,7 @@
     
     begin
       with Item[AIndex]^ do begin
    -    if IsEquivalent(Y, AValue) then exit; // IsEquivalent() can compare also NaNs
    +    if IsEquivalent(Y, AValue) then exit(AIndex); // IsEquivalent() can compare also NaNs
         oldY := Y;
         Y := AValue;
       end;
    @@ -836,6 +816,7 @@
       if FValuesTotalIsValid then
         FValuesTotal += NumberOr(AValue) - NumberOr(oldY);
       UpdateExtent;
    +  Result := ItemModified(AIndex);
       Notify;
     end;
     
    
    phase3.diff (18,275 bytes)
  • phase4.diff (7,500 bytes)
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 61228)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -236,7 +236,7 @@
         procedure EndUpdate; override;
       public
         class procedure CheckFormat(const AFormat: String);
    -    function BasicExtent: TDoubleRect;
    +    function BasicExtent: TDoubleRect; virtual;
         function Extent: TDoubleRect; virtual;
         function ExtentCumulative: TDoubleRect; virtual;
         function ExtentList: TDoubleRect; virtual;
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 61228)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -77,6 +77,41 @@
         property OnCompare;
       end;
     
    +  { TSortedChartSource }
    +
    +  TSortedChartSource = class(TCustomSortedChartSource)
    +  strict private
    +    FListener: TListener;
    +    FListenerSelf: TListener;
    +    FOrigin: TCustomChartSource;
    +    procedure Changed(ASender: TObject);
    +    procedure SetOrigin(AValue: TCustomChartSource);
    +  protected
    +    function DoCompare(AItem1, AItem2: Pointer): Integer; override;
    +    function GetCount: Integer; override;
    +    function GetItem(AIndex: Integer): PChartDataItem; override;
    +    procedure ResetTransformation(ACount: Integer);
    +    procedure SetXCount(AValue: Cardinal); override;
    +    procedure SetYCount(AValue: Cardinal); override;
    +  public
    +    constructor Create(AOwner: TComponent); override;
    +    destructor Destroy; override;
    +  published
    +    function BasicExtent: TDoubleRect; override;
    +    function Extent: TDoubleRect; override;
    +    function ExtentCumulative: TDoubleRect; override;
    +    function ExtentList: TDoubleRect; override;
    +    function ExtentXYList: TDoubleRect; override;
    +    function ValuesTotal: Double; override;
    +    property Origin: TCustomChartSource read FOrigin write SetOrigin;
    +    // Sorting
    +    property SortBy;
    +    property SortDir;
    +    property Sorted;
    +    property SortIndex;
    +    property OnCompare;
    +  end;
    +  
       { TMWCRandomGenerator }
     
       // Mutliply-with-carry random number generator.
    @@ -278,12 +313,11 @@
     begin
       RegisterComponents(
         CHART_COMPONENT_IDE_PAGE, [
    -      TListChartSource, TRandomChartSource, TUserDefinedChartSource,
    -      TCalculatedChartSource
    +      TListChartSource, TSortedChartSource, TRandomChartSource,
    +      TUserDefinedChartSource, TCalculatedChartSource
         ]);
     end;
     
    -
     { TListChartSourceStrings }
     
     procedure TListChartSourceStrings.Clear;
    @@ -483,7 +517,6 @@
         end;
     end;
     
    -
     { TListChartSource }
     
     function TListChartSource.Add(
    @@ -860,6 +893,185 @@
       (FDataPoints as TListChartSourceStrings).LoadingFinished;
     end;
     
    +{ TSortedChartSource }
    +
    +constructor TSortedChartSource.Create(AOwner: TComponent);
    +begin
    +  inherited Create(AOwner);
    +  FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
    +  FYCount := MaxInt;
    +  FListener := TListener.Create(@FOrigin, @Changed);
    +  FListenerSelf := TListener.Create(nil, @Changed);
    +  Broadcaster.Subscribe(FListenerSelf);
    +end;
    +
    +destructor TSortedChartSource.Destroy;
    +begin
    +  ResetTransformation(0);
    +  FreeAndNil(FListenerSelf);
    +  FreeAndNil(FListener);
    +  inherited;
    +end;
    +
    +procedure TSortedChartSource.Changed(ASender: TObject);
    +begin
    +  if ASender = Self then begin
    +    // We can get here only due to FListenerSelf's notification.
    +    // If some of our own (not Origin's) sorting properties was changed and we
    +    // are sorted, then our Sort() method has been called, so the transformation
    +    // is valid; but if we are no longer sorted, only notification is sent (so
    +    // we are here), so we must reinitialize the transformation to return to
    +    // the transparent (i.e. unsorted) state.
    +    if not IsSorted then
    +      ResetTransformation(Count);
    +    exit;
    +  end;
    +
    +  if FOrigin <> nil then begin
    +    FXCount := Origin.XCount;
    +    FYCount := Origin.YCount;
    +    ResetTransformation(Origin.Count);
    +    if IsSorted and (not HasSameSorting(Origin)) then Sort else Notify;
    +  end else begin
    +    FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
    +    FYCount := MaxInt;
    +    ResetTransformation(0);
    +    Notify;
    +  end;
    +end;
    +
    +function TSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
    +begin
    +  Result := inherited DoCompare(Origin.Item[PInteger(AItem1)^],
    +                                Origin.Item[PInteger(AItem2)^]);
    +end;
    +
    +function TSortedChartSource.GetCount: Integer;
    +begin
    +  if Origin <> nil then
    +    Result := Origin.Count
    +  else
    +    Result := 0;
    +end;
    +
    +function TSortedChartSource.GetItem(AIndex: Integer): PChartDataItem;
    +begin
    +  if Origin <> nil then
    +    Result := PChartDataItem(Origin.Item[PInteger(FData.Items[AIndex])^])
    +  else
    +    Result := nil;
    +end;
    +
    +procedure TSortedChartSource.ResetTransformation(ACount: Integer);
    +var
    +  i: Integer;
    +  pint: PInteger;
    +begin
    +  if ACount > FData.Count then begin
    +    for i := 0 to FData.Count - 1 do
    +      PInteger(FData.List^[i])^ := i;
    +
    +    FData.Capacity := ACount;
    +
    +    pint := nil;
    +    try // optimization: don't execute try..except..end in a loop
    +      for i := FData.Count to ACount - 1 do begin
    +        New(pint);
    +        pint^ := i;
    +        FData.Add(pint); // don't use ItemAdd() here
    +        pint := nil;
    +      end;
    +    except
    +      if pint <> nil then
    +        Dispose(pint);
    +      raise;
    +    end;
    +  end else
    +  begin
    +    for i := ACount to FData.Count - 1 do
    +      Dispose(PInteger(FData.List^[i]));
    +
    +    FData.Count := ACount;
    +    FData.Capacity := ACount; // release needless memory
    +
    +    for i := 0 to FData.Count - 1 do
    +      PInteger(FData.List^[i])^ := i;
    +  end;
    +end;
    +
    +procedure TSortedChartSource.SetOrigin(AValue: TCustomChartSource);
    +begin
    +  if AValue = Self then
    +    AValue := nil;
    +  if FOrigin = AValue then exit;
    +  if FOrigin <> nil then
    +    FOrigin.Broadcaster.Unsubscribe(FListener);
    +  FOrigin := AValue;
    +  if FOrigin <> nil then
    +    FOrigin.Broadcaster.Subscribe(FListener);
    +  Changed(nil);
    +end;
    +
    +procedure TSortedChartSource.SetXCount(AValue: Cardinal);
    +begin
    +  Unused(AValue);
    +  raise EXCountError.Create('Cannot set XCount');
    +end;
    +
    +procedure TSortedChartSource.SetYCount(AValue: Cardinal);
    +begin
    +  Unused(AValue);
    +  raise EYCountError.Create('Cannot set YCount');
    +end;
    +
    +function TSortedChartSource.BasicExtent: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.BasicExtent;
    +end;
    +
    +function TSortedChartSource.Extent: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.Extent;
    +end;
    +
    +function TSortedChartSource.ExtentCumulative: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.ExtentCumulative;
    +end;
    +
    +function TSortedChartSource.ExtentList: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.ExtentList;
    +end;
    +
    +function TSortedChartSource.ExtentXYList: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.ExtentXYList;
    +end;
    +
    +function TSortedChartSource.ValuesTotal: Double;
    +begin
    +  if Origin = nil then
    +    Result := 0
    +  else
    +    Result := Origin.ValuesTotal;
    +end;
    +
     { TMWCRandomGenerator }
     
     function TMWCRandomGenerator.Get: LongWord;
    
    phase4.diff (7,500 bytes)
  • Test-component.zip (2,886 bytes)
  • phase5.diff (640 bytes)
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 61245)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -1586,11 +1586,13 @@
     
     function CompareFloat(const x1, x2: Double): Integer;
     begin
    -  if IsNaN(x1) and IsNaN(x2) then
    -    Result := 0
    -  else if IsNaN(x1) then
    -    Result := +1
    -  else if IsNaN(x2) then
    +  if IsNaN(x1) then begin
    +    if IsNaN(x2) then
    +      Result := 0
    +    else
    +      Result := +1;
    +  end else
    +  if IsNaN(x2) then
         Result := -1
       else
         Result := CompareValue(x1, x2);
    
    phase5.diff (640 bytes)
  • sorted_source_userdef_source.zip (3,057 bytes)
  • phase4_updated.diff (8,733 bytes)
    Index: components/tachart/tacustomsource.pas
    ===================================================================
    --- components/tachart/tacustomsource.pas	(revision 61248)
    +++ components/tachart/tacustomsource.pas	(working copy)
    @@ -99,6 +99,7 @@
         procedure SetY(const AValue: Double);
         procedure MultiplyY(const ACoeff: Double);
         function Point: TDoublePoint; inline;
    +    procedure MakeUnique;
       end;
       PChartDataItem = ^TChartDataItem;
     
    @@ -237,7 +238,7 @@
         procedure EndUpdate; override;
       public
         class procedure CheckFormat(const AFormat: String);
    -    function BasicExtent: TDoubleRect;
    +    function BasicExtent: TDoubleRect; virtual;
         function Extent: TDoubleRect; virtual;
         function ExtentCumulative: TDoubleRect; virtual;
         function ExtentList: TDoubleRect; virtual;
    @@ -538,6 +539,15 @@
         Result := YList[AIndex - 1];
     end;
     
    +procedure TChartDataItem.MakeUnique;
    +begin
    +  // using SetLength() is a documented way of making the dynamic array unique:
    +  // "the reference count after a call to SetLength will be 1"
    +  UniqueString(Text);
    +  SetLength(XList, Length(XList));
    +  SetLength(YList, Length(YList));
    +end;
    +
     procedure TChartDataItem.MultiplyY(const ACoeff: Double);
     var
       i: Integer;
    Index: components/tachart/tasources.pas
    ===================================================================
    --- components/tachart/tasources.pas	(revision 61248)
    +++ components/tachart/tasources.pas	(working copy)
    @@ -75,6 +75,41 @@
         property OnCompare;
       end;
     
    +  { TSortedChartSource }
    +
    +  TSortedChartSource = class(TCustomSortedChartSource)
    +  strict private
    +    FListener: TListener;
    +    FListenerSelf: TListener;
    +    FOrigin: TCustomChartSource;
    +    procedure Changed(ASender: TObject);
    +    procedure SetOrigin(AValue: TCustomChartSource);
    +  protected
    +    function DoCompare(AItem1, AItem2: Pointer): Integer; override;
    +    function GetCount: Integer; override;
    +    function GetItem(AIndex: Integer): PChartDataItem; override;
    +    procedure ResetTransformation(ACount: Integer);
    +    procedure SetXCount(AValue: Cardinal); override;
    +    procedure SetYCount(AValue: Cardinal); override;
    +  public
    +    constructor Create(AOwner: TComponent); override;
    +    destructor Destroy; override;
    +  published
    +    function BasicExtent: TDoubleRect; override;
    +    function Extent: TDoubleRect; override;
    +    function ExtentCumulative: TDoubleRect; override;
    +    function ExtentList: TDoubleRect; override;
    +    function ExtentXYList: TDoubleRect; override;
    +    function ValuesTotal: Double; override;
    +    property Origin: TCustomChartSource read FOrigin write SetOrigin;
    +    // Sorting
    +    property SortBy;
    +    property SortDir;
    +    property Sorted;
    +    property SortIndex;
    +    property OnCompare;
    +  end;
    +
       { TMWCRandomGenerator }
     
       // Mutliply-with-carry random number generator.
    @@ -276,12 +311,11 @@
     begin
       RegisterComponents(
         CHART_COMPONENT_IDE_PAGE, [
    -      TListChartSource, TRandomChartSource, TUserDefinedChartSource,
    -      TCalculatedChartSource
    +      TListChartSource, TSortedChartSource, TRandomChartSource,
    +      TUserDefinedChartSource, TCalculatedChartSource
         ]);
     end;
     
    -
     { TListChartSourceStrings }
     
     procedure TListChartSourceStrings.Clear;
    @@ -481,7 +515,6 @@
         end;
     end;
     
    -
     { TListChartSource }
     
     function TListChartSource.Add(
    @@ -841,6 +874,196 @@
       (FDataPoints as TListChartSourceStrings).LoadingFinished;
     end;
     
    +{ TSortedChartSource }
    +
    +constructor TSortedChartSource.Create(AOwner: TComponent);
    +begin
    +  inherited Create(AOwner);
    +  FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
    +  FYCount := MaxInt;
    +  FListener := TListener.Create(@FOrigin, @Changed);
    +  FListenerSelf := TListener.Create(nil, @Changed);
    +  Broadcaster.Subscribe(FListenerSelf);
    +end;
    +
    +destructor TSortedChartSource.Destroy;
    +begin
    +  ResetTransformation(0);
    +  FreeAndNil(FListenerSelf);
    +  FreeAndNil(FListener);
    +  inherited;
    +end;
    +
    +procedure TSortedChartSource.Changed(ASender: TObject);
    +begin
    +  if ASender = Self then begin
    +    // We can get here only due to FListenerSelf's notification.
    +    // If some of our own (not Origin's) sorting properties was changed and we
    +    // are sorted, then our Sort() method has been called, so the transformation
    +    // is valid; but if we are no longer sorted, only notification is sent (so
    +    // we are here), so we must reinitialize the transformation to return to
    +    // the transparent (i.e. unsorted) state.
    +    if not IsSorted then
    +      ResetTransformation(Count);
    +    exit;
    +  end;
    +
    +  if FOrigin <> nil then begin
    +    FXCount := Origin.XCount;
    +    FYCount := Origin.YCount;
    +    ResetTransformation(Origin.Count);
    +    if IsSorted and (not HasSameSorting(Origin)) then Sort else Notify;
    +  end else begin
    +    FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
    +    FYCount := MaxInt;
    +    ResetTransformation(0);
    +    Notify;
    +  end;
    +end;
    +
    +function TSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
    +var
    +  item1, item2: TChartDataItem;
    +begin
    +  // some data sources use same memory buffer for every item read,
    +  // so local copies must be made before comparing two items
    +  item1 := Origin.Item[PInteger(AItem1)^]^;
    +
    +  // avoid sharing same memory by item1's and item2's reference-
    +  // counted variables
    +  item1.MakeUnique;
    +
    +  item2 := Origin.Item[PInteger(AItem2)^]^;
    +
    +  Result := inherited DoCompare(@item1, @item2);
    +end;
    +
    +function TSortedChartSource.GetCount: Integer;
    +begin
    +  if Origin <> nil then
    +    Result := Origin.Count
    +  else
    +    Result := 0;
    +end;
    +
    +function TSortedChartSource.GetItem(AIndex: Integer): PChartDataItem;
    +begin
    +  if Origin <> nil then
    +    Result := PChartDataItem(Origin.Item[PInteger(FData.Items[AIndex])^])
    +  else
    +    Result := nil;
    +end;
    +
    +procedure TSortedChartSource.ResetTransformation(ACount: Integer);
    +var
    +  i: Integer;
    +  pint: PInteger;
    +begin
    +  if ACount > FData.Count then begin
    +    for i := 0 to FData.Count - 1 do
    +      PInteger(FData.List^[i])^ := i;
    +
    +    FData.Capacity := ACount;
    +
    +    pint := nil;
    +    try // optimization: don't execute try..except..end in a loop
    +      for i := FData.Count to ACount - 1 do begin
    +        New(pint);
    +        pint^ := i;
    +        FData.Add(pint); // don't use ItemAdd() here
    +        pint := nil;
    +      end;
    +    except
    +      if pint <> nil then
    +        Dispose(pint);
    +      raise;
    +    end;
    +  end else
    +  begin
    +    for i := ACount to FData.Count - 1 do
    +      Dispose(PInteger(FData.List^[i]));
    +
    +    FData.Count := ACount;
    +    FData.Capacity := ACount; // release needless memory
    +
    +    for i := 0 to FData.Count - 1 do
    +      PInteger(FData.List^[i])^ := i;
    +  end;
    +end;
    +
    +procedure TSortedChartSource.SetOrigin(AValue: TCustomChartSource);
    +begin
    +  if AValue = Self then
    +    AValue := nil;
    +  if FOrigin = AValue then exit;
    +  if FOrigin <> nil then
    +    FOrigin.Broadcaster.Unsubscribe(FListener);
    +  FOrigin := AValue;
    +  if FOrigin <> nil then
    +    FOrigin.Broadcaster.Subscribe(FListener);
    +  Changed(nil);
    +end;
    +
    +procedure TSortedChartSource.SetXCount(AValue: Cardinal);
    +begin
    +  Unused(AValue);
    +  raise EXCountError.Create('Cannot set XCount');
    +end;
    +
    +procedure TSortedChartSource.SetYCount(AValue: Cardinal);
    +begin
    +  Unused(AValue);
    +  raise EYCountError.Create('Cannot set YCount');
    +end;
    +
    +function TSortedChartSource.BasicExtent: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.BasicExtent;
    +end;
    +
    +function TSortedChartSource.Extent: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.Extent;
    +end;
    +
    +function TSortedChartSource.ExtentCumulative: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.ExtentCumulative;
    +end;
    +
    +function TSortedChartSource.ExtentList: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.ExtentList;
    +end;
    +
    +function TSortedChartSource.ExtentXYList: TDoubleRect;
    +begin
    +  if Origin = nil then
    +    Result := EmptyExtent
    +  else
    +    Result := Origin.ExtentXYList;
    +end;
    +
    +function TSortedChartSource.ValuesTotal: Double;
    +begin
    +  if Origin = nil then
    +    Result := 0
    +  else
    +    Result := Origin.ValuesTotal;
    +end;
    +
     { TMWCRandomGenerator }
     
     function TMWCRandomGenerator.Get: LongWord;
    @@ -1452,7 +1675,7 @@
     procedure TCalculatedChartSource.SetOrigin(AValue: TCustomChartSource);
     begin
       if AValue = Self then
    -      AValue := nil;
    +    AValue := nil;
       if FOrigin = AValue then exit;
       if FOrigin <> nil then
         FOrigin.Broadcaster.Unsubscribe(FListener);
    
    phase4_updated.diff (8,733 bytes)

Relationships

related to 0035630 closedwp TAChart: sorting can be a bit faster for free 
related to 0035664 closedwp TAChart: fix for incompatibility, introduced in TListChartSource 
related to 0035666 closedwp TAChart: fix for TCustomSortedChartSource.ItemFind() 
related to 0035681 assignedwp TAChart: sorted data autodetection for TListChartSource 

Activities

Marcin Wiazowski

2019-04-10 16:20

reporter  

patch.diff (1,997 bytes)
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 60918)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -207,6 +207,7 @@
     function GetHasErrorBars(Which: Integer): Boolean;
     function GetItem(AIndex: Integer): PChartDataItem; virtual; abstract;
     procedure InvalidateCaches;
+    function IsSortedByX: Boolean; virtual;
     procedure SetXCount(AValue: Cardinal); virtual; abstract;
     procedure SetYCount(AValue: Cardinal); virtual; abstract;
     property XErrorBarData: TChartErrorBarData index 0 read GetErrorBarData
@@ -1011,7 +1012,7 @@
 
 // ALB -> leftmost item where X >= AXMin, or Count if no such item
 // ALB -> rightmost item where X <= AXMax, or -1 if no such item
-// If the source is sorted, performs binary search. Otherwise, skips NaNs.
+// If the source is sorted by X, performs binary search. Otherwise, skips NaNs.
 procedure TCustomChartSource.FindBounds(
   AXMin, AXMax: Double; out ALB, AUB: Integer);
 
@@ -1041,7 +1042,7 @@
 
 begin
   EnsureOrder(AXMin, AXMax);
-  if IsSorted then begin
+  if IsSortedByX then begin
     ALB := FindLB(AXMin, 0, Count - 1);
     AUB := FindUB(AXMax, 0, Count - 1);
   end
@@ -1267,8 +1268,13 @@
     (AYIndex > -1);
 end;
 
-function TCustomChartSource.IsSorted: Boolean;
+function TCustomChartSource.IsSortedByX: Boolean; inline;
 begin
+  Result := IsSorted; // By default, we assume that IsSorted means IsSortedByX
+end;
+
+function TCustomChartSource.IsSorted: Boolean; inline;
+begin
   Result := false;
 end;
 
@@ -1421,7 +1427,7 @@
     cnt += 1;
   end;
 
-  if not IsSorted and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
+  if not IsSortedByX and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
     SortValuesInRange(AValues, start, cnt - 1);
     if aipUseMinLength in AParams.FIntervals.Options then
       cnt := EnsureMinLength(start, cnt - 1);
patch.diff (1,997 bytes)

wp

2019-04-10 17:02

developer   ~0115396

Last edited: 2019-04-10 17:07

View 3 revisions

What are you trying to achieve? The YCount=0 story again? TAChart needs sorting by X and therefore it implements it and nothing else. There is no need for any other sorting criteria. If you need them implement them at application level or in your own derived classes but not inside the general library.

Marcin Wiazowski

2019-04-10 17:45

reporter   ~0115398

Yhm... this has nothing to do with XCount = 0, although - in fact - I found the issue, described here, when we were discussing in 0035188 about XCount = 0.


> TAChart needs sorting by X and therefore it implements it and nothing else.

I think that we both agree here. In any way, I'm no longer going to sort data in TAChart components by anything other than just X.


> If you need them implement them at application level or in your own derived classes

And this is exactly where the problem is. Unfortunately, currently I cannot inherit from TCustomChartSource to implement my own data source, that uses some custom sorting method. This is because TCustomChartSource assumes, that the only sorting method is by X.

Or more precisely: I can do this, but my TMyChartSource.IsSorted() method must always return False, to avoid confusing TCustomChartSource internals. But IsSorted() is a public method, so this is not too good...

Or, in other words: as far as I understand, "sorted" does not necessarily mean "sorted by X"; although this is true in TAChart's own components, when I inherit from TCustomChartSource to sort by Color, IsSorted returning True means - for me - "sorted by Color".

The change, that I attached, just allows the user to inherit from TCustomChartSource, implement his own sorting algorithm, have IsSorted() returning True when the custom source is sorted (let's say by Color) - but still don't confuse TCustomChartSource internals.

So the real question is: does "sorted" should always mean "sorted by x", even in the user's own code?

If so, there is nothing to do.

If not, the patch, that I attached, would be helpful...

wp

2019-04-10 18:38

developer   ~0115399

IsSorted is used by TCustomChartSource at two places only:

- in "ValuesInRange": used in conjunction with axis labels, probably not relevant here.

- in "FindBounds" where it evaluates the x values because that's the way TAChart works. If you'd redefine IsSorted to mean "sorted by color" this method will not work any more. To avoid this, "FindBounds" should be made virtual so that your derived source can override it. Buth then again, the responsibility for IsSorted has been moved to the derived class.

In derived sources, IsSorted can be overridden to get any meaning required.



> So the real question is: does "sorted" should always mean "sorted by x", even in the user's
> own code?

As far as the infrastructure provided by TCustomChartSource and the currently derived sources is concerned: IsSorted means "sorted by x" because this is what is needed. Of course the user can do in his own code what he wants.

I do not see why TCustomChartSource should be modified (except maybe to make FindBounds virtual) to allow the user to interpret "sorted" in a different way.

Marcin Wiazowski

2019-04-10 19:59

reporter   ~0115401

About FindBounds(): making it virtual, in order to force the derived class to reimplement it, would be a far overkill... Please take a look - that check for "IsSorted" is there only for optimization - so if the source is sorted (by X, of course), we can use binary search.

But we just need a way to distinguish between "sorted" and "sorted by X", if we want to use "sorted" in the meaning other than "sorted by X".



In other words: currently, all the TAChart's code assumes, that "IsSorted" means "is sorted by X". So, if we now replace all "IsSorted" words with "IsSortedByX" words in all the TAChart's source code, the code will still work in the exactly same way - and the new naming convention will be even more precise. But now, we can introduce a more generic "IsSorted" method - which may, by default, just call "IsSortedByX" - but the user may override it to give it any other meaning, that he needs - in particular "sorted by Y" or "sorted by Color". And all the TAChart's code will still work properly, because it will check now for "IsSortedByX" - no longer for "IsSorted", which may have now any meaning other than "sorted by X".



> Of course the user can do in his own code what he wants.

> I do not see why TCustomChartSource should be modified [...] to allow the user to interpret "sorted" in a different way.

Maybe I wasn't clear enough. Let's consider a small example: we have the following data points:

(1, 7)
(2, 2)
(3, 4)
(4, 0)

and we have TOurExampleChartSource, which sorts data points by Y - and TOurExampleChartSource.IsSorted() returns True, when data is sorted by Y. So we sort our data points by Y:

(4, 0)
(2, 2)
(3, 4)
(1, 7)

But TOurExampleChartSource inherits from TCustomChartSource, and TCustomChartSource.FindBounds() is executed by TAChart's machinery, when drawing the chart. So FindBounds() calls our TOurExampleChartSource.IsSorted(), which returns True. But this "True" means, that data is sorted by Y - although FindBounds() works as if it were sorted by X... oops, we have a problem.

After applying the patch, solution is simple:

  TOurExampleChartSource = class(TCustomChartSource)
  protected
    function IsSortedByX: Boolean; override;
  end;

  function TOurExampleChartSource.IsSortedByX: Boolean; inline;
  begin
    Result := false;
  end;

This way, TCustomChartSource.FindBounds() will call TOurExampleChartSource.IsSortedByX(), which will always return False - so FindBounds() will work in a slower, but correct way - while we can call TOurExampleChartSource.IsSorted() and get True, which, this time, means that data is sorted by Y. And there is no need to reimplement the whole FindBounds() method in TOurExampleChartSource.



The patch, that I attached, modifies only TCustomChartSource - but the best thing, that can be done, is to replace all "Sorted"/"IsSorted" calls with "IsSortedByX" calls, in all TAChart's classes. This sounds much worse than it really is - this means one additional change in tamultiseries.pas, one in tacustomseries.pas, and eight in tasources.pas. Then, the user will be able to inherit also from TListChartSource or TUserDefinedChartSource, and implement his own sorting algorithm - with a guarantee, that this will not confuse TAChart's internals. And a full compatibility with all the existing TAChart's and users' code will be retained.

wp

2019-04-10 22:51

developer   ~0115411

Let me express it differently:

Suppose I have a bar series with average temperatures of several cities entered into a ListChartSource in random order. It would certainly result in a nice chart, but it could be improved if the user had the possibility to re-sort the data in various ways: sorted by temperatures (ascending/descending), sorted by city name, unsorted.

Should this feature be in TAChart, i.e. in TCustomChartSource/TListChartSource? I think this is what you are preparing.

In order to have this feature, at first we would need your patch, I agree. But then there must be a way for the user to re-sort the ListSource. The cheapest possibility for TAChart would be to provide an event in which the user could sort the source by himself. Or the next comfortable option would be to implement a basic quick sort routine (in fact, it's already there) and prepare an event for the user to provide his compare function, and/or to hard-code compare-functions for the y values and texts and provide properties for sort item and sort direction.

In essence, it adds code to TAChart for a feature which is not urgently needed, and which could easily be provided with the existing machinery:

I'd add my data to a data structure which is well-prepared for sorting, maybe a TList, AVLTree, or whatever. This way I could re-use existing, well-tested code for sorting. Then, I'd use a TUserDefinedChartsource to provide the link between data storage and the series. That's all. Maybe a bit more code to type, but it would keep TAChart (and the IDE) free from little-used code.

Do this test: Compile an empty Lazarus GUI program - on my Win10 this results in an exe size of 1.989 kB (without debug symbols). Add a TChart, nothing else - the file size double to 3.835 kB.

Certainly, the sorting feature will not add dramatically to this. But piece comes to piece, and I have in mind a lot of features which are either more important or cannot be achieved with the existing infrastructure.

Marcin Wiazowski

2019-04-10 23:19

reporter   ~0115413

Well... No, I'm not going to add any other code to TAChart.



In fact, I'm only proposing two things:

- renaming all the "IsSorted" functions to "IsSortedByX"; this change, of course, will not add any new code to TAChart,

- adding a new "IsSorted" virtual function to TCustomChartSource, with a default 1-line implementation:

  function TCustomChartSource.IsSorted: Boolean;
  begin
    Result := IsSortedByX;
  end;

We can even make it inline, so there will be not even one more byte of code added to TAChart - only one additional pointer in TCustomChartSource's virtual method table will be placed.



I think, that your nice example should be implemented fully in the user's code. In fact, to provide alternative sorting methods for TListChartSource, some event could be added - but it's much simpler just to make the Sort() method virtual. This adds another one pointer, this time to TListChartSource's virtual method table - but nothing more.



Then the user will be able not only to use TUserDefinedChartSource, but also to inherit from TListChartSource - where he can use TList, AVLTree or any other existing, and well-tested, sorting method - as in your example. No such code should be added to the TListChartSource itself, and I don't even want to think about this.

wp

2019-04-10 23:48

developer   ~0115414

I think I still don't understand what is the intention of this report.

So, when you say that my example "should be implemented fully in user's code", I guess that it does not express what you mean.

I am not against adding code to TAChart, of course not, but I am against adding code from which I do not see any benefit.

Please explain what you would do if this "multi-sort" were available. Just being able to do it is not enough.

Marcin Wiazowski

2019-04-12 01:42

reporter   ~0115433

I prepared some demo, but 0035364 must be fixed first.

Marcin Wiazowski

2019-04-12 23:15

reporter  

Test.zip (3,441 bytes)

Marcin Wiazowski

2019-04-12 23:15

reporter  

Test.png (16,567 bytes)
Test.png (16,567 bytes)

Marcin Wiazowski

2019-04-12 23:16

reporter  

patch_ver2.diff (8,462 bytes)
Index: components/tachart/tacustomseries.pas
===================================================================
--- components/tachart/tacustomseries.pas	(revision 60943)
+++ components/tachart/tacustomseries.pas	(working copy)
@@ -1940,15 +1940,15 @@
   {FLoBound and FUpBound fields may be outdated here (if axis' range has been
    changed after the last series' painting). FLoBound and FUpBound will be fully
    updated later, in a PrepareGraphPoints() call. But we need them now. If data
-   source is sorted, obtaining FLoBound and FUpBound is very fast (binary search) -
-   so we call FindExtentInterval() with True as the second parameter. If data
-   source is not sorted, obtaining FLoBound and FUpBound requires enumerating all
-   the data points to see, if they are in the current chart's viewport. But this
-   is exactly what we are going to do in the loop below, so obtaining true FLoBound
-   and FUpBound values makes no sense in this case - so we call FindExtentInterval()
-   with False as the second parameter, thus setting FLoBound to 0 and FUpBound to
-   Count-1}
-  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
+   source is sorted by X, obtaining FLoBound and FUpBound is very fast (binary
+   search) - so we call FindExtentInterval() with True as the second parameter.
+   If data source is not sorted by X, obtaining FLoBound and FUpBound requires
+   enumerating all the data points to see, if they are in the current chart's
+   viewport. But this is exactly what we are going to do in the loop below, so
+   obtaining true FLoBound and FUpBound values makes no sense in this case - so
+   we call FindExtentInterval() with False as the second parameter, thus setting
+   FLoBound to 0 and FUpBound to Count-1}
+  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByX);
 
   with Extent do
     center := AxisToGraphY((a.y + b.y) * 0.5);
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 60943)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -246,6 +246,8 @@
     function HasYErrorBars: Boolean;
     function IsXErrorIndex(AXIndex: Integer): Boolean;
     function IsYErrorIndex(AYIndex: Integer): Boolean;
+    function IsSortedByX: Boolean; virtual; // must return true if -
+      // and only if - source is sorted by X in the ascending order
     function IsSorted: Boolean; virtual;
     procedure ValuesInRange(
       AParams: TValuesInRangeParams; var AValues: TChartValueTextArray); virtual;
@@ -1011,7 +1013,7 @@
 
 // ALB -> leftmost item where X >= AXMin, or Count if no such item
 // ALB -> rightmost item where X <= AXMax, or -1 if no such item
-// If the source is sorted, performs binary search. Otherwise, skips NaNs.
+// If the source is sorted by X, performs binary search. Otherwise, skips NaNs.
 procedure TCustomChartSource.FindBounds(
   AXMin, AXMax: Double; out ALB, AUB: Integer);
 
@@ -1041,7 +1043,7 @@
 
 begin
   EnsureOrder(AXMin, AXMax);
-  if IsSorted then begin
+  if IsSortedByX then begin
     ALB := FindLB(AXMin, 0, Count - 1);
     AUB := FindUB(AXMax, 0, Count - 1);
   end
@@ -1267,8 +1269,13 @@
     (AYIndex > -1);
 end;
 
-function TCustomChartSource.IsSorted: Boolean;
+function TCustomChartSource.IsSortedByX: Boolean; inline;
 begin
+  Result := IsSorted; // By default, we assume that IsSorted means IsSortedByX
+end;
+
+function TCustomChartSource.IsSorted: Boolean; inline;
+begin
   Result := false;
 end;
 
@@ -1421,7 +1428,7 @@
     cnt += 1;
   end;
 
-  if not IsSorted and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
+  if not IsSortedByX and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
     SortValuesInRange(AValues, start, cnt - 1);
     if aipUseMinLength in AParams.FIntervals.Options then
       cnt := EnsureMinLength(start, cnt - 1);
Index: components/tachart/tamultiseries.pas
===================================================================
--- components/tachart/tamultiseries.pas	(revision 60943)
+++ components/tachart/tamultiseries.pas	(working copy)
@@ -879,7 +879,7 @@
   if Count = 0 then exit;
   if not RequestValidChartScaling then exit;
 
-  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
+  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByX);
   with Extent do
     center := AxisToGraphY((a.y + b.y) * 0.5);
   UpdateLabelDirectionReferenceLevel(0, 0, center);
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 60943)
+++ components/tachart/tasources.pas	(working copy)
@@ -23,9 +23,7 @@
 
   TListChartSource = class(TCustomChartSource)
   private
-    FData: TFPList;
     FDataPoints: TStrings;
-    FSorted: Boolean;
     FXCountMin: Cardinal;
     FYCountMin: Cardinal;
     procedure AddAt(
@@ -36,6 +34,8 @@
     procedure SetSorted(AValue: Boolean);
     procedure UpdateCachesAfterAdd(AX, AY: Double);
   protected
+    FData: TFPList;
+    FSorted: Boolean;
     function GetCount: Integer; override;
     function GetItem(AIndex: Integer): PChartDataItem; override;
     procedure Loaded; override;
@@ -70,7 +70,7 @@
     procedure SetYList(AIndex: Integer; const AYList: array of Double);
     procedure SetYValue(AIndex: Integer; AValue: Double);
 
-    procedure Sort;
+    procedure Sort; virtual;
   published
     property DataPoints: TStrings read FDataPoints write SetDataPoints;
     property Sorted: Boolean read FSorted write SetSorted default false;
@@ -229,6 +229,7 @@
     constructor Create(AOwner: TComponent); override;
     destructor Destroy; override;
 
+    function IsSortedByX: Boolean; override;
     function IsSorted: Boolean; override;
   published
     property AccumulationDirection: TChartAccumulationDirection
@@ -490,7 +491,7 @@
   AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
 begin
   Result := FData.Count;
-  if Sorted then
+  if IsSortedByX then
     // Keep data points ordered by X coordinate.
     // Note that this leads to O(N^2) time except
     // for the case of adding already ordered points.
@@ -603,7 +604,18 @@
         SetXList(FData.Count - 1, XList);
         SetYList(FData.Count - 1, YList);
       end;
-    if Sorted and not ASource.IsSorted then Sort;
+    if IsSorted then
+      if IsSortedByX and ASource.IsSortedByX then
+        // both Self and ASource are sorted by X in the ascending order,
+        // so there is nothing more to do
+      else
+      if (ClassInfo = ASource.ClassInfo) and ASource.IsSorted then
+        // both Self and ASource are sorted by some custom algorithm -
+        // but both of them are objects of the exactly same class,
+        // so both use the exactly same sorting algorithm - so there
+        // is nothing more to do
+      else
+        Sort;
   finally
     EndUpdate;
   end;
@@ -670,7 +682,7 @@
   Result := PChartDataItem(FData.Items[AIndex]);
 end;
 
-function TListChartSource.IsSorted: Boolean;
+function TListChartSource.IsSorted: Boolean; inline;
 begin
   Result := Sorted;
 end;
@@ -766,7 +778,7 @@
   end;
 
 begin
-  if Sorted then
+  if IsSortedByX then
     if IsNan(AValue) then
       raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
   oldX := Item[AIndex]^.X;
@@ -774,7 +786,7 @@
   if IsEquivalent(oldX, AValue) then exit;
   Item[AIndex]^.X := AValue;
   UpdateExtent;
-  if Sorted then begin
+  if IsSortedByX then begin
     if AValue > oldX then
       while (Result < Count - 1) and (Item[Result + 1]^.X < AValue) do
         Inc(Result)
@@ -1035,7 +1047,7 @@
   Result := @FCurItem;
 end;
 
-function TRandomChartSource.IsSorted: Boolean;
+function TRandomChartSource.IsSorted: Boolean; inline;
 begin
   Result := not RandomX;
 end;
@@ -1142,7 +1154,7 @@
   Result := @FItem;
 end;
 
-function TUserDefinedChartSource.IsSorted: Boolean;
+function TUserDefinedChartSource.IsSorted: Boolean; inline;
 begin
   Result := Sorted;
 end;
@@ -1435,6 +1447,14 @@
   Result := AccumulationMethod in [camDerivative, camSmoothDerivative];
 end;
 
+function TCalculatedChartSource.IsSortedByX: Boolean;
+begin
+  if Origin <> nil then
+    Result := Origin.IsSortedByX
+  else
+    Result := false;
+end;
+
 function TCalculatedChartSource.IsSorted: Boolean;
 begin
   if Origin <> nil then
patch_ver2.diff (8,462 bytes)

Marcin Wiazowski

2019-04-12 23:19

reporter   ~0115455

I attached a sample application, with a realistic usage of some custom sorting algorithm. The application uses bubble series to show number of visitors of some fictitious exhibition.


As a data source, a TMyChartSource class has been implemented, which inherits from TCustomChartSource (it would be much easier to inherit from TListChartSource, but it currently has its FData variable declared as private, so deriving from it is useless in this case...).


After launching the application, we can see bubbles on the chart. The chart doesn't initially look very well - some of the bubbles are hidden below the others, larger. But there is a solution: we can sort bubbles by their size, in the descending order - so larger bubbles will be drawn before the smaller ones. We can do this by pressing "Sort data by number of visitors!" button - it calls TMyChartSource.MakeSorted() method, which sorts data by bubble size, i.e. by TChartDataItem.YList[0]. Now the chart looks perfectly.

We can choose between showing all the available data, only 1960s, or only 1970s (in a real application, this would be some limitation on size of horizontal extent, plus scroll left / scroll right buttons). So we choose "Show 1960s". Unfortunately, one of the labels (marks) - more precisely the "City B" mark - is not shown. So maybe choose "Show 1970s"? Now it's even worse - we get a "List index out of bounds" exception...


The attached patch_ver2.diff solves these problems. The only purpose of this patch is to allow using custom sorting algorithms in user classes, that are derived from TAChart standard classes. The patch:
- implements an "IsSortedByX" method in TCustomChartSource,
- changes "Sorted"/"IsSorted" calls to "IsSortedByX" calls if needed,
- in TListChartSource, makes "Sort" method virtual, and moves two variables from private to protected section - to make deriving from TListChartSource practically possible,
- also makes some trivial "IsSorted" implementations inlined.


After:
- applying patch_ver2.diff
AND
- uncommenting "override" in TMyChartSource.IsSortedByX() declaration,

everything works properly again. The patch isn't in fact very complicated, but it's all that is needed to assume the problem to be fully solved.

wp

2019-04-13 01:07

developer   ~0115459

Last edited: 2019-04-13 10:28

View 3 revisions

I am attaching my variant of your demo which is based on ready-made classes: TObjectList for storage and sorting and TUserDefinedChartSource for interfacing to the series. I think the code is very similar, it's even a bit shorter (146 lines vs 184). (The main thing which I don't like - making an inherited virtual method non-virtual - is probably related to this particular demo.)

Let me repeat your words from above: "I think, that your nice example should be implemented fully in the user's code."

On the other hand: Sorting the bubble series by radius is a nice idea to overcome the "covered bubble issue". But should this happen in the source? Some sources are not sortable at all or require external code, and ideally there should be a TBubbleSeries property "ShowCoveredBubbles" whatever the source.

And: I looked into Delphi's TeeChart. It handles the sources differently in detail, but essentially it can sort by every numerical element of the source, and in ascending and descending direction. (Although I did not get it to work...).

wp

2019-04-13 01:09

developer  

Test-wp.zip (3,227 bytes)

Marcin Wiazowski

2019-04-13 16:36

reporter   ~0115474

Your version is very elegant. My would be also more concise, if I could inherit from TListChartSource (but I couldn't due to FData declared as private there).



> Let me repeat your words from above: "I think, that your nice example should be implemented fully in the user's code."

Well, when I was saying that, I was thinking of "I think, that your nice example should be implemented fully in the user's code, by deriving from standard TAChart classes" and not "by inheriting from TObject and implementing everything from scratch" :)



> On the other hand: Sorting the bubble series by radius is a nice idea to overcome the "covered bubble issue". But should this happen in the source? [...] and ideally there should be a TBubbleSeries property "ShowCoveredBubbles" whatever the source.

But adding "ShowCoveredBubbles" leads to one of two things:
- sorting an external or internal source by YList[0],
- sorting data internally, bu using some custom code, each time, when series is being drawn (although some caching could be implemented).

The more universal method is to sort the source.





Now the most important thing. There is one, small difference between our demos - which is essential in this case: your demo doesn't handle "Sorted" property. After changing to:

  procedure TForm1.MakeSorted;
  begin
    FData.Sort(@CompareByNumberOfVisitors);
    UserDefinedChartSource1.Sorted := true;
    UserDefinedChartSource1.Reset;
  end;

exactly same problems occur as in my demo: source has been sorted, so it set "Sorted" to True - but this confuses TAChart internals.



This way, again, we meet the root problem: does "Sorted"/"IsSorted" always means "sorted by x"?

If so, there is only one thing to do: it should be ensured, that the documentation clearly informs, that the user is allowed to make "Sorted"/"IsSorted" returning True if - and only if! - data is sorted by X, in the ascending order.

Otherwise, we need a way to allow TAChart internals to distinguish between "sorting by X in ascending order" and other sorting methods. Which, in particular, can be achieved by using the patch that I attached.



Now, when you mentioned about Delphi's TeeChart, I can also see another (more universal) solution: TCustomChartSource could implement properties like:
- SortBy: (sbX, sbY, sbColor, sbText, sbCustom)
- SortDir: (sdAscending, sdDescending)

with default handling like raising an exception, when trying to change SortBy <> sbX or change SortDir <> sdAscending (which could be overridden in derived classes).



So, basically, the decision to made (by you, in this case...) is: should TAChart stick to "Sorted"/"IsSorted" always meaning "sorted by x", or not?



> Although I did not get it to work

Maybe, as in TAChart, only sorting by X in the ascending order is implemented - but other options are there, to allow users to implement their own solutions in derived classes?

wp

2019-04-13 18:37

developer   ~0115475

The more I think about it the more I get the feeling that sorting a source might be a welcome feature. The convincing argument was that it would allow to display the BubbleSeries so that all bubbles are visible (although it has the problem, that it works only for list sources - a "real" solution required for a "ShowCoveredBubbles" property would be a sorted integer list which is updated along with the extent, but see below...).

And maybe there are more examples of this kind.

Unlike your current solution I'd prefer, however, complete built-in sorting support similar to the Delphi solution.

There should be properties (like you wrote)
- SortBy (sbX, sbY, sbText, sbColor, sbCustom)
- SortDir (sdAscending, sdDescending)
- SortIndex (0 = regular x/y value, 1 = XList[0] or YList[0], 2 = XList[1] or YList[1], ...)

The abstract routines should be included in TCustomChartSource, but the properties should remain protected. TListChartSource should override the abstract methods and publish the new properties.

The other sources cannot be sorted under control of TAChart, not even TRandomChartSource where sorting would require excessive buffering which would make it something like a "RandomListChartSource".

In order to sort these sources I'd propose to add a new TSortedChartSource which links to another chart source (similar to the TCalculatedChartSource) and stores the sorted data indexes in a sorted integer list. So, there would be a chain of ChartSources
- the "primary" chart source which directly provides the data (db, user-defined, random)
- the sorting source which has a property "Origin" linked to the primary source. it looks up the "true" source index of the data point queried by the series and reads the TChartDataItem from the primary source.
Of course, the TSortedChartSource should publish the new SortBy, SortDir and SortIndex properties. The XCount and YCount values should be passed through from the primary source.

Marcin Wiazowski

2019-04-13 19:45

reporter   ~0115476

I find TSortedChartSource to be a very nice idea!

This way, any source could be tied to bubble series - and an internal, intermediate TSortedChartSource instance could be used in bubble series for conversion.



Although, even in this case, we still need to distinguish between "sorted by X in the ascending order" and "sorted by any other algorithm" in TAChart's code - just to avoid problems like in my demo (and also yours, after setting Sorted to True).

After implementing these SortBy, SortDir and SortIndex properties in TCustomChartSource, there will be a possibility to check this:

  if Sorted and (SortBy = sbX) and (SortIndex = 0) and (SortDir = sdAscending)

but, to make life easier, I think that there still would be useful to have a method like "IsSortedByX" (or maybe "IsSortedByXAsc"), to avoid such complicated comparisons like above... There is no need to have other variants, like "IsSortedByXDesc", "IsSortedByYAsc", etc. - this is because sorting by X in the ascending order is the only special case, that is used in TAChart's internals.

wp

2019-04-13 20:01

developer   ~0115477

Yes, no problem with it any more. Are you planning to implement it, or should I put it on my own list?

Marcin Wiazowski

2019-04-13 20:24

reporter   ~0115478

I have a great experience with low-level internals - like exceptions - so maybe I will focus on EMultipleExceptions in 0035317. From the other side, you definitely have greater experience with TAChart, so maybe you could implement TSortedChartSource?

However, I will attach patch_ver3.diff here, which will implement these SortBy / SortDir / SortIndex properties at the lowest level in TCustomChartSource. And maybe change "IsSortedByX" to "IsSortedByXAsc".

wp

2019-04-13 23:43

developer   ~0115480

ok

Marcin Wiazowski

2019-04-14 03:53

reporter  

patch_ver3.diff (13,836 bytes)
Index: components/tachart/tachartstrconsts.pas
===================================================================
--- components/tachart/tachartstrconsts.pas	(revision 60963)
+++ components/tachart/tachartstrconsts.pas	(working copy)
@@ -78,12 +78,12 @@
   rsSourceNotEditable = 'Editable chart source required';
   rsSourceCountError = '%0:s requires a chart source with at least %1:d %2:s value(s) per data point.';
   rsSourceCountError2 = 'This %0:s instance must have at least %1:d %2:s value(s) per data point.';
+  rsSourceSortError = 'Selected sorting parameters are not supported by %s.';
   rsListSourceStringFormatError = 'The data value count in the %0:s.DataPoints '+
     'string "%1:s" differs from what is expected from XCount and YCount.';
   rsListSourceNumericError = 'The %0:s.DataPoints string "%1:s" is not a valid number.';
   rsListSourceColorError = 'The %0:s.DataPoints string "%1:s" is not an integer.';
 
-
   // Transformations
   tasAxisTransformsEditorTitle = 'Edit axis transformations';
   rsAutoScale = 'Auto scale';
Index: components/tachart/tacustomseries.pas
===================================================================
--- components/tachart/tacustomseries.pas	(revision 60963)
+++ components/tachart/tacustomseries.pas	(working copy)
@@ -1940,15 +1940,15 @@
   {FLoBound and FUpBound fields may be outdated here (if axis' range has been
    changed after the last series' painting). FLoBound and FUpBound will be fully
    updated later, in a PrepareGraphPoints() call. But we need them now. If data
-   source is sorted, obtaining FLoBound and FUpBound is very fast (binary search) -
-   so we call FindExtentInterval() with True as the second parameter. If data
-   source is not sorted, obtaining FLoBound and FUpBound requires enumerating all
-   the data points to see, if they are in the current chart's viewport. But this
-   is exactly what we are going to do in the loop below, so obtaining true FLoBound
-   and FUpBound values makes no sense in this case - so we call FindExtentInterval()
-   with False as the second parameter, thus setting FLoBound to 0 and FUpBound to
-   Count-1}
-  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
+   source is sorted by X in the ascending order, obtaining FLoBound and FUpBound
+   is very fast (binary search) - so we call FindExtentInterval() with True as
+   the second parameter. Otherwise, obtaining FLoBound and FUpBound requires
+   enumerating all the data points to see, if they are in the current chart's
+   viewport. But this is exactly what we are going to do in the loop below, so
+   obtaining true FLoBound and FUpBound values makes no sense in this case - so
+   we call FindExtentInterval() with False as the second parameter, thus setting
+   FLoBound to 0 and FUpBound to Count-1}
+  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByXAsc);
 
   with Extent do
     center := AxisToGraphY((a.y + b.y) * 0.5);
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 60963)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -71,9 +71,10 @@
 type
   EBufferError = class(EChartError);
   EEditableSourceRequired = class(EChartError);
+  EListSourceStringError = class(EChartError);
+  ESortError = class(EChartError);
   EXCountError = class(EChartError);
   EYCountError = class(EChartError);
-  EListSourceStringError = class(EChartError);
 
   TChartValueText = record
     FText: String;
@@ -175,9 +176,11 @@
     property IndexPlus: Integer index 0 read GetIndex write SetIndex default -1;
     property ValueMinus: Double index 1 read GetValue write SetValue stored IsErrorBarValueStored;
     property ValuePlus: Double index 0 read GetValue write SetValue stored IsErrorBarValueStored;
-
   end;
 
+  TChartSortBy = (sbX, sbY, sbColor, sbText, sbCustom);
+  TChartSortDir = (sdAscending, sdDescending);
+
   TCustomChartSource = class(TBasicChartSource)
   strict private
     FErrorBarData: array[0..1] of TChartErrorBarData;
@@ -197,6 +200,9 @@
     FYListExtentIsValid: Boolean;
     FValuesTotal: Double;
     FValuesTotalIsValid: Boolean;
+    FSortBy: TChartSortBy;
+    FSortDir: TChartSortDir;
+    FSortIndex: Cardinal;
     FXCount: Cardinal;
     FYCount: Cardinal;
     function CalcExtentXYList(UseXList: Boolean): TDoubleRect;
@@ -207,6 +213,9 @@
     function GetHasErrorBars(Which: Integer): Boolean;
     function GetItem(AIndex: Integer): PChartDataItem; virtual; abstract;
     procedure InvalidateCaches;
+    procedure SetSortBy(AValue: TChartSortBy); virtual;
+    procedure SetSortDir(AValue: TChartSortDir); virtual;
+    procedure SetSortIndex(AValue: Cardinal); virtual;
     procedure SetXCount(AValue: Cardinal); virtual; abstract;
     procedure SetYCount(AValue: Cardinal); virtual; abstract;
     property XErrorBarData: TChartErrorBarData index 0 read GetErrorBarData
@@ -247,6 +256,7 @@
     function IsXErrorIndex(AXIndex: Integer): Boolean;
     function IsYErrorIndex(AYIndex: Integer): Boolean;
     function IsSorted: Boolean; virtual;
+    function IsSortedByXAsc: Boolean;
     procedure ValuesInRange(
       AParams: TValuesInRangeParams; var AValues: TChartValueTextArray); virtual;
     function ValuesTotal: Double; virtual;
@@ -255,6 +265,9 @@
 
     property Count: Integer read GetCount;
     property Item[AIndex: Integer]: PChartDataItem read GetItem; default;
+    property SortBy: TChartSortBy read FSortBy write SetSortBy default sbX;
+    property SortDir: TChartSortDir read FSortDir write SetSortDir default sdAscending;
+    property SortIndex: Cardinal read FSortIndex write SetSortIndex default 0;
     property XCount: Cardinal read FXCount write SetXCount default 1;
     property YCount: Cardinal read FYCount write SetYCount default 1;
   end;
@@ -288,7 +301,7 @@
 implementation
 
 uses
-  Math, StrUtils, SysUtils, TAMath;
+  Math, StrUtils, SysUtils, TAMath, TAChartStrConsts;
 
 function CompareChartValueTextPtr(AItem1, AItem2: Pointer): Integer;
 begin
@@ -895,8 +908,6 @@
   end;
 end;
 
-
-
 class procedure TCustomChartSource.CheckFormat(const AFormat: String);
 begin
   Format(AFormat, [0.0, 0.0, '', 0.0, 0.0]);
@@ -907,6 +918,9 @@
   i: Integer;
 begin
   inherited Create(AOwner);
+  FSortBy := sbX;
+  FSortDir := sdAscending;
+  FSortIndex := 0;
   FXCount := 1;
   FYCount := 1;
   for i:=Low(FErrorBarData) to High(FErrorBarData) do begin
@@ -1011,7 +1025,8 @@
 
 // ALB -> leftmost item where X >= AXMin, or Count if no such item
 // ALB -> rightmost item where X <= AXMax, or -1 if no such item
-// If the source is sorted, performs binary search. Otherwise, skips NaNs.
+// If the source is sorted by X in the ascending order, performs
+// binary search. Otherwise, skips NaNs.
 procedure TCustomChartSource.FindBounds(
   AXMin, AXMax: Double; out ALB, AUB: Integer);
 
@@ -1041,7 +1056,7 @@
 
 begin
   EnsureOrder(AXMin, AXMax);
-  if IsSorted then begin
+  if IsSortedByXAsc then begin
     ALB := FindLB(AXMin, 0, Count - 1);
     AUB := FindUB(AXMax, 0, Count - 1);
   end
@@ -1267,11 +1282,34 @@
     (AYIndex > -1);
 end;
 
-function TCustomChartSource.IsSorted: Boolean;
+function TCustomChartSource.IsSorted: Boolean; inline;
 begin
   Result := false;
 end;
 
+function TCustomChartSource.IsSortedByXAsc: Boolean;
+begin
+  Result := IsSorted and (FSortBy = sbX) and (FSortDir = sdAscending) and (FSortIndex = 0);
+end;
+
+procedure TCustomChartSource.SetSortBy(AValue: TChartSortBy);
+begin
+  if FSortBy <> AValue then
+    raise ESortError.CreateFmt(rsSourceSortError, [ClassName]);
+end;
+
+procedure TCustomChartSource.SetSortDir(AValue: TChartSortDir);
+begin
+  if FSortDir <> AValue then
+    raise ESortError.CreateFmt(rsSourceSortError, [ClassName]);
+end;
+
+procedure TCustomChartSource.SetSortIndex(AValue: Cardinal);
+begin
+  if FSortIndex <> AValue then
+    raise ESortError.CreateFmt(rsSourceSortError, [ClassName]);
+end;
+
 procedure TCustomChartSource.SetErrorBarData(AIndex: Integer;
   AValue: TChartErrorBarData);
 begin
@@ -1421,7 +1459,7 @@
     cnt += 1;
   end;
 
-  if not IsSorted and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
+  if not IsSortedByXAsc and not IsValueTextsSorted(AValues, start, cnt - 1) then begin
     SortValuesInRange(AValues, start, cnt - 1);
     if aipUseMinLength in AParams.FIntervals.Options then
       cnt := EnsureMinLength(start, cnt - 1);
Index: components/tachart/tamultiseries.pas
===================================================================
--- components/tachart/tamultiseries.pas	(revision 60963)
+++ components/tachart/tamultiseries.pas	(working copy)
@@ -879,7 +879,7 @@
   if Count = 0 then exit;
   if not RequestValidChartScaling then exit;
 
-  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSorted);
+  FindExtentInterval(ParentChart.CurrentExtent, Source.IsSortedByXAsc);
   with Extent do
     center := AxisToGraphY((a.y + b.y) * 0.5);
   UpdateLabelDirectionReferenceLevel(0, 0, center);
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 60963)
+++ components/tachart/tasources.pas	(working copy)
@@ -23,9 +23,7 @@
 
   TListChartSource = class(TCustomChartSource)
   private
-    FData: TFPList;
     FDataPoints: TStrings;
-    FSorted: Boolean;
     FXCountMin: Cardinal;
     FYCountMin: Cardinal;
     procedure AddAt(
@@ -36,6 +34,8 @@
     procedure SetSorted(AValue: Boolean);
     procedure UpdateCachesAfterAdd(AX, AY: Double);
   protected
+    FData: TFPList;
+    FSorted: Boolean;
     function GetCount: Integer; override;
     function GetItem(AIndex: Integer): PChartDataItem; override;
     procedure Loaded; override;
@@ -70,7 +70,7 @@
     procedure SetYList(AIndex: Integer; const AYList: array of Double);
     procedure SetYValue(AIndex: Integer; AValue: Double);
 
-    procedure Sort;
+    procedure Sort; virtual;
   published
     property DataPoints: TStrings read FDataPoints write SetDataPoints;
     property Sorted: Boolean read FSorted write SetSorted default false;
@@ -203,6 +203,7 @@
     FOriginYCount: Cardinal;
     FPercentage: Boolean;
     FReorderYList: String;
+    FSorted: Boolean;
     FYOrder: array of Integer;
 
     procedure CalcAccumulation(AIndex: Integer);
@@ -490,7 +491,7 @@
   AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
 begin
   Result := FData.Count;
-  if Sorted then
+  if IsSortedByXAsc then
     // Keep data points ordered by X coordinate.
     // Note that this leads to O(N^2) time except
     // for the case of adding already ordered points.
@@ -603,7 +604,22 @@
         SetXList(FData.Count - 1, XList);
         SetYList(FData.Count - 1, YList);
       end;
-    if Sorted and not ASource.IsSorted then Sort;
+
+    if IsSorted then begin
+      if
+        ASource.IsSorted and (SortBy = ASource.SortBy) and
+        (SortDir = ASource.SortDir) and (SortIndex = ASource.SortIndex) and
+        (
+          (SortBy <> sbCustom) or // data is already sorted as needed -
+            // so there is nothing more to do
+          (ClassInfo = ASource.ClassInfo) // both Self and ASource are
+            // sorted by some custom algorithm - but both of them are
+            // objects of the exactly same class, so they use the
+            // exactly same algorithm - so there is nothing more to do
+        ) then
+          exit;
+      Sort;
+    end;
   finally
     EndUpdate;
   end;
@@ -670,9 +686,9 @@
   Result := PChartDataItem(FData.Items[AIndex]);
 end;
 
-function TListChartSource.IsSorted: Boolean;
+function TListChartSource.IsSorted: Boolean; inline;
 begin
-  Result := Sorted;
+  Result := FSorted;
 end;
 
 function TListChartSource.NewItem: PChartDataItem;
@@ -766,7 +782,7 @@
   end;
 
 begin
-  if Sorted then
+  if IsSortedByXAsc then
     if IsNan(AValue) then
       raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
   oldX := Item[AIndex]^.X;
@@ -774,7 +790,7 @@
   if IsEquivalent(oldX, AValue) then exit;
   Item[AIndex]^.X := AValue;
   UpdateExtent;
-  if Sorted then begin
+  if IsSortedByXAsc then begin
     if AValue > oldX then
       while (Result < Count - 1) and (Item[Result + 1]^.X < AValue) do
         Inc(Result)
@@ -1035,7 +1051,7 @@
   Result := @FCurItem;
 end;
 
-function TRandomChartSource.IsSorted: Boolean;
+function TRandomChartSource.IsSorted: Boolean; inline;
 begin
   Result := not RandomX;
 end;
@@ -1142,9 +1158,9 @@
   Result := @FItem;
 end;
 
-function TUserDefinedChartSource.IsSorted: Boolean;
+function TUserDefinedChartSource.IsSorted: Boolean; inline;
 begin
-  Result := Sorted;
+  Result := FSorted;
 end;
 
 procedure TUserDefinedChartSource.Reset;
@@ -1350,6 +1366,22 @@
 
 procedure TCalculatedChartSource.Changed(ASender: TObject);
 begin
+  if FOrigin <> nil then begin
+    FSortBy := Origin.SortBy;
+    FSortDir := Origin.SortDir;
+    FSortIndex := Origin.SortIndex;
+    // We recalculate Y values, so we can't guarantee, that transformed
+    // data is still sorted by Y or by Origin's custom algorithm
+    FSorted := (FSortBy in [sbX, sbColor, sbText]) and Origin.IsSorted;
+    FXCount := Origin.XCount;
+  end else begin
+    FSortBy := sbX;
+    FSortDir := sdAscending;
+    FSortIndex := 0;
+    FSorted := false;
+    FXCount := 0;
+  end;
+
   if
     (FOrigin <> nil) and (ASender = FOrigin) and
     (FOrigin.YCount <> FOriginYCount)
@@ -1437,10 +1469,7 @@
 
 function TCalculatedChartSource.IsSorted: Boolean;
 begin
-  if Origin <> nil then
-    Result := Origin.IsSorted
-  else
-    Result := false;
+  Result := FSorted;
 end;
 
 procedure TCalculatedChartSource.RangeAround(
patch_ver3.diff (13,836 bytes)

Marcin Wiazowski

2019-04-14 03:58

reporter   ~0115487

Patch_ver3.diff attached. Some notes:



* Some IsSorted() methods use now FSorted field instead of Sorted property, to make it clearly visible, that the result depends only on the FSorted field, and no other code is executed (so there is no complicated dependency between the IsSorted() method and FSorted field).



* The attached "Test" application, to work properly, no longer needs TMyChartSource.IsSortedByX() method - it can be removed. Instead, the following lines should be added to TMyChartSource.Create():

  FSortBy := sbY;
  FSortDir := sdDescending;
  FSortIndex := 1;



* To adhere to the updated code, some change was made in TCalculatedChartSource: since TCalculatedChartSource.Changed() is called for every Origin's change, it became a central (and the only) place of synchronization with Origin.



* Some tips for implementing TSortedChartSource:


A) Similarly to TCalculatedChartSource, TSortedChartSource should have its Changed() method - it should include the following code:

  if FOrigin <> nil then begin
    FXCount := Origin.XCount;
    FYCount := Origin.YCount;
  end else begin
    FXCount := 0;
    FYCount := 0;
  end;

Note: TCalculatedChartSource doesn't update FYCount here, because is performs this below, in an UpdateYOrder() call.


B) TCalculatedChartSource must make the following overrides:

  procedure TCalculatedChartSource.SetSortBy(AValue: TChartSortBy);
  begin
    if FSortBy = AValue then exit;
    FSortBy := AValue;
    if Sorted then Sort; // Sort() must call Notify() !
  end;

  procedure TCalculatedChartSource.SetSortDir(AValue: TChartSortDir);
  begin
    if FSortDir = AValue then exit;
    FSortDir := AValue;
    if Sorted then Sort; // Sort() must call Notify() !
  end;

  procedure TCalculatedChartSource.SetSortIndex(AValue: Cardinal);
  begin
    if FSortIndex = AValue then exit;
    FSortIndex := AValue;
    if Sorted then Sort; // Sort() must call Notify() !
  end;

  function TCalculatedChartSource.IsSorted: Boolean; inline;
  begin
    Result := FSorted;
  end;

and declare:

  FSorted: Boolean;
  property Sorted: Boolean read FSorted write SetSorted default false; // maybe default should be true?

where:

  procedure TCalculatedChartSource.SetSorted(AValue: Boolean);
  begin
    if FSorted = AValue then exit;
    FSorted := AValue;
    if Sorted then Sort; // Sort() must call Notify() !
  end;

Marcin Wiazowski

2019-04-14 04:12

reporter   ~0115488

Some additional note: you advised to make SortBy, SortDir and SortIndex properties protected. But it turned out that they must be public, to allow some code to make comparisons between sorting methods in self and in some other source. So I made these properties public (so they can be read), but their setters are protected, and they only raise ESortError exceptions in their default implementations (and there is currently no other implementation, although it will appear in TCalculatedChartSource).

wp

2019-04-14 22:54

developer  

BubbleSortTest.zip (3,572 bytes)

wp

2019-04-14 23:07

developer   ~0115509

Last edited: 2019-04-14 23:09

View 3 revisions

Applied your patch with modifications:

- Kept SortDir, SortBy, SortIndex properties protected. I prefer "hacking" type-casting to avoid giving the impression that TCustomChartSource in general can be sorted (by having them as public properties).

- Major refactoring of TListChartSource. It has ancestor TCustomSortedChartSource now which handles sorting for TListChartSource and the future TSortedChartSource. TListChartSource now can be sorted by X, Y, XList, YList, Color, Text and in a user-defined way specified by the event OnCompare. SortDirection and SortIndex as discussed above. I removed the "virtual" from Sort again because this would allow the user to by-pass all built-in compare functions. Instead, there is a virtual and protected method "ExecSort" which can be overridden by the user if he absolutely wants a different sorting algorithm - default is quicksort. I had to copy some code from TFPList's implementation to here because TFPList does not work with a compare *method*, only with a standard function.

I uploaded a modified version of your bubble sort demo which shows some variations. The first impression was disappointing because sorting the ListChartSource does NOT reorder the data points - stupid thinking: the x values are retained, of course! The only effect sorting has is the order of painting; therefore, the bubble series is a good example. Another example would be 3D data plotted from the back to the front - but we don't have it yet.

I think I should write a tutorial about sorting because users will expect sorting to rearrange the x values - the tutorial must show how this can be achieved.

Marcin Wiazowski

2019-04-15 04:14

reporter  

patch_new_update.diff (4,705 bytes)
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 60976)
+++ components/tachart/tasources.pas	(working copy)
@@ -27,6 +27,7 @@
   private
     FOnCompare: TChartSortCompare;
     procedure SetSorted(AValue: Boolean);
+    procedure SetOnCompare(AValue: TChartSortCompare);
   protected
     FData: TFPList;
     FSorted: Boolean;
@@ -36,7 +37,7 @@
     procedure SetSortDir(AValue: TChartSortDir); override;
     procedure SetSortIndex(AValue: Cardinal); override;
     property Sorted: Boolean read FSorted write SetSorted default false;
-    property OnCompare: TChartSortCompare read FOnCompare write FOnCompare;
+    property OnCompare: TChartSortCompare read FOnCompare write SetOnCompare;
   public
     function IsSorted: Boolean; override;
     procedure Sort;
@@ -528,23 +529,23 @@
 
   procedure QuickSort(L, R: Longint);
   var
-    I, J : Longint;
-    P, Q : Pointer;
+    I, J: Longint;
+    P, Q: Pointer;
   begin
    repeat
      I := L;
      J := R;
-     P := FData[(L + R) div 2];
+     P := FData.List^[(L + R) div 2];
      repeat
-       while ACompare(P, FData[i]) > 0 do
+       while ACompare(P, FData.List^[I]) > 0 do
          I := I + 1;
-       while ACompare(P, FData[J]) < 0 do
+       while ACompare(P, FData.List^[J]) < 0 do
          J := J - 1;
        If I <= J then
        begin
-         Q := FData[I];
-         FData[I] := FData[J];
-         FData[J] := Q;
+         Q := FData.List^[I];
+         FData.List^[I] := FData.List^[J];
+         FData.List^[J] := Q;
          I := I + 1;
          J := J - 1;
        end;
@@ -578,7 +579,7 @@
 begin
   if FSortBy = AValue then exit;
   FSortBy := AValue;
-  Sort;
+  if Sorted then Sort;
 end;
 
 procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
@@ -585,7 +586,7 @@
 begin
   if FSorted = AValue then exit;
   FSorted := AValue;
-  Sort;
+  if Sorted then Sort;
 end;
 
 procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
@@ -592,7 +593,7 @@
 begin
   if FSortDir = AValue then exit;
   FSortDir := AValue;
-  Sort;
+  if Sorted then Sort;
 end;
 
 procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
@@ -599,15 +600,27 @@
 begin
   if FSortIndex = AValue then exit;
   FSortIndex := AValue;
-  Sort;
+  if Sorted then Sort;
 end;
 
+procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
+begin
+  if FOnCompare = AValue then exit;
+  FOnCompare := AValue;
+  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
+end;
+
 procedure TCustomSortedChartSource.Sort;
 begin
+  if csLoading in ComponentState then exit;
   if (FSortBy = sbCustom) then begin
-    if Assigned(FOnCompare) then ExecSort(FOnCompare);
-  end else
+    if not Assigned(FOnCompare) then exit;
+    ExecSort(FOnCompare);
+  end else begin
+    if (FSortBy = sbX) and (FSortIndex > 0) and (FSortIndex >= FXCount) then exit;
+    if (FSortBy = sbY) and (FSortIndex > 0) and (FSortIndex >= FYCount) then exit;
     ExecSort(@DefaultCompare);
+  end;
   Notify;
 end;
 
@@ -767,17 +780,27 @@
 end;
 
 function TListChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
+const
+  sInvalidCall = 'Invalid call to %s.DefaultCompare';
 var
   item1: PChartDataItem absolute AItem1;
   item2: PChartDataItem absolute AItem2;
 begin
- case FSortBy of
-   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
-   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
-   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
-   sbText: Result := CompareText(item1^.Text, item2^.Text);
-   sbCustom: Result := FOnCompare(AItem1, AItem2);
- end;
+  case FSortBy of
+    sbX: begin
+           if (FSortIndex > 0) and (FSortIndex >= FXCount) then
+             ESortError.CreateFmt(sInvalidCall, [ClassName]);
+           Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
+         end;
+    sbY: begin
+           if (FSortIndex > 0) and (FSortIndex >= FYCount) then
+             ESortError.CreateFmt(sInvalidCall, [ClassName]);
+           Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
+         end;
+    sbColor: Result := CompareValue(item1^.Color, item2^.Color);
+    sbText: Result := CompareText(item1^.Text, item2^.Text);
+    else raise ESortError.CreateFmt(sInvalidCall, [ClassName]);
+  end;
  if FSortDir = sdDescending then Result := -Result;
 end;
 
@@ -845,7 +868,7 @@
   BeginUpdate;
   try
     FDataPoints.Assign(AValue);
-    if IsSorted then Sort;
+    if Sorted then Sort;
   finally
     EndUpdate;
   end;
patch_new_update.diff (4,705 bytes)

Marcin Wiazowski

2019-04-15 04:18

reporter   ~0115513

I reviewed the new code carefully. I found some issues, so I'm attaching a patch for them (it modifies only the just-introduced code, so I decided to not split it into many small patches...).


1) I introduced a SetOnCompare() setter. This is because the custom comparing function is a sorting parameter in the same way, as SortBy / SortDir / SortIndex are - so changing the comparing function should resort the data.


2) To make the QuickSort() procedure even quicker, I changed a way, in which FData is accessed, to avoid executing range checks at each data access (QuickSort implementation guarantees, that indices are always valid).


3) A normal way of preparing the sorting algorithm is to call a sequence of code like:

  Source.SortBy := sbY;
  Source.SortDir := sdDescending;
  Source.SortIndex := 1;
  Source.Sorted := true;

Unfortunately, each of these lines causes currently resorting. This is because SetSortBy(), SetSortDir() and SetSortIndex() make a Sort() call unconditionally; also SetSorted() calls Sort() unconditionally - so sorting is executed even when it has been just disabled...

The attached patch solves the problem by using "if Sorted then Sort" statements, instead of just "Sort".


4) In TCustomSortedChartSource.Sort():

- When data is loaded from LFM stream, it is assumed, that LFM is coherent - i.e. all data points and all the properties are in a coherent state. So I disabled sorting in case of loading source from LFM, to avoid sorting on each SetSortBy(), SetSortDir() and SetSortIndex() call, if Sorted has already been set to True (as we remember, we can't depend on data ordering in the LFM stream).

- If FOnCompare is null, Notify() is no longer called (because nothing happens).

- Checks for out-of-range XList/YList indices were added:

  if (FSortBy = sbX) and (FSortIndex > 0) and (FSortIndex >= FXCount) then exit;
  if (FSortBy = sbY) and (FSortIndex > 0) and (FSortIndex >= FYCount) then exit;


5) To be on the safe side, DefaultCompare() function (which, potentially, may be called directly in descendant implementations), also checks for out-of-range XList / YList accesses.

In addition, it no longer accepts the SortBy = sbCustom case - this should never happen (see the TCustomSortedChartSource.Sort() implementation). Currently, FOnCompare() is called - which can even be null.


6) Tiny change in TListChartSource.SetDataPoints() was made, just to make its code more similar to the other code.

Marcin Wiazowski

2019-04-15 16:37

reporter   ~0115523

In introduced ugly bugs: virtual methods, marked as "inline", lose their virtuality!


So "inline" must be removed from these implementations:

In tasources.pas:
- TCustomSortedChartSource.IsSorted: Boolean;
- TRandomChartSource.IsSorted: Boolean;
- TUserDefinedChartSource.IsSorted: Boolean;

In tacustomsource.pas:
- TCustomChartSource.IsSorted: Boolean;


I'm surprised - compiler should return some error or just ignore inlining.

Marcin Wiazowski

2019-04-15 17:27

reporter   ~0115526

I have some notices about TListChartSource, which may be useful.

Probably it's now a good moment to solve some problems, to avoid going the wrong way. Currently, only sorting by X in the ascending order is handled internally in TListChartSource (these IsSortedByXAsc calls) - but due to last changes, also other sorting methods must be added to methods like Add(). This can be made in a painless way - but not now, when there is a big mess:

- TListChartSource.Sort() uses CompareFloat() for comparisons, which handles NaNs
- on the contrary, TListChartSource.Add() uses normal ">" operations to handle sorted data, which doesn't handle NaNs
- TListChartSource.Add() allows to add NaNs to sorted data sources
- on the contrary, TListChartSource.SetXValue() doesn't allow to change value to NaN in a sorted data source (raises an exception)...
- ... but NaN may still be set for a non-sorted source - which may become sorted later...
- and for sorted souces, TListChartSource.Add() and TListChartSource.SetXValue() use sequential search to find a proper index for new data - but sources are sorted, so binary search should be used, which is much faster...



Now possible solutions. Currently, TCustomSortedChartSource uses "FData: TFPList" to store TChartDataItems. But, in our case, there are better lists than TFPList:


1) TFPSList ("Basic list of memory blocks") can be used - it has a built-in Sort() method, which gets compare function being a **method**. It doesn't implement binary search algorithm (doesn't have Find() method) - but binary search can be grabbed from fgl.pp, from TFPSMap.Find().


2) TFPSMap = class(TFPSList) ("Basic map object, used in generic maps") can be used, which has binary search - i.e. Find() method - built-in.


3) TFPGMap <TKey,TData>= class(TFPSMap) ("Generic map") can be also used, which - maybe - is more convenient than pure TFPSMap.



Having TFPSList + binary search / TFPSMap / TFPGMap we can:

- Use DefaultCompare() as a compare function for TFPSList / TFPSMap / TFPGMap.

- Just use a single Find() call, to find a proper index for new/modified data in TListChartSource's Add() / SetXValue() / SetYValue() / SetColor() / SetText() etc. No more manually created loops, like in TListChartSource.Add() or TListChartSource.SetXValue().

- Use list's built-in Sort() method.



FData was private all the time, so nobody will suffer from incompatibility, after changing it to some other type of list.



In this case, limitation for NaNs in TListChartSource.SetXValue() will no longer be needed (well, even now it is not needed...).

wp

2019-04-15 18:52

developer   ~0115527

Uhhh, I should have known that this will happen...

So please go ahead and prepare a patch for whatever list that fits you. The solution should be general enough so that a "list" of indexes stored by the planned TSortedChartList can be sorted smoothly (I was planning to call the DefaultCompare and FOnCompare, now bundeled as FCompareProc, from the compare function of the integer list; I should mention that the PChartDataItems of the Origin source cannot be stored in the FData of the TSortedChartList directly because they may have volatile storage).

Marcin Wiazowski

2019-04-15 23:42

reporter   ~0115532

I performed some additional research, so I'm placing some conclusions for informational purposes.


Using TFPSList / TFPSMap / TFPGMap would be an overkill. They have great possibilities, but are less efficient. They use configurable item size, and make Move() calls even to exchange two items - while TFPList stores and exchanges just pointers. What's more, TFPSMap and TFPGMap store not just data, but records containing data-key pairs; we don't need this.


Currently, TListChartSource uses the following interesting references to its FData object:
- Insert()
- sorting

In addition, there are also references that we don't have to care about (i.e. no changes are required in their functionality):
- Move()
- Delete()
- Clear()
- reading Count()
- reading Items[]


So now I think, that the best solution would be to create a descendant like: TSortedList = class(TFPList). There are already some TFPList descendants in tachartutils.pas, so TSortedList should also be implemented there.

This TSortedList should:
- reimplement Sort(), to handle compare function being a method,
- implement binary search, i.e. Find(),
- reimplement Insert() (which is currently used mainly to add data at the end of data set): in case of sorted sources, Insert() should use Find() to find a proper index for the the data being inserted; this should be implemented in the similar way as it is now in TFPSMap.Add(),
- implement Modify(), which should use Find() to find a new location (index) for the the data being modified.


About TCalculatedChartSource: it seems, that TCustomSortedChartSource's FData list should be just ignored and remain empty. Instead, some integer list should be used.


I'll make some experiments with TSortedList.

Marcin Wiazowski

2019-04-17 04:07

reporter   ~0115576

I can see some subtle change in:

  procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
  begin
    if FSorted = AValue then exit;
    FSorted := AValue;
    if Sorted then Sort;
  end;

to:

    if Sorted then Sort else Notify;


Is there any specific reason for adding the Notify() call? When changing FSorted to false, nothing happens - data points are not modified in any way, so it seems that there is no need to notify client series (which will redraw the chart)...

wp

2019-04-17 09:12

developer   ~0115580

I see many reasons for the new code.

The user may have hooked some code into, e.g., TChart.OnAfterPaint in order to display the sorted state in a label or whatever.

  procedure TMainForm.ChartAfterPaint(ASender: TChart);
  begin
    if ListChartSource.Sorted then
      Label1.Caption := 'SORTED'
    else
      Label1.Caption := 'NOT SORTED';
  end;

Or the user may want to restore the unsorted order.

Marcin Wiazowski

2019-04-17 12:27

reporter   ~0115588

As far as I noticed, all the current TAChart's code makes a notification when some change has been made to source's data contents - or when it cannot be determined if some change in source's data contents occurred or not (like in the EndUpdate() call).

Calling the notification, when data source **property's value** was changed (that doesn't alter data contents themselves) is something new... Notification causes redrawing the chart, and redrawing the chart only to get the OnChartAfterPaint call, to set a label, is a brave idea, I think :)

I can see two solutions, that aren't so expensive:

  procedure DisableSource;
  begin
    SomeSource.Sorted := false;
    UpdateSourceLabel;
  end;

or SetSorted() method might be moved from private to protected section, so someone could - if really musts - override it... SetSortBy(), SetSortDir() and SetSortIndex() are all protected, so, in fact, it would be even more consistent to make also SetSorted() protected.

wp

2019-04-17 16:45

developer   ~0115594

When the "else" part of

   if Sorted then Sort else Notify;

is removed then notification is asymmetric. Sort is calling "Notify" when the source is sorted. So why shouldn't it be called when the source is no longer sorted. Suppose the user wants the series to return to the unsorted state. How should the user know that the series is not sorted any more?

What's the problem when the chart is redrawn unnecessarily? All the internal chart notifications end this way, I am sure this is not the only unnecessary one.

When drawing the chart takes too long the user has a problem anyway.

Marcin Wiazowski

2019-04-17 17:25

reporter   ~0115599

> When the "else" part [..] is removed then notification is asymmetric.

Yes, of course.


> Sort is calling "Notify" when the source is sorted. So why shouldn't it be called when the source is no longer sorted.

I don't want to say, that the behavior cannot be changed - but please note, that this asymmetric behavior was always there: for example, when we look at latest official Lazarus release, we can see:

  procedure TListChartSource.SetSorted(AValue: Boolean);
  begin
    if FSorted = AValue then exit;
    FSorted := AValue;
    if Sorted then begin
      Sort;
      Notify;
    end;
  end;

the only difference was that Sort() wasn't calling Notify() internally at these times (which was a bug, since Sort() is a public method) - so I added Notify() call to Sort(), and also removed if from SetSorted(), which was introduced somewhere between r60921 and r60932:

  procedure TListChartSource.SetSorted(AValue: Boolean);
  begin
    if FSorted = AValue then exit;
    FSorted := AValue;
    if Sorted then Sort;
  end;


> Suppose the user wants the series to return to the unsorted state. How should the user know that the series is not sorted any more?

What has been seen, cannot be unseen - and what has been sorted, cannot be unsorted :)

The series just visually represents (on the chart) source's data. Setting Sorted to False doesn't change source's data in any way, so - according to current TAChart conventions - we don't need to notify client series - because the only thing, that series can do in this case, is to draw itself with the exactly same visual representation as it is...


> What's the problem when the chart is redrawn unnecessarily? All the internal chart notifications end this way

Well, we have a lot of code like:

  procedure TChartAxisIntervalParams.SetCount(AValue: Integer);
  begin
    if FCount = AValue then exit;
    FCount := AValue;
    Changed; <==== caled only if value has been really changed
  end;

  procedure TChartAxisIntervalParams.SetMaxLength(AValue: Integer);
  begin
    if FMaxLength = AValue then exit;
    FMaxLength := AValue;
    Changed; <==== caled only if value has been really changed
  end;

  procedure TChartAxisIntervalParams.SetMinLength(AValue: Integer);
  begin
    if FMinLength = AValue then exit;
    FMinLength := AValue;
    Changed; <==== caled only if value has been really changed
  end;

  procedure TChartAxisIntervalParams.SetNiceSteps(const AValue: String);
  begin
    if FNiceSteps = AValue then exit;
    FNiceSteps := AValue;
    ParseNiceSteps;
    Changed; <==== caled only if value has been really changed
  end;

  procedure TChartAxisIntervalParams.SetOptions(
    AValue: TAxisIntervalParamOptions);
  begin
    if FOptions = AValue then exit;
    FOptions := AValue;
    Changed; <==== caled only if value has been really changed
  end;

and same is also when modifying any property of any series...

So I would rather say, that special efforts have been made, to avoid useless notifications...

wp

2019-04-17 18:14

developer   ~0115602

I don't want to deepen this discussion of two words. But I want to put this right:

> What has been seen, cannot be unseen - and what has been sorted, cannot be unsorted :)

Maybe true for "seen", but sometimes wrong for "sorted": In the recent modifications of some series types and the new demo "sorted_source" the (by myself much-hated) XCount = 0 has turned into a benefit. When a source has XCount=0 these series types assume now that the x value is equal to the data point index. So, when such a source is sorted by, say, ascending Y, then the data points are automatically replotted in the new order of ascending Y from left to right. Now, when the original order has been put into the "real" x values in ascending order the original order can be restored by setting XCount to 1 and sorting by x (it is not contained in the demo, but it must work).

Of course, not useful in general, but a demonstration of advantages of sorting - we don't have so many so far, regarding all the efforts put into it...

(Note in the demo containing a variety of series types that only the first two pages (Pie series and bar series) have a "real" meaning, the others only use the same source as test cases to verify that the data points move when sort criteria are changed.)

Marcin Wiazowski

2019-04-17 23:51

reporter   ~0115612

I played with your demo and understood your words, that you have to prepare some demo about sorting, otherwise nobody will be able to comprehend sorting...


Ok. About SetSorted(): I can live with or without a notification when sorting is being disabled - well, disabling the sorting is something, that occurs maybe two times per application run, rather than per second.

However, it seems, that chart's machinery is designed properly, because - even without this notification in SetSorted() - your example with "unsorting" will work gracefully; this is because changing XCount to 0 or back to 1 will execute the notification (because changing XCount definitely modifies source's data)...

Marcin Wiazowski

2019-04-18 01:04

reporter   ~0115617

Last edited: 2019-04-18 01:06

View 2 revisions

Semi-related: new bug in TCustomChartSource.FindBounds(). There is:

  if (XCount = 0) then begin
    ALB := trunc(AXMin);
    AUB := ceil(AXMax);
  end else

but "trunc" should be changed to "floor": trunc(-2.8) is -2, floor(-2.8) is -3.

wp

2019-04-18 01:15

developer   ~0115618

I had this, but the comment of the function says: "ALB -> leftmost item where X >= AXMin". Since x is the data point index when XCount=0, there can be no negative values and I used trunc.

Marcin Wiazowski

2019-04-18 02:36

reporter   ~0115622

No, unfortunately not. In practice, AXMin .. AXMax is a range on the horizontal axis, that is going to be displayed. Chart.Extent or Axis.Range may be set in such way, that AXMin .. AXMax will be -5 .. -2. Or zooming tool may be used. You can also try this:

  procedure TForm1.Button1Click(Sender: TObject);
  var
    ALB, AUB: Integer;
  begin
    ListChartSource1.FindBounds(-30, -20, ALB, AUB);
  end;



By the way, you found a bug in the comment. There is:

// ALB -> leftmost ...
// ALB -> rightmost ...

so we have ALB twice. There should be:

// ALB -> leftmost ...
// AUB -> rightmost ...

Marcin Wiazowski

2019-04-18 15:19

reporter   ~0115649

Some test:

procedure TForm1.Button1Click(Sender: TObject);
var
  I: Integer;
  ALB, AUB: Integer;
begin
  ListChartSource1.Clear;
  for I := 0 to 10 do
    ListChartSource1.Add(I, I);

  ListChartSource1.FindBounds(-30, -20, ALB, AUB);
  // ALB = 0
  // AUB = -1

  ListChartSource1.FindBounds(20, 30, ALB, AUB);
  // ALB = 11
  // AUB = 10

end;

so these ceil/floor/trunc calls are not enough.

wp

2019-04-18 16:28

developer   ~0115653

Yes, should have read the comment above the routine more carefully...

wp

2019-04-19 14:38

developer   ~0115674

Last edited: 2019-04-20 11:28

View 2 revisions

I did not really notice that you change FSortBy, FSortDir, FSortIndex and FSorted in TCalculatedChartSource.Changed here when FOrigin is nil. Is this really necessary? This source cannot be sorted at all, so why should it modify its inherited sorting parameters?

Marcin Wiazowski

2019-04-27 23:30

reporter   ~0115861

That trunc/ceil problem is now solved.

Marcin Wiazowski

2019-04-27 23:45

reporter   ~0115862

> This source cannot be sorted at all

Not exactly. TCustomAnimatedChartSource in fact cannot be sorted in any case (because the user may freely modify each data point), so assigning sorting properties there is useless.

But TCalculatedChartSource modifies only Y values (coming from the original data source), so TCalculatedChartSource can still be sorted as long, as the selected sorting method does not rely on Y values for sure - i.e. can be sorted for sbX, sbColor and sbText modes.



> you change FSortBy, FSortDir, FSortIndex and FSorted in TCalculatedChartSource.Changed here when FOrigin is nil. Is this really necessary?

In fact - not. If you don't like them, you can freely remove the following lines in TCalculatedChartSource.Changed():

  FSortBy := sbX;
  FSortDir := sdAscending;
  FSortIndex := 0;

I placed these lines for three reasons:
- lack of any assignments for these properties might be supposed to be some unintentional omission,
- object returns always to the exactly same internal state (which is not necessary; man could consider this as a good design practice),
- if the user will inherit from TCalculatedChartSource and publish sorting properties, their default state will prevent them from being written to the LFM stream (which would be useless).

wp

2019-04-29 10:59

developer   ~0115887

None of these sources can be sorted themselves because for this purpose they would require something to store the sorted values. What can be sorted is the Origin (maybe), and the question is: Should the sorted state of the Origin be propagated to the properties of the calculated or animated chart sources? Maybe for x and maybe for the calculated source because then the Extent calculation can be simplified when the source knows that it is sorted by x. But even here I am in doubt since I am planning to add an X-Y exchange option to the calculated source (to make a data listsource with labels usable for axis labels for a rotated series). Putting it all together, there are so many assumptions and cases which may be justified now but may be invalid in the future and become stumbling blocks. So, in total, I would like to follow the original author, Alexander Klenin, and assume these source to be unsorted.

Marcin Wiazowski

2019-04-29 12:20

reporter   ~0115889

I don't have a problem with it. However, if your decision is not yet final, I can see a solution for handling also the X-Y exchange option: if I understand correctly, the idea is to virtually switch X coordinate with Y coordinate in every data point, in the whole data set - so every X coordinate becomes just an Y coordinate for the calculated source's client, and vice versa; in this case:

a) if origin is sorted by X - then calculated source just becomes sorted by Y,
b) if origin is sorted by Y - then calculated source just becomes sorted by X,
c) if origin is sorted by Color or Text - then calculated source is also sorted by Color or Text
d) if origin is sorted by some custom algorithm - then calculated source is always unsorted

so the code could look just like:


  procedure TCalculatedChartSource.Changed(ASender: TObject);
  begin
    if FOrigin <> nil then begin
      FSortBy := TCustomChartSourceAccess(Origin).SortBy;

====> ADDED:
      if ExchangingIsActive then
        case FSortBy of
          sbX : FSortBy := sbY;
          sbY : FSortBy := sbX;
        end;

      FSortDir := TCustomChartSourceAccess(Origin).SortDir;
      FSortIndex := TCustomChartSourceAccess(Origin).SortIndex;
      // We recalculate Y values, so we can't guarantee, that transformed
      // data is still sorted by Y or by Origin's custom algorithm
      FSorted := (FSortBy in [sbX, sbColor, sbText]) and Origin.IsSorted;

      ...


This way, rotated series could still work as good as not rotated ones, when using calculated series.

wp

2019-04-30 10:34

developer   ~0115914

Of course, an IF here and there, and lots of more cases will be possible, but the sources will become more and more difficult to maintain.

There is another issue related to sorting which came to my mind recently when I wrote a demo for a forum post: I think that time-series data are not yet optimized. In this case, data arrive with increasing time, i.e. they are quasi "sorted by X" although the source itself does not do any sorting on its own. The extent calculation should take advantage of it. The straightforward solution would set SortBy=sbX and Sorted=true, but this causes a dramatic slowdown since the insertion point of a new data point must be found by binary search although the correct index - at the end of the list - is trivial. Or setting Sorted=true at the end of data collection would re-sort an already sorted source.

Marcin Wiazowski

2019-05-01 15:02

reporter   ~0115944

I have some useful notes, but I can't respond now in a more detailed way.

I'll be back on Sunday.

Marcin Wiazowski

2019-05-05 22:55

reporter   ~0116030

Ok, I'm back.


I have some observations about time series, so I attached a SpeedTest application here. Performed test are:

1) add many points to list source with ascending X values

2) set source's Sorted to True, then add many points to list source with ascending X values

3) add many points to list source with ascending X values, then set source's Sorted to True

4) set source's Sorted to True, then add many points to list source with RANDOM X values

5) add many points to list source with RANDOM X values, then set source's Sorted to True


Sample results:

1) 187 ms
2) 187 ms
3) 1310 ms
4) 5880 ms
5) 2580 ms


Note: TListChartSource.Add() method is currently very non-optimal for case 4) - it doesn't use binary search to find a placement for the new item; this will change after implementing TSortedList = class(TFPList). To avoid hanging, test 4 adds only 40000 points, instead of 1000000.


The conclusion is: for time series, setting sorting properties BEFORE adding the points doesn't cause any (noticeable) slow down. On the contrary, setting sorting properties AFTER adding the points executes the quick sort algorithm - which is highly time-consuming even for sorted data (and much more time-consuming for random data).


Some additional note:

> this causes a dramatic slowdown since the insertion point of a new data point must be found by binary search although the correct index - at the end of the list - is trivial

Fortunately, as the test shows, case 2) is optimal enough even now. However, when implementing TSortedList = class(TFPList), I will take special care of the time series case - so adding sorted data at the end will not even trigger the binary search.




About X-Y exchanging functionality: I think, that you are a bit too pessimistic... TCalculatedChartSource is in fact quite complicated - but most of the complicated code is related to Y transformations - i.e. to the core TCalculatedChartSource's functionality. The remaining code are mainly simple constructor and destructor, and quite simple property setters. And there are in fact two methods, that are responsible for reflecting the original source: GetCount() and Changed(); once reflecting is implemented properly, it doesn't need to be changed anymore. Adding any new functionality will always need some adjustments in the currently existing code - this is rather obvious. But it seems that adding the X-Y exchanging is not a very complicated task; I made some test - please take a look at the attached exchange_test.diff.

SpeedTest.zip (2,536 bytes)
exchange_test.diff (3,605 bytes)
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 61162)
+++ components/tachart/tasources.pas	(working copy)
@@ -219,6 +219,7 @@
     FAccumulationDirection: TChartAccumulationDirection;
     FAccumulationMethod: TChartAccumulationMethod;
     FAccumulationRange: Cardinal;
+    FExchangeXY: Boolean;
     FHistory: TChartSourceBuffer;
     FIndex: Integer;
     FItem: TChartDataItem;
@@ -241,6 +242,7 @@
     procedure SetAccumulationDirection(AValue: TChartAccumulationDirection);
     procedure SetAccumulationMethod(AValue: TChartAccumulationMethod);
     procedure SetAccumulationRange(AValue: Cardinal);
+    procedure SetExchangeXY(AValue: Boolean);
     procedure SetOrigin(AValue: TCustomChartSource);
     procedure SetPercentage(AValue: Boolean);
     procedure SetReorderYList(const AValue: String);
@@ -263,6 +265,8 @@
     property AccumulationRange: Cardinal
       read FAccumulationRange write SetAccumulationRange default 2;
 
+    property ExchangeXY: Boolean
+      read FExchangeXY write SetExchangeXY default false;
     property Origin: TCustomChartSource read FOrigin write SetOrigin;
     property Percentage: Boolean
       read FPercentage write SetPercentage default false;
@@ -1491,13 +1495,20 @@
 procedure TCalculatedChartSource.Changed(ASender: TObject);
 begin
   if FOrigin <> nil then begin
-    FSortBy := TCustomChartSourceAccess(Origin).SortBy;
-    FSortDir := TCustomChartSourceAccess(Origin).SortDir;
-    FSortIndex := TCustomChartSourceAccess(Origin).SortIndex;
+    FSortBy := TCustomChartSourceAccess(FOrigin).SortBy;
+
+    if FExchangeXY then
+      case FSortBy of
+        sbX : FSortBy := sbY;
+        sbY : FSortBy := sbX;
+      end;
+
+    FSortDir := TCustomChartSourceAccess(FOrigin).SortDir;
+    FSortIndex := TCustomChartSourceAccess(FOrigin).SortIndex;
     // We recalculate Y values, so we can't guarantee, that transformed
     // data is still sorted by Y or by Origin's custom algorithm
-    FSorted := (FSortBy in [sbX, sbColor, sbText]) and Origin.IsSorted;
-    FXCount := Origin.XCount;
+    FSorted := (FSortBy in [sbX, sbColor, sbText]) and FOrigin.IsSorted;
+    FXCount := IfThen(not FExchangeXY, FOrigin.XCount, FOrigin.YCount);
     // FYCount is set below, in the UpdateYOrder() call
   end else begin
     FSortBy := sbX;
@@ -1510,7 +1521,7 @@
 
   if
     (FOrigin <> nil) and (ASender = FOrigin) and
-    (FOrigin.YCount <> FOriginYCount)
+    (IfThen(not FExchangeXY, FOrigin.YCount, FOrigin.XCount) <> FOriginYCount)
   then begin
     UpdateYOrder;
     exit;
@@ -1559,10 +1570,21 @@
   end;
 
 var
+  d: double;
   t: TDoubleDynArray;
   i: Integer;
 begin
   FItem := Origin[AIndex]^;
+
+  if FExchangeXY then begin
+    d := FItem.X;
+    FItem.X := FItem.Y;
+    FItem.Y := d;
+    t := FItem.XList;
+    FItem.XList := FItem.YList;
+    FItem.YList := t;
+  end;
+
   if Length(FYOrder) > 0 then begin
     SetLength(t, High(FYOrder));
     for i := 1 to High(FYOrder) do
@@ -1639,6 +1661,13 @@
   Changed(nil);
 end;
 
+procedure TCalculatedChartSource.SetExchangeXY(AValue: Boolean);
+begin
+  if FExchangeXY = AValue then exit;
+  FExchangeXY := AValue;
+  UpdateYOrder;
+end;
+
 procedure TCalculatedChartSource.SetOrigin(AValue: TCustomChartSource);
 begin
   if AValue = Self then
@@ -1692,7 +1721,7 @@
     exit;
   end;
 
-  FOriginYCount := FOrigin.YCount;
+  FOriginYCount := IfThen(not FExchangeXY, FOrigin.YCount, FOrigin.XCount);
   if FOriginYCount = 0 then
     FYOrder := nil
   else
exchange_test.diff (3,605 bytes)

wp

2019-05-06 01:08

developer   ~0116033

> Fortunately, as the test shows, case 2) is optimal enough even now.

Imagine a measurement device which collects some count of measurement values. When its buffer is full it sets a "data ready" line and the chart can read the new data to add them to the previous ones. In this case, we have two loops: the inner one, like in your test, when data points are added one by one to the source; and an outer one for the individual measurement cycles. The inner loop should be enclosed by a BeginUpdate/EndUpdate pair to avoid repainting the series with every data point. And this is what makes the trouble because EndUpdate invalidates the cache. Replace your Test1 and Test2 by these procedures, and see what happens already with only two measurement cycles...

procedure Test1(List: TListChartSource);
var
  I, J: Integer;
begin
  for J := 1 to 2 do begin
    List.BeginUpdate;
    for I := 1 to 1000000 do
      List.Add(I,I);
    List.EndUpdate;
  end;
end;

procedure Test2(List: TListChartSource);
var
  I, J: Integer;
begin
  List.Sorted := true;
  for J := 1 to 2 do begin
    List.BeginUpdate;
    for I := 1 to 1000000 do
      List.Add(I,I);
    List.EndUpdate;
  end;
end;

Marcin Wiazowski

2019-05-06 01:53

reporter   ~0116035

There are following problems with the modified Test2:

1) It adds X values from 1 to 1000000, and then back from 1 to 1000000 - so we have no longer a time series; so this in fact transforms test 2 into test 4. Using:

  List.Add(I+(J-1)*1000000,I);

gives us fast execution back.

2) As already mentioned above, current implementation is highly non-optimal for test 4 - although binary search will be used in this case in TSortedList = class(TFPList).

wp

2019-05-07 11:41

developer   ~0116060

Too stupid thinking...

As for exchanging X-Y: the way you do it is probably the way it should be done for a standalone feature. But there is no need for such a feature, exchanging x/y is done by the series already. For the purpose that I want it is overkill and probably leads to wrong user actions. I only want to exchange X and Y, nothing else. This is for labeling the y axis of a rotated series by the series data labels. Exchanging x and y is required here because the y axis draws the axis labels at the y values of the chartsource (because it is a y axis), but it should use the x values because datapoint x values are drawn on the y axis now. But it does not know that these particular labels are for a rotated series and thus cannot exchange x and y on its own. Therefore the source must take care of exchanging x and y for labeling purposes, but not for the data points which are already taken care of by the series. Sound confusing? Yes... I'll have to write a separate report, this topic is leading too far away.

Marcin Wiazowski

2019-05-07 14:47

reporter   ~0116064

Yes, please create a separate report - maybe I'll have some notice there.

Marcin Wiazowski

2019-05-08 23:50

reporter   ~0116085

After a more deep research I realized, that implementing TSortedList = class(TFPList) would only cause additional problems - many methods would need to be overridden in its implementation, and denied to be used (raise an exception), just to make the TSortedList internally consistent.

From the other side, the needed Sort() and Find() methods need some modifications to handle PChartDataItems properly (I'll describe this later, in another post) - so there would be also no advantage of using TFPSList / TFPSMap / TFPGMap.

So, finally, I decided to use the already implemented "FData: TFPList". Interestingly, it can also handle integer indices, as needed by the expected TSortedChartSource implementation (I'll describe this later, in another post).



To avoid providing a one big patch (which is not desired), I decided to split it into smaller pieces. So I'm attaching the first patch (phase1.diff), that makes some initial cleaning up - no logic is changed, so everything still works in the exactly same way (ok, it is also large in size - but mostly due to moving large portions of code between units):


1) TCustomSortedChartSource was moved from tasources.pas to tacustomsource.pas (which seems to be a better place for a "custom" source).


2) Methods were sorted (almost) alphabetically.


3) Since "FData: TFPList" has been moved from TListChartSource to TCustomSortedChartSource some time ago, also some basic FData handling is now moved to TCustomSortedChartSource - so copying same code between TListChartSource and future TSortedChartSource implementation will be avoided:
- FData is now created and destroyed in TCustomSortedChartSource,
- GetCount() and GetItem() were moved to TCustomSortedChartSource,
- DefaultCompare() was also moved to TCustomSortedChartSource - it will also be reused by both TListChartSource and future TSortedChartSource.


4) Some "consts" were added for Double and string method parameters (this is an optimization - Double no more needs to be copied on the stack, and string reference counter is not incremented when entering the method and then decremented when exiting).


5) Some small optimizations/refactorings were made in TListChartSource's SetXList() and SetYList().



Please apply the patch if you have no objections, so I'll provide the next one.

phase1.diff (23,495 bytes)
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 61181)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -93,11 +93,11 @@
     YList: TDoubleDynArray;
     function GetX(AIndex: Integer): Double;
     function GetY(AIndex: Integer): Double;
-    procedure SetX(AIndex: Integer; AValue: Double);
-    procedure SetX(AValue: Double);
-    procedure SetY(AIndex: Integer; AValue: Double);
-    procedure SetY(AValue: Double);
-    procedure MultiplyY(ACoeff: Double);
+    procedure SetX(AIndex: Integer; const AValue: Double);
+    procedure SetX(const AValue: Double);
+    procedure SetY(AIndex: Integer; const AValue: Double);
+    procedure SetY(const AValue: Double);
+    procedure MultiplyY(const ACoeff: Double);
     function Point: TDoublePoint; inline;
   end;
   PChartDataItem = ^TChartDataItem;
@@ -165,7 +165,7 @@
     function IsErrorBarValueStored(AIndex: Integer): Boolean;
     procedure SetKind(AValue: TChartErrorBarKind);
     procedure SetIndex(AIndex, AValue: Integer);
-    procedure SetValue(AIndex: Integer; AValue: Double);
+    procedure SetValue(AIndex: Integer; const AValue: Double);
   public
     constructor Create;
     procedure Assign(ASource: TPersistent); override;
@@ -245,7 +245,7 @@
     function FormatItem(
       const AFormat: String; AIndex, AYIndex: Integer): String; inline;
     function FormatItemXYText(
-      const AFormat: String; AX, AY: Double; AText: String): String;
+      const AFormat: String; const AX, AY: Double; const AText: String): String;
     function GetEnumerator: TCustomChartSourceEnumerator;
     function GetXErrorBarLimits(APointIndex: Integer;
       out AUpperLimit, ALowerLimit: Double): Boolean;
@@ -273,8 +273,35 @@
     property YCount: Cardinal read FYCount write SetYCount default 1;
   end;
 
-  { TChartSourceBuffer }
+  TChartSortCompare = function(AItem1, AItem2: Pointer): Integer of object;
 
+  TCustomSortedChartSource = class(TCustomChartSource)
+  private
+    FOnCompare: TChartSortCompare;
+    procedure SetOnCompare(AValue: TChartSortCompare);
+    procedure SetSorted(AValue: Boolean);
+  protected
+    FCompareProc: TChartSortCompare;
+    FData: TFPList;
+    FSorted: Boolean;
+    function DefaultCompare(AItem1, AItem2: Pointer): Integer; virtual;
+    function DoCompare(AItem1, AItem2: Pointer): Integer; virtual;
+    procedure ExecSort(ACompare: TChartSortCompare); virtual;
+    function GetCount: Integer; override;
+    function GetItem(AIndex: Integer): PChartDataItem; override;
+    procedure SetSortBy(AValue: TChartSortBy); override;
+    procedure SetSortDir(AValue: TChartSortDir); override;
+    procedure SetSortIndex(AValue: Cardinal); override;
+    property Sorted: Boolean read FSorted write SetSorted default false;
+    property OnCompare: TChartSortCompare read FOnCompare write SetOnCompare;
+  public
+    constructor Create(AOwner: TComponent); override;
+    destructor Destroy; override;
+  public
+    function IsSorted: Boolean; override;
+    procedure Sort;
+  end;
+
   TChartSourceBuffer = class
   strict private
     FBuf: array of TChartDataItem;
@@ -352,7 +379,7 @@
 
 procedure TValuesInRangeParams.RoundToImage(var AValue: Double);
 
-  function A2I(AX: Double): Integer; inline;
+  function A2I(const AX: Double): Integer; inline;
   begin
     Result := FGraphToImage(FAxisToGraph(AX));
   end;
@@ -504,7 +531,7 @@
     Result := YList[AIndex - 1];
 end;
 
-procedure TChartDataItem.MultiplyY(ACoeff: Double);
+procedure TChartDataItem.MultiplyY(const ACoeff: Double);
 var
   i: Integer;
 begin
@@ -519,7 +546,7 @@
   Result.Y := Y;
 end;
 
-procedure TChartDataItem.SetX(AValue: Double);
+procedure TChartDataItem.SetX(const AValue: Double);
 var
   i: Integer;
 begin
@@ -528,7 +555,7 @@
     XList[i] := AValue;
 end;
 
-procedure TChartDataItem.SetX(AIndex: Integer; AValue: Double);
+procedure TChartDataItem.SetX(AIndex: Integer; const AValue: Double);
 begin
   if AIndex = 0 then
     X := AValue
@@ -536,7 +563,7 @@
     XList[AIndex - 1] := AValue;
 end;
 
-procedure TChartDataItem.SetY(AValue: Double);
+procedure TChartDataItem.SetY(const AValue: Double);
 var
   i: Integer;
 begin
@@ -545,7 +572,7 @@
     YList[i] := AValue;
 end;
 
-procedure TChartDataItem.SetY(AIndex: Integer; AValue: Double);
+procedure TChartDataItem.SetY(AIndex: Integer; const AValue: Double);
 begin
   if AIndex = 0 then
     Y := AValue
@@ -787,7 +814,7 @@
   Changed;
 end;
 
-procedure TChartErrorBarData.SetValue(AIndex: Integer; AValue: Double);
+procedure TChartErrorBarData.SetValue(AIndex: Integer; const AValue: Double);
 begin
   if FValue[AIndex] = AValue then exit;
   FValue[AIndex] := AValue;
@@ -1049,7 +1076,7 @@
 procedure TCustomChartSource.FindBounds(
   AXMin, AXMax: Double; out ALB, AUB: Integer);
 
-  function FindLB(X: Double; L, R: Integer): Integer;
+  function FindLB(const X: Double; L, R: Integer): Integer;
   begin
     while L <= R do begin
       Result := (R - L) div 2 + L;
@@ -1061,7 +1088,7 @@
     Result := L;
   end;
 
-  function FindUB(X: Double; L, R: Integer): Integer;
+  function FindUB(const X: Double; L, R: Integer): Integer;
   begin
     while L <= R do begin
       Result := (R - L) div 2 + L;
@@ -1107,11 +1134,11 @@
   const AFormat: String; AIndex, AYIndex: Integer): String;
 begin
   with Item[AIndex]^ do
-    Result := FormatItemXYText(AFormat, IfThen(XCount > 0, X, double(AIndex)), GetY(AYIndex), Text);
+    Result := FormatItemXYText(AFormat, IfThen(XCount > 0, X, Double(AIndex)), GetY(AYIndex), Text);
 end;
 
 function TCustomChartSource.FormatItemXYText(
-  const AFormat: String; AX, AY: Double; AText: String): String;
+  const AFormat: String; const AX, AY: Double; const AText: String): String;
 const
   TO_PERCENT = 100;
 var
@@ -1406,7 +1433,7 @@
 var
   prevImagePos: Integer = MaxInt;
 
-  function IsTooClose(AValue: Double): Boolean;
+  function IsTooClose(const AValue: Double): Boolean;
   var
     imagePos: Integer;
   begin
@@ -1538,5 +1565,164 @@
   Result := 0.0;
 end;
 
+
+{ TCustomSortedChartSource }
+
+constructor TCustomSortedChartSource.Create(AOwner: TComponent);
+begin
+  inherited Create(AOwner);
+  FData := TFPList.Create;
+end;
+
+destructor TCustomSortedChartSource.Destroy;
+begin
+  FreeAndNil(FData);
+  inherited;
+end;
+
+function CompareFloat(const x1, x2: Double): Integer;
+begin
+  if IsNaN(x1) and IsNaN(x2) then
+    Result := 0
+  else if IsNaN(x1) then
+    Result := +1
+  else if IsNaN(x2) then
+    Result := -1
+  else
+    Result := CompareValue(x1, x2);
+end;
+
+function TCustomSortedChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
+var
+  item1: PChartDataItem absolute AItem1;
+  item2: PChartDataItem absolute AItem2;
+begin
+ case FSortBy of
+   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
+   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
+   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
+   sbText: Result := CompareText(item1^.Text, item2^.Text);
+   sbCustom: Result := FOnCompare(AItem1, AItem2);
+ end;
+ if FSortDir = sdDescending then Result := -Result;
+end;
+
+function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
+begin
+  Result := FCompareProc(AItem1, AItem2);
+end;
+
+{ Built-in sorting algorithm of the ChartSource, a standard QuickSort.
+  Copied from the classes unit because the compare function must be a method. }
+procedure TCustomSortedChartSource.ExecSort(ACompare: TChartSortCompare);
+
+  procedure QuickSort(L, R: Longint);
+  var
+    I, J: Longint;
+    P, Q: Pointer;
+  begin
+   repeat
+     I := L;
+     J := R;
+     P := FData.List^[(L + R) div 2];
+     repeat
+       while ACompare(P, FData.List^[I]) > 0 do
+         I := I + 1;
+       while ACompare(P, FData.List^[J]) < 0 do
+         J := J - 1;
+       If I <= J then
+       begin
+         Q := FData.List^[I];
+         FData.List^[I] := FData.List^[J];
+         FData.List^[J] := Q;
+         I := I + 1;
+         J := J - 1;
+       end;
+     until I > J;
+     if J - L < R - I then
+     begin
+       if L < J then
+         QuickSort(L, J);
+       L := I;
+     end
+     else
+     begin
+       if I < R then
+         QuickSort(I, R);
+       R := J;
+     end;
+   until L >= R;
+  end;
+
+begin
+  if FData.Count < 2 then exit;
+  QuickSort(0, FData.Count-1);
+end;
+
+function TCustomSortedChartSource.GetCount: Integer;
+begin
+  Result := FData.Count;
+end;
+
+function TCustomSortedChartSource.GetItem(AIndex: Integer): PChartDataItem;
+begin
+  Result := PChartDataItem(FData.Items[AIndex]);
+end;
+
+function TCustomSortedChartSource.IsSorted: Boolean;
+begin
+  Result := FSorted;
+end;
+
+procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
+begin
+  if FOnCompare = AValue then exit;
+  FOnCompare := AValue;
+  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
+end;
+
+procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
+begin
+  if FSorted = AValue then exit;
+  FSorted := AValue;
+  if Sorted then Sort else Notify;
+end;
+
+procedure TCustomSortedChartSource.SetSortBy(AValue: TChartSortBy);
+begin
+  if FSortBy = AValue then exit;
+  FSortBy := AValue;
+  if Sorted then Sort;
+end;
+
+procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
+begin
+  if FSortDir = AValue then exit;
+  FSortDir := AValue;
+  if Sorted then Sort;
+end;
+
+procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
+begin
+  if FSortIndex = AValue then exit;
+  FSortIndex := AValue;
+  if Sorted then Sort;
+end;
+
+procedure TCustomSortedChartSource.Sort;
+begin
+  if csLoading in ComponentState then exit;
+  if (FSortBy = sbCustom) then begin
+    if not Assigned(FOnCompare) then exit;
+    FCompareProc := FOnCompare;
+  end else begin
+    if (FSortBy = sbX) and (FSortIndex <> 0) and (FSortIndex >= FXCount) then exit;
+    if (FSortBy = sbY) and (FSortIndex <> 0) and (FSortIndex >= FYCount) then exit;
+    FCompareProc := @DefaultCompare;
+  end;
+  ExecSort(@DoCompare);
+  Notify;
+end;
+
 end.
 
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 61181)
+++ components/tachart/tasources.pas	(working copy)
@@ -19,33 +19,9 @@
   Classes, Types, TAChartUtils, TACustomSource;
 
 type
-  TChartSortCompare = function(AItem1, AItem2: Pointer): Integer of object;
 
-  { TCustomSortedChartSource }
+  { TListChartSource }
 
-  TCustomSortedChartSource = class(TCustomChartSource)
-  private
-    FOnCompare: TChartSortCompare;
-    procedure SetSorted(AValue: Boolean);
-    procedure SetOnCompare(AValue: TChartSortCompare);
-  protected
-    FCompareProc: TChartSortCompare;
-    FData: TFPList;
-    FSorted: Boolean;
-    function DefaultCompare(AItem1, AItem2: Pointer): Integer; virtual; abstract;
-    function DoCompare(AItem1, AItem2: Pointer): Integer; virtual;
-    procedure ExecSort(ACompare: TChartSortCompare); virtual;
-    procedure SetSortBy(AValue: TChartSortBy); override;
-    procedure SetSortDir(AValue: TChartSortDir); override;
-    procedure SetSortIndex(AValue: Cardinal); override;
-    property Sorted: Boolean read FSorted write SetSorted default false;
-    property OnCompare: TChartSortCompare read FOnCompare write SetOnCompare;
-  public
-    function IsSorted: Boolean; override;
-    procedure Sort;
-  end;
-
-  { TListChartSource }
   TListChartSource = class(TCustomSortedChartSource)
   private
     FDataPoints: TStrings;
@@ -56,11 +32,8 @@
     procedure ClearCaches;
     function NewItem: PChartDataItem;
     procedure SetDataPoints(AValue: TStrings);
-    procedure UpdateCachesAfterAdd(AX, AY: Double);
+    procedure UpdateCachesAfterAdd(const AX, AY: Double);
   protected
-    function DefaultCompare(AItem1, AItem2: Pointer): Integer; override;
-    function GetCount: Integer; override;
-    function GetItem(AIndex: Integer): PChartDataItem; override;
     procedure Loaded; override;
     procedure SetXCount(AValue: Cardinal); override;
     procedure SetYCount(AValue: Cardinal); override;
@@ -74,22 +47,22 @@
     destructor Destroy; override;
   public
     function Add(
-      AX, AY: Double; const ALabel: String = '';
+      const AX, AY: Double; const ALabel: String = '';
       AColor: TChartColor = clTAColor): Integer;
-    function AddXListYList(const AX, AY: array of Double; ALabel: String = '';
+    function AddXListYList(const AX, AY: array of Double; const ALabel: String = '';
       AColor: TChartColor = clTAColor): Integer;
     function AddXYList(
-      AX: Double; const AY: array of Double; const ALabel: String = '';
+      const AX: Double; const AY: array of Double; const ALabel: String = '';
       AColor: TChartColor = clTAColor): Integer;
     procedure Clear;
     procedure CopyFrom(ASource: TCustomChartSource);
     procedure Delete(AIndex: Integer);
     procedure SetColor(AIndex: Integer; AColor: TChartColor);
-    procedure SetText(AIndex: Integer; AValue: String);
-    function SetXValue(AIndex: Integer; AValue: Double): Integer;
+    procedure SetText(AIndex: Integer; const AValue: String);
     procedure SetXList(AIndex: Integer; const AXList: array of Double);
+    function SetXValue(AIndex: Integer; const AValue: Double): Integer;
     procedure SetYList(AIndex: Integer; const AYList: array of Double);
-    procedure SetYValue(AIndex: Integer; AValue: Double);
+    procedure SetYValue(AIndex: Integer; const AValue: Double);
   published
     property DataPoints: TStrings read FDataPoints write SetDataPoints;
     property XCount;
@@ -141,10 +114,10 @@
     procedure SetPointsNumber(AValue: Integer);
     procedure SetRandomX(AValue: Boolean);
     procedure SetRandSeed(AValue: Integer);
-    procedure SetXMax(AValue: Double);
-    procedure SetXMin(AValue: Double);
-    procedure SetYMax(AValue: Double);
-    procedure SetYMin(AValue: Double);
+    procedure SetXMax(const AValue: Double);
+    procedure SetXMin(const AValue: Double);
+    procedure SetYMax(const AValue: Double);
+    procedure SetYMin(const AValue: Double);
     procedure SetYNanPercent(AValue: TPercent);
   protected
     procedure ChangeErrorBars(Sender: TObject); override;
@@ -285,7 +258,7 @@
   strict private
     FSource: TListChartSource;
     FLoadingCache: TStringList;
-    procedure Parse(AString: String; ADataItem: PChartDataItem);
+    procedure Parse(const AString: String; ADataItem: PChartDataItem);
   private
     procedure LoadingFinished;
   protected
@@ -301,18 +274,6 @@
     procedure Insert(Index: Integer; const S: String); override;
   end;
 
-function CompareFloat(x1, x2: Double): Integer;
-begin
-  if IsNaN(x1) and IsNaN(x2) then
-    Result := 0
-  else if IsNaN(x1) then
-    Result := +1
-  else if IsNaN(x2) then
-    Result := -1
-  else
-    Result := CompareValue(x1, x2);
-end;
-
 procedure Register;
 begin
   RegisterComponents(
@@ -352,7 +313,7 @@
 
 function TListChartSourceStrings.Get(Index: Integer): String;
 
-  function NumberStr(AValue: Double): String;
+  function NumberStr(const AValue: Double): String;
   begin
     if IsNaN(AValue) then
       Result := '|'
@@ -421,7 +382,7 @@
 end;
 
 procedure TListChartSourceStrings.Parse(
-  AString: String; ADataItem: PChartDataItem);
+  const AString: String; ADataItem: PChartDataItem);
 var
   p: Integer = 0;
   parts: TStrings;
@@ -523,120 +484,10 @@
 end;
 
 
-{ TCustomSortedChartSource }
-
-function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
-begin
-  Result := FCompareProc(AItem1, AItem2);
-end;
-
-{ Built-in sorting algorithm of the ChartSource, a standard QuickSort.
-  Copied from the classes unit because the compare function must be a method. }
-procedure TCustomSortedChartSource.ExecSort(ACompare: TChartSortCompare);
-
-  procedure QuickSort(L, R: Longint);
-  var
-    I, J: Longint;
-    P, Q: Pointer;
-  begin
-   repeat
-     I := L;
-     J := R;
-     P := FData.List^[(L + R) div 2];
-     repeat
-       while ACompare(P, FData.List^[I]) > 0 do
-         I := I + 1;
-       while ACompare(P, FData.List^[J]) < 0 do
-         J := J - 1;
-       If I <= J then
-       begin
-         Q := FData.List^[I];
-         FData.List^[I] := FData.List^[J];
-         FData.List^[J] := Q;
-         I := I + 1;
-         J := J - 1;
-       end;
-     until I > J;
-     if J - L < R - I then
-     begin
-       if L < J then
-         QuickSort(L, J);
-       L := I;
-     end
-     else
-     begin
-       if I < R then
-         QuickSort(I, R);
-       R := J;
-     end;
-   until L >= R;
-  end;
-
-begin
-  if FData.Count < 2 then exit;
-  QuickSort(0, FData.Count-1);
-end;
-
-function TCustomSortedChartSource.IsSorted: Boolean;
-begin
-  Result := FSorted;
-end;
-
-procedure TCustomSortedChartSource.SetSortBy(AValue: TChartSortBy);
-begin
-  if FSortBy = AValue then exit;
-  FSortBy := AValue;
-  if Sorted then Sort;
-end;
-
-procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
-begin
-  if FSorted = AValue then exit;
-  FSorted := AValue;
-  if Sorted then Sort else Notify;
-end;
-
-procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
-begin
-  if FSortDir = AValue then exit;
-  FSortDir := AValue;
-  if Sorted then Sort;
-end;
-
-procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
-begin
-  if FSortIndex = AValue then exit;
-  FSortIndex := AValue;
-  if Sorted then Sort;
-end;
-
-procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
-begin
-  if FOnCompare = AValue then exit;
-  FOnCompare := AValue;
-  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
-end;
-
-procedure TCustomSortedChartSource.Sort;
-begin
-  if csLoading in ComponentState then exit;
-  if (FSortBy = sbCustom) then begin
-    if not Assigned(FOnCompare) then exit;
-    FCompareProc := FOnCompare;
-  end else begin
-    if (FSortBy = sbX) and (FSortIndex <> 0) and (FSortIndex >= FXCount) then exit;
-    if (FSortBy = sbY) and (FSortIndex <> 0) and (FSortIndex >= FYCount) then exit;
-    FCompareProc := @DefaultCompare;
-  end;
-  ExecSort(@DoCompare);
-  Notify;
-end;
-
-
 { TListChartSource }
 
 function TListChartSource.Add(
-  AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
+  const AX, AY: Double; const ALabel: String; AColor: TChartColor): Integer;
 begin
   Result := FData.Count;
   if IsSortedByXAsc then
@@ -665,7 +516,7 @@
 end;
 
 function TListChartSource.AddXListYList(const AX, AY: array of Double;
-  ALabel: String = ''; AColor: TChartColor = clTAColor): Integer;
+  const ALabel: String; AColor: TChartColor): Integer;
 begin
   if Length(AX) = 0 then
     raise EXListEmptyError.Create('AddXListYList: XList is empty');
@@ -688,7 +539,7 @@
 end;
 
 function TListChartSource.AddXYList(
-  AX: Double; const AY: array of Double;
+  const AX: Double; const AY: array of Double;
   const ALabel: String; AColor: TChartColor): Integer;
 begin
   if Length(AY) = 0 then
@@ -711,7 +562,7 @@
 var
   i: Integer;
 begin
-  for i := 0 to FData.Count - 1 do
+  for i := 0 to Count - 1 do
     Dispose(Item[i]);
   FData.Clear;
   ClearCaches;
@@ -775,7 +626,6 @@
 constructor TListChartSource.Create(AOwner: TComponent);
 begin
   inherited Create(AOwner);
-  FData := TFPList.Create;
   FDataPoints := TListChartSourceStrings.Create(Self);
   ClearCaches;
 end;
@@ -791,21 +641,6 @@
     FYCount := FYCountMin;
 end;
 
-function TListChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
-var
-  item1: PChartDataItem absolute AItem1;
-  item2: PChartDataItem absolute AItem2;
-begin
- case FSortBy of
-   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
-   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
-   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
-   sbText: Result := CompareText(item1^.Text, item2^.Text);
-   sbCustom: Result := FOnCompare(AItem1, AItem2);
- end;
- if FSortDir = sdDescending then Result := -Result;
-end;
-
 procedure TListChartSource.Delete(AIndex: Integer);
 begin
   // Optimization
@@ -834,20 +669,9 @@
 begin
   Clear;
   FreeAndNil(FDataPoints);
-  FreeAndNil(FData);
   inherited;
 end;
 
-function TListChartSource.GetCount: Integer;
-begin
-  Result := FData.Count;
-end;
-
-function TListChartSource.GetItem(AIndex: Integer): PChartDataItem;
-begin
-  Result := PChartDataItem(FData.Items[AIndex]);
-end;
-
 function TListChartSource.NewItem: PChartDataItem;
 begin
   New(Result);
@@ -876,7 +700,7 @@
   end;
 end;
 
-procedure TListChartSource.SetText(AIndex: Integer; AValue: String);
+procedure TListChartSource.SetText(AIndex: Integer; const AValue: String);
 begin
   with Item[AIndex]^ do begin
     if Text = AValue then exit;
@@ -912,7 +736,7 @@
   Notify;
 end;
 
-function TListChartSource.SetXValue(AIndex: Integer; AValue: Double): Integer;
+function TListChartSource.SetXValue(AIndex: Integer; const AValue: Double): Integer;
 var
   oldX: Double;
 
@@ -935,10 +759,12 @@
   if IsSortedByXAsc then
     if IsNan(AValue) then
       raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
-  oldX := Item[AIndex]^.X;
   Result := AIndex;
-  if IsEquivalent(oldX, AValue) then exit;
-  Item[AIndex]^.X := AValue;
+  with Item[AIndex]^ do begin
+    if IsEquivalent(X, AValue) then exit; // IsEquivalent() can compare also NaNs
+    oldX := X;
+    X := AValue;
+  end;
   UpdateExtent;
   if IsSortedByXAsc then begin
     if AValue > oldX then
@@ -981,7 +807,7 @@
   Notify;
 end;
 
-procedure TListChartSource.SetYValue(AIndex: Integer; AValue: Double);
+procedure TListChartSource.SetYValue(AIndex: Integer; const AValue: Double);
 var
   oldY: Double;
 
@@ -1001,9 +827,11 @@
   end;
 
 begin
-  oldY := Item[AIndex]^.Y;
-  if IsEquivalent(oldY, AValue) then exit;
-  Item[AIndex]^.Y := AValue;
+  with Item[AIndex]^ do begin
+    if IsEquivalent(Y, AValue) then exit; // IsEquivalent() can compare also NaNs
+    oldY := Y;
+    Y := AValue;
+  end;
   if FValuesTotalIsValid then
     FValuesTotal += NumberOr(AValue) - NumberOr(oldY);
   UpdateExtent;
@@ -1010,7 +838,7 @@
   Notify;
 end;
 
-procedure TListChartSource.UpdateCachesAfterAdd(AX, AY: Double);
+procedure TListChartSource.UpdateCachesAfterAdd(const AX, AY: Double);
 begin
   if IsUpdating then exit; // Optimization
   if FBasicExtentIsValid then begin
@@ -1216,7 +1044,7 @@
   Reset;
 end;
 
-procedure TRandomChartSource.SetXMax(AValue: Double);
+procedure TRandomChartSource.SetXMax(const AValue: Double);
 begin
   if FXMax = AValue then exit;
   FXMax := AValue;
@@ -1223,7 +1051,7 @@
   Reset;
 end;
 
-procedure TRandomChartSource.SetXMin(AValue: Double);
+procedure TRandomChartSource.SetXMin(const AValue: Double);
 begin
   if FXMin = AValue then exit;
   FXMin := AValue;
@@ -1237,7 +1065,7 @@
   Reset;
 end;
 
-procedure TRandomChartSource.SetYMax(AValue: Double);
+procedure TRandomChartSource.SetYMax(const AValue: Double);
 begin
   if FYMax = AValue then exit;
   FYMax := AValue;
@@ -1245,7 +1073,7 @@
   Notify;
 end;
 
-procedure TRandomChartSource.SetYMin(AValue: Double);
+procedure TRandomChartSource.SetYMin(const AValue: Double);
 begin
   if FYMin = AValue then exit;
   FYMin := AValue;
phase1.diff (23,495 bytes)

wp

2019-05-09 14:52

developer   ~0116101

Applied in r61189 and 61190. I split off the "const" changes, they have nothing to do with the sorted chartsources. (BTW, I think I have never seen any noticable improvement by "const" declarations; it's more for documentation purposes to indicate that this parameter will not be changed in the subroutine).

Please don't split into too many pieces. If everything belongs together logically it can be in the same patch even if it is large. Maybe the next patch should contain the TSortedList (*) along with the related modification of TChartListSource, and the final patch should contain TSortedChartSource.

(*) The name TSortedList is too general in my opinion and calls for naming conflicts. Maybe better: TChartSortedList or something similar.

Marcin Wiazowski

2019-05-11 18:09

reporter   ~0116138

About consts: In fact, the compiler currently doesn't seem to differentiate between Double values passed as parameters with or without the "const" keyword (although this may change in future, so adding "const", when possible, is still a good practice).

On the contrary, there is a noticeable difference when passing records, and a huge difference when passing strings - see the attached const_rec.png and const_str.png.



About sorting: Ok, I won't split into too many pieces - however, before going further, some problem should be fixed first. Currently, setting the TCustomSortedChartSource.Sorted property makes the IsSorted() method returning just True - and this was valid until adding the new sorting properties. Currently, setting SortBy to sbX or sbY, along with SortIndex to some too large value, causes a situation, when sorting cannot be performed in any way; similar situation is when setting SortBy to sbCustom, but without providing the OnCompare handler. So, for such situations, the IsSorted() method should return False - and this is what is provided by the attached phase2.diff.

The above means, that reading "Sorted" means reading a user's will, and calling "IsSorted" means reading the true component's ability to sort. So, in all places, reading the "Sorted" property should be replaced with calling the "IsSorted" method.

This also means, that changing some property related to sorting - other than "Sorted" - may result in changing the "IsSorted" returned value. As a consequence, to ensure proper work of TCalculatedChartSource, additional notifications should be added to the property setters (yes, I didn't like the Notify() call in the SetSorted() method, but it finally turned out, that this call is needed not only there, but also in the other sorting setters).

Since "IsSorted" can be further modified (i.e. overridden) in TCustomSortedChartSource's descendants, a hardcoded validation in the TCustomSortedChartSource.Sort() method was replaced with an "IsSorted" call.

const_rec.png (25,093 bytes)
const_rec.png (25,093 bytes)
const_str.png (38,842 bytes)
const_str.png (38,842 bytes)
phase2.diff (3,019 bytes)
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 61203)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -1675,7 +1675,16 @@
 
 function TCustomSortedChartSource.IsSorted: Boolean;
 begin
-  Result := FSorted;
+  case FSortBy of
+    sbX:
+      Result := FSorted and ((FSortIndex = 0) or (FSortIndex < FXCount));
+    sbY:
+      Result := FSorted and ((FSortIndex = 0) or (FSortIndex < FYCount));
+    sbColor, sbText:
+      Result := FSorted;
+    sbCustom:
+      Result := FSorted and Assigned(FOnCompare);
+  end;
 end;
 
 procedure TCustomSortedChartSource.SetOnCompare(AValue: TChartSortCompare);
@@ -1682,7 +1691,7 @@
 begin
   if FOnCompare = AValue then exit;
   FOnCompare := AValue;
-  if Assigned(FOnCompare) and (FSortBy = sbCustom) and Sorted then Sort;
+  if IsSorted then Sort else Notify;
 end;
 
 procedure TCustomSortedChartSource.SetSortBy(AValue: TChartSortBy);
@@ -1689,7 +1698,7 @@
 begin
   if FSortBy = AValue then exit;
   FSortBy := AValue;
-  if Sorted then Sort;
+  if IsSorted then Sort else Notify;
 end;
 
 procedure TCustomSortedChartSource.SetSortDir(AValue: TChartSortDir);
@@ -1696,7 +1705,7 @@
 begin
   if FSortDir = AValue then exit;
   FSortDir := AValue;
-  if Sorted then Sort;
+  if IsSorted then Sort else Notify;
 end;
 
 procedure TCustomSortedChartSource.SetSorted(AValue: Boolean);
@@ -1703,7 +1712,7 @@
 begin
   if FSorted = AValue then exit;
   FSorted := AValue;
-  if Sorted then Sort else Notify;
+  if IsSorted then Sort else Notify;
 end;
 
 procedure TCustomSortedChartSource.SetSortIndex(AValue: Cardinal);
@@ -1710,20 +1719,28 @@
 begin
   if FSortIndex = AValue then exit;
   FSortIndex := AValue;
-  if Sorted then Sort;
+  if IsSorted then Sort else Notify;
 end;
 
 procedure TCustomSortedChartSource.Sort;
+var
+  SaveSorted: Boolean;
 begin
   if csLoading in ComponentState then exit;
-  if (FSortBy = sbCustom) then begin
-    if not Assigned(FOnCompare) then exit;
-    FCompareProc := FOnCompare;
-  end else begin
-    if (FSortBy = sbX) and (FSortIndex <> 0) and (FSortIndex >= FXCount) then exit;
-    if (FSortBy = sbY) and (FSortIndex <> 0) and (FSortIndex >= FYCount) then exit;
+
+  // Avoid useless sorting and notification
+  SaveSorted := FSorted;
+  try
+    FSorted := true;
+    if not IsSorted then exit;
+  finally
+    FSorted := SaveSorted;
+  end;
+
+  if FSortBy = sbCustom then
+    FCompareProc := FOnCompare
+  else
     FCompareProc := @DefaultCompare;
-  end;
   ExecSort(@DoCompare);
   Notify;
 end;
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 61203)
+++ components/tachart/tasources.pas	(working copy)
@@ -695,7 +695,7 @@
   BeginUpdate;
   try
     FDataPoints.Assign(AValue);
-    if Sorted then Sort;
+    if IsSorted then Sort;
   finally
     EndUpdate;
   end;
phase2.diff (3,019 bytes)

wp

2019-05-12 22:21

developer   ~0116150

Applied, r61211.

Marcin Wiazowski

2019-05-14 23:53

reporter   ~0116200

I'm attaching a patch, that makes sorting fully implemented in TCustomSortedChartSource and TListChartSource.



Let's start from TListChartSource. There are six lowest-level methods, that can modify an already existing data point:
- SetColor(),
- SetText(),
- SetXList(),
- SetXValue(),
- SetYList(),
- SetYValue().

All of them are now functions, and return a new index of the modified data point - previously, only SetXValue() was behaving in this way, and the others were procedures. This change is needed because - after being modified - data point may be moved to another location, to retain the source still sorted.

To handle moving, all these methods call a newly introduced, protected TCustomSortedChartSource.ItemModified() method. It internally uses a binary search algorithm - implemented in TCustomSortedChartSource.ItemFind() - to efficiently search for a new item location.



There are also two higher-level methods, that can modify (internally, in their implementation) an already existing data point:
- AddXListYList(),
- AddXYList().

Internally, they call some of the above-listed lowest-level methods - which are now used as functions.



An internal AddAt() helper method is no longer needed and was removed.



Calls to FData.Insert() were replaced with calls to newly introduced, protected TCustomSortedChartSource.ItemAdd() or ItemInsert() methods - if our source is sorted, they call ItemFind() internally, to find a location for the item being added, or verify a location of the item being inserted.



The CopyFrom() method is partially refactored now, to avoid calling the removed AddAt() method - as a nice side effect, data points having multiple X and/or Y values are now handled more efficiently. Also a newly-implemented HasSameSorting() method is used now - see below.



In SetXValue() method, a no-longer-needed check for NaN values was removed.



============



Let's take a look at TCustomSortedChartSource now. Previously, FCompareProc variable was used to hold the current sorting procedure - it was initialized in the Sort() method. But, currently, there was also a need to have sorting initialized in ItemFind() and ItemModified() methods, so using FCompareProc became troublesome. So I refactored the code a bit:
- DefaultCompare() and DoCompare() were merged to DoCompare(),
- ExecSort() was renamed to DoSort() (using "Do" in name seems to be a more often used convention, and is also coherent with "DoCompare"), and it also just uses DoCompare() for comparisons now, instead of getting the compare function as a parameter.

Both DoSort() and DoCompare() are virtual:
- DoSort() can be overridden to use some algorithm other than quick sort,
- DoCompare() can be overridden to implement some own comparing function (this will be also used in the last patch, that I'll provide later).

DoCompare() makes now some additional validations in sbX and sbY cases - but this doesn't impact the execution speed, because X / Y / XList / YList are now used instead of GetX() / GetY() calls. This change is because GetX() and GetY() return other-than-requested XList / YList item, in case of out-of-range index; after the change, NaN is used instead, which is a valid solution.



DoSort() introduces now an improvement, that is required when using quick sort algorithm on multi-dimensional data (where dimensions are: X, Y, XList, YList, Color and Text) to sort the data by using only one of the dimensions: equal items are no longer exchanged. There is a more detailed explanation in the code.



ItemFind() has a special modification, so - if data points are added at the end, and in the sorted order - only one comparison is made, instead of executing the whole binary search algorithm.



============



The last modification is in TCustomChartSource: a protected HasSameSorting(ASource: TCustomChartSource) method was added. It is currently used in TListChartSource.CopyFrom() method (and will be also used in the last patch, that I'll provide later). The method takes TCustomChartSource as a parameter (this is needed, in particular, in TListChartSource.CopyFrom()), so it's implemented also in TCustomChartSource.

HasSameSorting() can be used to check if two sources have some sorting, when assigning one source's data to the another source - if so, data resorting can be avoided.

The method is virtual, so can be overridden by the user, to check also if some custom sorting algorithm gives same data order in two user-implemented TCustomChartSource descendants.



============



I executed the attached SpeedTest application again, after applying the described changes. Comparison is (without changes / with changes):

1) 187 ms ===> 234 ms
2) 187 ms ===> 265 ms
3) 1310 ms ===> 920 ms
4) 5880 ms ===> 140 ms
5) 2580 ms ===> 2090 ms

Longer execution in 1) is due to added lacking try..except..end statement in TListChartSource.Add().

The difference between 234 ms and 265 ms is a price of the more universal sorting implementation.

Although I haven't checked it in detail, faster execution in 3), 4) and 5) cases seems to be due to using binary search algorithm and a more efficient DoCompare() and DoSort() implementations.



The last patch, that I'll provide later, will bring the sorted source implementation.

phase3.diff (18,275 bytes)
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 61227)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -212,6 +212,7 @@
       out AUpperDelta, ALowerDelta: Double): Boolean;
     function GetHasErrorBars(Which: Integer): Boolean;
     function GetItem(AIndex: Integer): PChartDataItem; virtual; abstract;
+    function HasSameSorting(ASource: TCustomChartSource): Boolean; virtual;
     procedure InvalidateCaches;
     procedure SetSortBy(AValue: TChartSortBy); virtual;
     procedure SetSortDir(AValue: TChartSortDir); virtual;
@@ -283,14 +284,16 @@
     procedure SetOnCompare(AValue: TChartSortCompare);
     procedure SetSorted(AValue: Boolean);
   protected
-    FCompareProc: TChartSortCompare;
     FData: TFPList;
     FSorted: Boolean;
-    function DefaultCompare(AItem1, AItem2: Pointer): Integer; virtual;
     function DoCompare(AItem1, AItem2: Pointer): Integer; virtual;
-    procedure ExecSort(ACompare: TChartSortCompare); virtual;
+    procedure DoSort; virtual;
     function GetCount: Integer; override;
     function GetItem(AIndex: Integer): PChartDataItem; override;
+    function ItemAdd(AItem: PChartDataItem): Integer;
+    procedure ItemInsert(AIndex: Integer; AItem: PChartDataItem);
+    function ItemFind(AItem: PChartDataItem; L: Integer = 0; R: Integer = High(Integer)): Integer;
+    function ItemModified(AIndex: Integer): Integer;
     procedure SetSortBy(AValue: TChartSortBy); override;
     procedure SetSortDir(AValue: TChartSortDir); override;
     procedure SetSortIndex(AValue: Cardinal); override;
@@ -1298,6 +1301,20 @@
     end;
 end;
 
+function TCustomChartSource.HasSameSorting(ASource: TCustomChartSource): Boolean;
+begin
+  case SortBy of
+    sbX, sbY:
+      Result := ASource.IsSorted and (ASource.SortBy = SortBy) and
+                (ASource.SortDir = SortDir) and (ASource.SortIndex = SortIndex);
+    sbColor, sbText:
+      Result := ASource.IsSorted and (ASource.SortBy = SortBy) and
+                (ASource.SortDir = SortDir);
+    sbCustom:
+      Result := false;
+  end;
+end;
+
 function TCustomChartSource.HasXErrorBars: Boolean;
 begin
   Result := GetHasErrorBars(0);
@@ -1596,71 +1613,129 @@
     Result := CompareValue(x1, x2);
 end;
 
-function TCustomSortedChartSource.DefaultCompare(AItem1, AItem2: Pointer): Integer;
+function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
 var
   item1: PChartDataItem absolute AItem1;
   item2: PChartDataItem absolute AItem2;
+  d1, d2: Double;
 begin
- case FSortBy of
-   sbX: Result := CompareFloat(item1^.GetX(FSortIndex), item2^.GetX(FSortIndex));
-   sbY: Result := CompareFloat(item1^.GetY(FSortIndex), item2^.GetY(FSortIndex));
-   sbColor: Result := CompareValue(item1^.Color, item2^.Color);
-   sbText: Result := CompareText(item1^.Text, item2^.Text);
-   sbCustom: Result := FOnCompare(AItem1, AItem2);
- end;
- if FSortDir = sdDescending then Result := -Result;
+  case FSortBy of
+    sbX:
+      if FSortIndex = 0 then
+        Result := CompareFloat(item1^.X, item2^.X)
+      else
+      if FSortIndex < FXCount then begin
+        if FSortIndex <= Cardinal(Length(item1^.XList)) then
+          d1 := item1^.XList[FSortIndex - 1]
+        else
+          d1 := SafeNan;
+        if FSortIndex <= Cardinal(Length(item2^.XList)) then
+          d2 := item2^.XList[FSortIndex - 1]
+        else
+          d2 := SafeNan;
+        Result := CompareFloat(d1, d2);
+      end else
+        Result := 0;
+    sbY:
+      if FSortIndex = 0 then
+        Result := CompareFloat(item1^.Y, item2^.Y)
+      else
+      if FSortIndex < FYCount then begin
+        if FSortIndex <= Cardinal(Length(item1^.YList)) then
+          d1 := item1^.YList[FSortIndex - 1]
+        else
+          d1 := SafeNan;
+        if FSortIndex <= Cardinal(Length(item2^.YList)) then
+          d2 := item2^.YList[FSortIndex - 1]
+        else
+          d2 := SafeNan;
+        Result := CompareFloat(d1, d2);
+      end else
+        Result := 0;
+    sbColor:
+      Result := CompareValue(item1^.Color, item2^.Color);
+    sbText:
+      Result := CompareText(item1^.Text, item2^.Text);
+    sbCustom:
+      if Assigned(FOnCompare) then
+        Result := FOnCompare(AItem1, AItem2)
+      else
+        Result := 0;
+  end;
+  if FSortDir = sdDescending then Result := -Result;
 end;
 
-function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
-begin
-  Result := FCompareProc(AItem1, AItem2);
-end;
+{ Built-in sorting algorithm of the ChartSource - a QuickSort algorithm, copied
+  from the Classes unit and modified. Modifications are:
+  - uses a DoCompare() virtual method for comparisons,
+  - does NOT exchange equal items - this would have some side effect here: let's
+    consider sorting by X, in the ascending order, for the following data points:
+      X=3, Text='ccc'
+      X=2, Text='bbb 1'
+      X=2, Text='bbb 2'
+      X=2, Text='bbb 3'
+      X=1, Text='aaa'
 
-{ Built-in sorting algorithm of the ChartSource, a standard QuickSort.
-  Copied from the classes unit because the compare function must be a method. }
-procedure TCustomSortedChartSource.ExecSort(ACompare: TChartSortCompare);
+    after sorting, data would be (note the reversed 'bbb' order):
+      X=1, Text='aaa'
+      X=2, Text='bbb 3'
+      X=2, Text='bbb 2'
+      X=2, Text='bbb 1'
+      X=3, Text='ccc'
 
+    after sorting AGAIN, data would be (note the original 'bbb' order):
+      X=1, Text='aaa'
+      X=2, Text='bbb 1'
+      X=2, Text='bbb 2'
+      X=2, Text='bbb 3'
+      X=3, Text='ccc'
+}
+procedure TCustomSortedChartSource.DoSort;
+
   procedure QuickSort(L, R: Longint);
   var
     I, J: Longint;
     P, Q: Pointer;
   begin
-   repeat
-     I := L;
-     J := R;
-     P := FData.List^[(L + R) div 2];
-     repeat
-       while ACompare(P, FData.List^[I]) > 0 do
-         I := I + 1;
-       while ACompare(P, FData.List^[J]) < 0 do
-         J := J - 1;
-       If I <= J then
-       begin
-         Q := FData.List^[I];
-         FData.List^[I] := FData.List^[J];
-         FData.List^[J] := Q;
-         I := I + 1;
-         J := J - 1;
-       end;
-     until I > J;
-     if J - L < R - I then
-     begin
-       if L < J then
-         QuickSort(L, J);
-       L := I;
-     end
-     else
-     begin
-       if I < R then
-         QuickSort(I, R);
-       R := J;
-     end;
-   until L >= R;
+    repeat
+      I := L;
+      J := R;
+      P := FData.List^[(L + R) div 2];
+      repeat
+        while DoCompare(P, FData.List^[I]) > 0 do
+          I := I + 1;
+        while DoCompare(P, FData.List^[J]) < 0 do
+          J := J - 1;
+        if I <= J then
+        begin
+          // do NOT exchange equal items
+          if DoCompare(FData.List^[I], FData.List^[J]) <> 0 then begin
+            Q := FData.List^[I];
+            FData.List^[I] := FData.List^[J];
+            FData.List^[J] := Q;
+          end;
+          I := I + 1;
+          J := J - 1;
+        end;
+      until I > J;
+      if J - L < R - I then
+      begin
+        if L < J then
+          QuickSort(L, J);
+        L := I;
+      end
+      else
+      begin
+        if I < R then
+          QuickSort(I, R);
+        R := J;
+      end;
+    until L >= R;
   end;
 
 begin
   if FData.Count < 2 then exit;
-  QuickSort(0, FData.Count-1);
+  QuickSort(0, FData.Count - 1);
 end;
 
 function TCustomSortedChartSource.GetCount: Integer;
@@ -1673,6 +1748,78 @@
   Result := PChartDataItem(FData.Items[AIndex]);
 end;
 
+function TCustomSortedChartSource.ItemAdd(AItem: PChartDataItem): Integer;
+begin
+  if IsSorted then begin
+    Result := ItemFind(AItem);
+    FData.Insert(Result, AItem);
+  end else
+    Result := FData.Add(AItem);
+end;
+
+procedure TCustomSortedChartSource.ItemInsert(AIndex: Integer; AItem: PChartDataItem);
+begin
+  if IsSorted then
+    if AIndex <> ItemFind(AItem) then
+      raise ESortError.CreateFmt('%0:s.ItemInsert cannot insert data at the requested '+
+        'position, because source is sorted', [ClassName]);
+  FData.Insert(AIndex, AItem);
+end;
+
+function TCustomSortedChartSource.ItemFind(AItem: PChartDataItem; L: Integer = 0; R: Integer = High(Integer)): Integer;
+var
+  I: Integer;
+begin
+  if not IsSorted then
+    raise ESortError.CreateFmt('%0:s.ItemFind can be called only for sorted source', [ClassName]);
+
+  if R >= FData.Count then
+    R := FData.Count - 1;
+
+  // special optimization for adding sorted data at the end
+  if R >= 0 then
+    if DoCompare(FData.List^[R], AItem) <= 0 then
+      exit(R + 1);
+
+  // use binary search
+  if L < 0 then
+    L := 0;
+  while L <= R do
+  begin
+    I := L + (R - L) div 2;
+    if DoCompare(FData.List^[I], AItem) <= 0 then
+      L := I + 1
+    else
+      R := I - 1;
+  end;
+  Result := L;
+end;
+
+function TCustomSortedChartSource.ItemModified(AIndex: Integer): Integer;
+begin
+  Result := AIndex;
+  if IsSorted then begin
+    if FData.Count < 2 then exit;
+    if (AIndex < 0) or (AIndex >= FData.Count) then exit;
+
+    if AIndex > 0 then
+      if DoCompare(FData.List^[AIndex - 1], FData.List^[AIndex]) > 0 then begin
+        Result := ItemFind(FData.List^[AIndex], 0, AIndex - 1);
+        // no Dec(Result) here, as it is below
+        FData.Move(AIndex, Result);
+        exit; // optimization: the item cannot be unsorted from both sides
+              // simultaneously, so we can exit now
+      end;
+
+    if AIndex < FData.Count - 1 then
+      if DoCompare(FData.List^[AIndex], FData.List^[AIndex + 1]) > 0 then begin
+        Result := ItemFind(FData.List^[AIndex], AIndex + 1, FData.Count - 1);
+        Dec(Result);
+        FData.Move(AIndex, Result);
+      end;
+  end;
+end;
+
 function TCustomSortedChartSource.IsSorted: Boolean;
 begin
   case FSortBy of
@@ -1737,11 +1884,7 @@
     FSorted := SaveSorted;
   end;
 
-  if FSortBy = sbCustom then
-    FCompareProc := FOnCompare
-  else
-    FCompareProc := @DefaultCompare;
-  ExecSort(@DoCompare);
+  DoSort;
   Notify;
 end;
 
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 61227)
+++ components/tachart/tasources.pas	(working copy)
@@ -27,8 +27,6 @@
     FDataPoints: TStrings;
     FXCountMin: Cardinal;
     FYCountMin: Cardinal;
-    procedure AddAt(
-      APos: Integer; const AX, AY: Double; const ALabel: String; AColor: TChartColor);
     procedure ClearCaches;
     function NewItem: PChartDataItem;
     procedure SetDataPoints(const AValue: TStrings);
@@ -57,12 +55,12 @@
     procedure Clear;
     procedure CopyFrom(ASource: TCustomChartSource);
     procedure Delete(AIndex: Integer);
-    procedure SetColor(AIndex: Integer; AColor: TChartColor);
-    procedure SetText(AIndex: Integer; const AValue: String);
-    procedure SetXList(AIndex: Integer; const AXList: array of Double);
+    function SetColor(AIndex: Integer; AColor: TChartColor): Integer;
+    function SetText(AIndex: Integer; const AValue: String): Integer;
+    function SetXList(AIndex: Integer; const AXList: array of Double): Integer;
     function SetXValue(AIndex: Integer; const AValue: Double): Integer;
-    procedure SetYList(AIndex: Integer; const AYList: array of Double);
-    procedure SetYValue(AIndex: Integer; const AValue: Double);
+    function SetYList(AIndex: Integer; const AYList: array of Double): Integer;
+    function SetYValue(AIndex: Integer; const AValue: Double): Integer;
   published
     property DataPoints: TStrings read FDataPoints write SetDataPoints;
     property XCount;
@@ -373,7 +371,7 @@
   item := FSource.NewItem;
   try
     Parse(S, item);
-    FSource.FData.Insert(Index, item);
+    FSource.ItemInsert(Index, item);
   except
     Dispose(item);
     raise;
@@ -489,30 +487,20 @@
 function TListChartSource.Add(
   const AX, AY: Double; const ALabel: String = '';
   const AColor: TChartColor = clTAColor): Integer;
-begin
-  Result := FData.Count;
-  if IsSortedByXAsc then
-    // Keep data points ordered by X coordinate.
-    // Note that this leads to O(N^2) time except
-    // for the case of adding already ordered points.
-    // So, is the user wants to add many (>10000) points to a graph,
-    // he should pre-sort them to avoid performance penalty.
-    while (Result > 0) and (Item[Result - 1]^.X > AX) do
-      Dec(Result);
-  AddAt(Result, AX, AY, ALabel, AColor);
-end;
-
-procedure TListChartSource.AddAt(
-  APos: Integer; const AX, AY: Double; const ALabel: String; AColor: TChartColor);
 var
   pcd: PChartDataItem;
 begin
   pcd := NewItem;
-  pcd^.X := AX;
-  pcd^.Y := AY;
-  pcd^.Color := AColor;
-  pcd^.Text := ALabel;
-  FData.Insert(APos, pcd);
+  try
+    pcd^.X := AX;
+    pcd^.Y := AY;
+    pcd^.Color := AColor;
+    pcd^.Text := ALabel;
+    Result := ItemAdd(pcd);
+  except
+    Dispose(pcd);
+    raise;
+  end;
   UpdateCachesAfterAdd(AX, AY);
 end;
 
@@ -530,9 +518,9 @@
   try
     Result := Add(AX[0], AY[0], ALabel, AColor);
     if Length(AX) > 1 then
-      SetXList(Result, AX[1..High(AX)]);
+      Result := SetXList(Result, AX[1..High(AX)]);
     if Length(AY) > 1 then
-      SetYList(Result, AY[1..High(AY)]);
+      Result := SetYList(Result, AY[1..High(AY)]);
   finally
     Dec(FUpdateCount);
   end;
@@ -552,7 +540,7 @@
   try
     Result := Add(AX, AY[0], ALabel, AColor);
     if Length(AY) > 1 then
-      SetYList(Result, AY[1..High(AY)]);
+      Result := SetYList(Result, AY[1..High(AY)]);
   finally
     Dec(FUpdateCount);
   end;
@@ -591,6 +579,7 @@
 procedure TListChartSource.CopyFrom(ASource: TCustomChartSource);
 var
   i: Integer;
+  pcd: PChartDataItem;
 begin
   if ASource.XCount < FXCountMin then
     raise EXCountError.CreateFmt(rsSourceCountError2, [ClassName, FXCountMin, 'x']);
@@ -602,23 +591,23 @@
     Clear;
     XCount := ASource.XCount;
     YCount := ASource.YCount;
-    for i := 0 to ASource.Count - 1 do
-      with ASource[i]^ do begin
-        AddAt(FData.Count, X, Y, Text, Color);
-        SetXList(FData.Count - 1, XList);
-        SetYList(FData.Count - 1, YList);
+    FData.Capacity := ASource.Count;
+
+    pcd := nil;
+    try // optimization: don't execute try..except..end in a loop
+      for i := 0 to ASource.Count - 1 do begin
+        pcd := NewItem;
+        pcd^ := ASource[i]^;
+        FData.Add(pcd); // don't use ItemAdd() here
+        pcd := nil;
       end;
+    except
+      if pcd <> nil then
+        Dispose(pcd);
+      raise;
+    end;
 
-    if IsSorted then begin
-      if ASource.IsSorted and
-        (SortBy = TCustomChartSourceAccess(ASource).SortBy) and
-        (SortDir = TCustomChartSourceAccess(ASource).SortDir) and
-        (SortIndex = TCustomChartSourceAccess(ASource).SortIndex) and
-        (SortBy <> sbCustom)
-      then
-        exit;
-      Sort;
-    end;
+    if IsSorted and (not HasSameSorting(ASource)) then Sort;
   finally
     EndUpdate;
   end;
@@ -680,12 +669,13 @@
   if YCount > 1 then SetLength(Result^.YList, YCount - 1);
 end;
 
-procedure TListChartSource.SetColor(AIndex: Integer; AColor: TChartColor);
+function TListChartSource.SetColor(AIndex: Integer; AColor: TChartColor): Integer;
 begin
   with Item[AIndex]^ do begin
-    if Color = AColor then exit;
+    if Color = AColor then exit(AIndex);
     Color := AColor;
   end;
+  Result := ItemModified(AIndex);
   Notify;
 end;
 
@@ -701,12 +691,13 @@
   end;
 end;
 
-procedure TListChartSource.SetText(AIndex: Integer; const AValue: String);
+function TListChartSource.SetText(AIndex: Integer; const AValue: String): Integer;
 begin
   with Item[AIndex]^ do begin
-    if Text = AValue then exit;
+    if Text = AValue then exit(AIndex);
     Text := AValue;
   end;
+  Result := ItemModified(AIndex);
   Notify;
 end;
 
@@ -725,8 +716,8 @@
   Notify;
 end;
 
-procedure TListChartSource.SetXList(
-  AIndex: Integer; const AXList: array of Double);
+function TListChartSource.SetXList(
+  AIndex: Integer; const AXList: array of Double): Integer;
 var
   i: Integer;
 begin
@@ -734,6 +725,7 @@
     for i := 0 to Min(High(AXList), High(XList)) do
       XList[i] := AXList[i];
   FXListExtentIsValid := false;
+  Result := ItemModified(AIndex);
   Notify;
 end;
 
@@ -757,26 +749,13 @@
   end;
 
 begin
-  if IsSortedByXAsc then
-    if IsNan(AValue) then
-      raise EChartError.CreateFmt('X = NaN in sorted source %s', [NameOrClassName(Self)]);
-  Result := AIndex;
   with Item[AIndex]^ do begin
-    if IsEquivalent(X, AValue) then exit; // IsEquivalent() can compare also NaNs
+    if IsEquivalent(X, AValue) then exit(AIndex); // IsEquivalent() can compare also NaNs
     oldX := X;
     X := AValue;
   end;
   UpdateExtent;
-  if IsSortedByXAsc then begin
-    if AValue > oldX then
-      while (Result < Count - 1) and (Item[Result + 1]^.X < AValue) do
-        Inc(Result)
-    else
-      while (Result > 0) and (Item[Result - 1]^.X > AValue) do
-        Dec(Result);
-    if Result <> AIndex then
-      FData.Move(AIndex, Result);
-  end;
+  Result := ItemModified(AIndex);
   Notify;
 end;
 
@@ -795,8 +774,8 @@
   Notify;
 end;
 
-procedure TListChartSource.SetYList(
-  AIndex: Integer; const AYList: array of Double);
+function TListChartSource.SetYList(
+  AIndex: Integer; const AYList: array of Double): Integer;
 var
   i: Integer;
 begin
@@ -805,10 +784,11 @@
       YList[i] := AYList[i];
   FCumulativeExtentIsValid := false;
   FYListExtentIsValid := false;
+  Result := ItemModified(AIndex);
   Notify;
 end;
 
-procedure TListChartSource.SetYValue(AIndex: Integer; const AValue: Double);
+function TListChartSource.SetYValue(AIndex: Integer; const AValue: Double): Integer;
 var
   oldY: Double;
 
@@ -829,7 +809,7 @@
 
 begin
   with Item[AIndex]^ do begin
-    if IsEquivalent(Y, AValue) then exit; // IsEquivalent() can compare also NaNs
+    if IsEquivalent(Y, AValue) then exit(AIndex); // IsEquivalent() can compare also NaNs
     oldY := Y;
     Y := AValue;
   end;
@@ -836,6 +816,7 @@
   if FValuesTotalIsValid then
     FValuesTotal += NumberOr(AValue) - NumberOr(oldY);
   UpdateExtent;
+  Result := ItemModified(AIndex);
   Notify;
 end;
 
phase3.diff (18,275 bytes)

Marcin Wiazowski

2019-05-15 19:40

reporter   ~0116211

I'm attaching last of the patches (phase4.diff - it's based on r61228, so requires also phase3.diff to be applied) - it contains a TSortedChartSource = class(TCustomSortedChartSource) implementation.

Such a name was chosen for two reasons:

1) It's a common convention in TAChart to remove the "Custom" phrase, when implementing a "final" class:

  TChartAxisMarks = class(TCustomChartAxisMarks)
  TDrawFuncHelper = class(TCustomDrawFuncHelper)
  TChartSeries = class(TCustomChartSeries)
  TFuncSeries = class(TCustomFuncSeries)
  TChartMarks = class(TCustomChartMarks)
  TPieSeries = class(TCustomPieSeries)

2) Such a name is also coherent with TCalculatedChartSource or TRandomChartSource names - we have just "Sorted" instead of "Calculated" or "Random" here.



Implementation is mostly based on TCalculatedChartSource. It's not very complicated, so I'll describe only some non-obvious aspects here:

A) FData is now used NOT to hold PChartDataItem pointers, as usual - but PInteger pointers. Each integer is an index to Origin's data - initially, they have values from 0 to Count-1 (this is initialized in TSortedChartSource.ResetTransformation()), so TSortedChartSource behaves just transparently. The DoCompare() and GetItem() methods are overridden, to convert these indices into true Origin's data.

B) TSortedChartSource creates not only a listener monitoring Origin's changes, but also a listener monitoring changes in Self. This is needed to return to transparent state, when sorting becomes off - ResetTransformation() must be called, otherwise last sorting settings would be still effective.

C) Sorting does not alter extents, so they are read directly from the Origin. To implement this, TCustomChartSource.BasicExtent() - maybe surprisingly - needs to become virtual. Otherwise, TSortedChartSource would need to maintain basic extent on its own - this would mean making same work twice, once in the Origin, and second time in TSortedChartSource - which doesn't make much sense.



I'm attaching a one-more-time modified test application (Test-component.zip), which contains the following changes:
- it uses TSortedChartSource component now,
- pressing the button many times makes TSortedChartSource not only sorted, but also transparent again.

phase4.diff (7,500 bytes)
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 61228)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -236,7 +236,7 @@
     procedure EndUpdate; override;
   public
     class procedure CheckFormat(const AFormat: String);
-    function BasicExtent: TDoubleRect;
+    function BasicExtent: TDoubleRect; virtual;
     function Extent: TDoubleRect; virtual;
     function ExtentCumulative: TDoubleRect; virtual;
     function ExtentList: TDoubleRect; virtual;
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 61228)
+++ components/tachart/tasources.pas	(working copy)
@@ -77,6 +77,41 @@
     property OnCompare;
   end;
 
+  { TSortedChartSource }
+
+  TSortedChartSource = class(TCustomSortedChartSource)
+  strict private
+    FListener: TListener;
+    FListenerSelf: TListener;
+    FOrigin: TCustomChartSource;
+    procedure Changed(ASender: TObject);
+    procedure SetOrigin(AValue: TCustomChartSource);
+  protected
+    function DoCompare(AItem1, AItem2: Pointer): Integer; override;
+    function GetCount: Integer; override;
+    function GetItem(AIndex: Integer): PChartDataItem; override;
+    procedure ResetTransformation(ACount: Integer);
+    procedure SetXCount(AValue: Cardinal); override;
+    procedure SetYCount(AValue: Cardinal); override;
+  public
+    constructor Create(AOwner: TComponent); override;
+    destructor Destroy; override;
+  published
+    function BasicExtent: TDoubleRect; override;
+    function Extent: TDoubleRect; override;
+    function ExtentCumulative: TDoubleRect; override;
+    function ExtentList: TDoubleRect; override;
+    function ExtentXYList: TDoubleRect; override;
+    function ValuesTotal: Double; override;
+    property Origin: TCustomChartSource read FOrigin write SetOrigin;
+    // Sorting
+    property SortBy;
+    property SortDir;
+    property Sorted;
+    property SortIndex;
+    property OnCompare;
+  end;
+  
   { TMWCRandomGenerator }
 
   // Mutliply-with-carry random number generator.
@@ -278,12 +313,11 @@
 begin
   RegisterComponents(
     CHART_COMPONENT_IDE_PAGE, [
-      TListChartSource, TRandomChartSource, TUserDefinedChartSource,
-      TCalculatedChartSource
+      TListChartSource, TSortedChartSource, TRandomChartSource,
+      TUserDefinedChartSource, TCalculatedChartSource
     ]);
 end;
 
-
 { TListChartSourceStrings }
 
 procedure TListChartSourceStrings.Clear;
@@ -483,7 +517,6 @@
     end;
 end;
 
-
 { TListChartSource }
 
 function TListChartSource.Add(
@@ -860,6 +893,185 @@
   (FDataPoints as TListChartSourceStrings).LoadingFinished;
 end;
 
+{ TSortedChartSource }
+
+constructor TSortedChartSource.Create(AOwner: TComponent);
+begin
+  inherited Create(AOwner);
+  FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
+  FYCount := MaxInt;
+  FListener := TListener.Create(@FOrigin, @Changed);
+  FListenerSelf := TListener.Create(nil, @Changed);
+  Broadcaster.Subscribe(FListenerSelf);
+end;
+
+destructor TSortedChartSource.Destroy;
+begin
+  ResetTransformation(0);
+  FreeAndNil(FListenerSelf);
+  FreeAndNil(FListener);
+  inherited;
+end;
+
+procedure TSortedChartSource.Changed(ASender: TObject);
+begin
+  if ASender = Self then begin
+    // We can get here only due to FListenerSelf's notification.
+    // If some of our own (not Origin's) sorting properties was changed and we
+    // are sorted, then our Sort() method has been called, so the transformation
+    // is valid; but if we are no longer sorted, only notification is sent (so
+    // we are here), so we must reinitialize the transformation to return to
+    // the transparent (i.e. unsorted) state.
+    if not IsSorted then
+      ResetTransformation(Count);
+    exit;
+  end;
+
+  if FOrigin <> nil then begin
+    FXCount := Origin.XCount;
+    FYCount := Origin.YCount;
+    ResetTransformation(Origin.Count);
+    if IsSorted and (not HasSameSorting(Origin)) then Sort else Notify;
+  end else begin
+    FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
+    FYCount := MaxInt;
+    ResetTransformation(0);
+    Notify;
+  end;
+end;
+
+function TSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
+begin
+  Result := inherited DoCompare(Origin.Item[PInteger(AItem1)^],
+                                Origin.Item[PInteger(AItem2)^]);
+end;
+
+function TSortedChartSource.GetCount: Integer;
+begin
+  if Origin <> nil then
+    Result := Origin.Count
+  else
+    Result := 0;
+end;
+
+function TSortedChartSource.GetItem(AIndex: Integer): PChartDataItem;
+begin
+  if Origin <> nil then
+    Result := PChartDataItem(Origin.Item[PInteger(FData.Items[AIndex])^])
+  else
+    Result := nil;
+end;
+
+procedure TSortedChartSource.ResetTransformation(ACount: Integer);
+var
+  i: Integer;
+  pint: PInteger;
+begin
+  if ACount > FData.Count then begin
+    for i := 0 to FData.Count - 1 do
+      PInteger(FData.List^[i])^ := i;
+
+    FData.Capacity := ACount;
+
+    pint := nil;
+    try // optimization: don't execute try..except..end in a loop
+      for i := FData.Count to ACount - 1 do begin
+        New(pint);
+        pint^ := i;
+        FData.Add(pint); // don't use ItemAdd() here
+        pint := nil;
+      end;
+    except
+      if pint <> nil then
+        Dispose(pint);
+      raise;
+    end;
+  end else
+  begin
+    for i := ACount to FData.Count - 1 do
+      Dispose(PInteger(FData.List^[i]));
+
+    FData.Count := ACount;
+    FData.Capacity := ACount; // release needless memory
+
+    for i := 0 to FData.Count - 1 do
+      PInteger(FData.List^[i])^ := i;
+  end;
+end;
+
+procedure TSortedChartSource.SetOrigin(AValue: TCustomChartSource);
+begin
+  if AValue = Self then
+    AValue := nil;
+  if FOrigin = AValue then exit;
+  if FOrigin <> nil then
+    FOrigin.Broadcaster.Unsubscribe(FListener);
+  FOrigin := AValue;
+  if FOrigin <> nil then
+    FOrigin.Broadcaster.Subscribe(FListener);
+  Changed(nil);
+end;
+
+procedure TSortedChartSource.SetXCount(AValue: Cardinal);
+begin
+  Unused(AValue);
+  raise EXCountError.Create('Cannot set XCount');
+end;
+
+procedure TSortedChartSource.SetYCount(AValue: Cardinal);
+begin
+  Unused(AValue);
+  raise EYCountError.Create('Cannot set YCount');
+end;
+
+function TSortedChartSource.BasicExtent: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.BasicExtent;
+end;
+
+function TSortedChartSource.Extent: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.Extent;
+end;
+
+function TSortedChartSource.ExtentCumulative: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.ExtentCumulative;
+end;
+
+function TSortedChartSource.ExtentList: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.ExtentList;
+end;
+
+function TSortedChartSource.ExtentXYList: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.ExtentXYList;
+end;
+
+function TSortedChartSource.ValuesTotal: Double;
+begin
+  if Origin = nil then
+    Result := 0
+  else
+    Result := Origin.ValuesTotal;
+end;
+
 { TMWCRandomGenerator }
 
 function TMWCRandomGenerator.Get: LongWord;
phase4.diff (7,500 bytes)
Test-component.zip (2,886 bytes)

wp

2019-05-16 10:07

developer   ~0116216

Last edited: 2019-05-16 10:08

View 2 revisions

I am hesistant to apply these patches because I more and more get the feeling that sorting is a very expensive feature. I wonder if I ever used sorting in any chart with real data, even sorting by x. There is a small benefit - the bubble series example drawn from back to front, yes, but I never had such a case in practice. Sorting for series having no x value - again, not a practical example for me. On the other side, every chart using a ListChartSource now carries the sorting code although it most probably will not use it. And you had to touch many method in the central TACustomSource and TASources units which increases the chance of creating regressions.

Here is another idea which I got when seeing the phase3+4 patches:
- We revert all the changes in TCustomChartSource and TListSource to the pre-sorting state, i.e. remove the SortBy, SortDir etc properties. TListChartSource inherits from TCustomChartSource again.
- We add a TSortedChartSource similar to what you did in the 4th step. The difference is: It should be applicable to TListChartSource as well.
- As a consequence, the user has sorting capabilities, but adds the related code only when it is needed. Since the changes are more local the chance of regressions is greatly reduced.

Of course this picture is too simple. Still some adaptions will have to be made in TCustomChartSource and TListChartSource. For example: A new notification mechanism must be established to trigger finding the new position when an item is changed (your "ItemModified"). There is also the question what to do with the current IsSorted functions when no SortedChartSource is attached to a series? Keep them to indicate that the data points are sorted for Extent calculation, but without any other sorting functionality (similar to current UserDefinedChartSource)? Or sorting by X in the List source: Probably it should remain, but get a FindItem function for finding the correct index when a new data point is added into a sorted list source?

Maybe all these additions add up to all the code which is contained in the phase3+4 patches. In this case, this idea would be useless, of course, and I'll apply phase3+4 immediately.
---------------
Putting this new concept aside, another remark:
Why do you stuff so many cases into the Compare function? This function is called many many times and should be as efficient as possible. This is why an appropriate Compare function can be passed to the Sort method as a procedure parameter. Even my own example containing a "case of" all SortBy items in the same function was too much (I was just too lazy...). Additionally: the case that the SortIndex is less than the List.Count can be handled outside the Compare function.

Marcin Wiazowski

2019-05-19 23:59

reporter   ~0116266

I made some additional research, and here come observations and conclusions. I'll split them into several posts.



Let's start from:

> Why do you stuff so many cases into the Compare function? This function is called many many times and should be as efficient as possible. This is why an appropriate Compare function can be passed to the Sort method as a procedure parameter. Even my own example containing a "case of" all SortBy items in the same function was too much (I was just too lazy...). Additionally: the case that the SortIndex is less than the List.Count can be handled outside the Compare function.



Well, I was aware of your intention here, and I even considered some more complicated optimization - creating a set of comparing functions, which could be assigned to FCompareProc:
- SortByXAsc,
- SortByXDesc,
- SortByXListAsc,
- SortByXListDesc,
- SortByYAsc,
- SortByYDesc,
- SortByYListAsc,
- SortByYListDesc,
- SortByColorAsc,
- SortByColorDesc,
- SortByTextAsc,
- SortByTextDesc,
- SortByCustomAsc,
- SortByCustomDesc,
- SortByNothing (for invalid sorting parameters - this function would always return 0).

I planned to create an UpdateCompareProc() method, which would be called by all the sorting property setters.



But, (un)fortunately, I finally found, that this gives no measurable improvement. Let's make some quick test: replace TCustomSortedChartSource.DoCompare() just with:

  function TCustomSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
  var
    item1: PChartDataItem absolute AItem1;
    item2: PChartDataItem absolute AItem2;
  begin
    Result := CompareFloat(item1^.X, item2^.X);
  end;

an try the SpeedTest application. At least in my case, I couldn't see any measurable improvement.



My final decision of placing all the stuff in DoCompare() was because:

1) There is no measurable time difference, as the experiment above shows.

2) There is no need to have the UpdateCompareProc() method.

3) DoCompare() is a protected method - thanks to placing all the stuff inside, it cannot fail (i.e. cannot generate invalid result or out-of-range memory access), so it can be called from derived classes without performing any additional validation and/or initialization.

4) When comparing with the current (i.e. r61245) code, there are two main differences:

a) For sbCustom case, an additional comparison is added ("if Assigned(FOnCompare) then") - this gives only one additional integer comparison.

b) For sbX and sbY cases, in r61245, there are GetX() / GetY() calls, that perform range checks internally (but return data from other-than-requested XList/YList index in case of range failure). Moving these checks to DoCompare():

- gives faster execution for the most common FSortIndex = 0 case (only "if FSortIndex = 0 then" is executed),

- allows to handle out-of-range SortIndex in a proper way, i.e. NaNs are used, instead of reading data from XList/YList index.



Interestingly, there is a potential for improvement in some other place:

  function CompareFloat(const x1, x2: Double): Integer;
  begin
    if IsNaN(x1) and IsNaN(x2) then
      Result := 0
    else if IsNaN(x1) then
      Result := +1
    else if IsNaN(x2) then
      Result := -1
    else
      Result := CompareValue(x1, x2);
  end;

As we can see here, for most common cases - i.e. when both compared values are not NaNs - IsNaN(x1) is called twice, and IsNaN(x2) is also called twice. Maybe these calls are not highly complicated, but CompareFloat() is a time-critical function, and the issue can be fixed just for free. So I'm attaching phase5.diff, which solves the problem.

phase5.diff (640 bytes)
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 61245)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -1586,11 +1586,13 @@
 
 function CompareFloat(const x1, x2: Double): Integer;
 begin
-  if IsNaN(x1) and IsNaN(x2) then
-    Result := 0
-  else if IsNaN(x1) then
-    Result := +1
-  else if IsNaN(x2) then
+  if IsNaN(x1) then begin
+    if IsNaN(x2) then
+      Result := 0
+    else
+      Result := +1;
+  end else
+  if IsNaN(x2) then
     Result := -1
   else
     Result := CompareValue(x1, x2);
phase5.diff (640 bytes)

Marcin Wiazowski

2019-05-20 00:00

reporter   ~0116267

Let's discuss moving sorting functionality to some another, helper class.


The main difference between sorting in TListChartSource, and sorting by using the new, intermediate TSortChartSource instance, is: TListChartSource is much faster, because it can insert new points - or move the modified points - directly to right places; this is good, because TListChartSource is a base data storage for most of the series classes, so it must be efficient. On the contrary, TSortChartSource can only react to "change" notifications, and the whole sorting algorithm must be executed after adding/modifying each point.


Creating a separate sorting class, that could be used only when the user needs sorting, is an interesting idea - although there is some thing to note: TSortChartSource leaves its Origin's data untouched, while TListChartSource needs its data points to be physically exchanged. This means, that the helper object, attached to TListChartSource, must work on TListChartSource's FData list, instead of using its own internal list of integers - so this cannot be just a helper TSortChartSource instance. So, in fact, this sounds just like splitting TListChartSource into two pieces - the basic one and the one with sorting implementation. Expected advantage: less code in the executable file, when sorting is not used.


Now bad news: unfortunately, no code can be removed in practice, even when the user himself doesn't reference sorting in any way. Setting Sorted property to True must make the data sorted, so SetSorted() implementation must reference the sorting code - i.e. must call the Sort() method, as it is now - or must create a helper object. So - as long, as Sorted property is implemented - sorting code will be attached, because the compiler can see a reference to the sorting code in the SetSorted() method. And Sorted property is published, so SetSorted() is always included in the compiled code - even if Sorted is not referenced directly, LFM stream loader must be able to use Sorted, if such a a request is placed in the stream contents.


So sorting code cannot be removed from the compiled executable in any way, unfortunately.

Marcin Wiazowski

2019-05-20 00:02

reporter   ~0116268

To evaluate changes between the current 2.0.2 Lazarus release, and the state after applying all the phaseX.diffs, I created a list of changes (I omitted some most trivial changes - like adding "const" to string parameters, or reordering methods alphabetically):


C1: In TCustomChartSource, function IsSortedByXAsc() is added, and some calls to Sorted/IsSorted() were replaced with calls to IsSortedByXAsc().


C2: In TCustomChartSource, SortBy, SortDir, SortIndex properties are added, along with their setters (setters only raise an exception, unless overridden).


C3: In TCustomChartSource.BasicExtent(), some optimizations for sorted sources are added.


C4: In TCustomChartSource, function HasSameSorting() is added.


C5: The following fields, methods and properties are just moved from TListChartSource to the newly implemented TCustomSortedChartSource (some with modifications - but see below):
- FData,
- FSorted,
- procedure SetSorted(),
- function GetCount(),
- function GetItem(),
- function IsSorted(),
- procedure Sort(),
- property Sorted.


C6: In TCustomSortedChartSource.SetSorted(), a Notify() call is added for not-sorted cases.


C7: In TCustomSortedChartSource.IsSorted(), an additional validation of the new sorting options is added to see, if sorting can really be performed. As a consequence, all reads of the "Sorted" property are now changed to calls to "IsSorted" function.


C8: In TCustomSortedChartSource.Sorted() method, some changes are made:
a) for comparisons, a trivial CompareDataItemX() function is removed, and a more universal TCustomSortedChartSource.DoCompare() method is used,
b) instead of using FData.Sort() for sorting, TCustomSortedChartSource.DoSort() method is now used - the difference is that DoSort() fixes a problem with exchanging equal items (which should be avoided).


C9: In TCustomSortedChartSource, additional OnCompare property is implemented, along with its setter.


C10: In TCustomSortedChartSource, Sort() method is now just a simple wrapper around the DoSort() method.


C11: In TCustomSortedChartSource, SetSortBy(), SetSortDir() and SetSortIndex() are overridden, so they call Sort() or Notify() when needed.


C12: TCustomSortedChartSource implements four new methods:
a) ItemAdd() - if not sorted, just calls FData.Add() - otherwise calls ItemFind() to find a new position first,
b) ItemInsert() - if not sorted, just calls FData.Insert() - otherwise calls ItemFind() to verify a new position first,
c) ItemFind() - new feature: implements binary search (with additional optimization), to efficiently find a new position in the data set,
d) ItemModified() - if not sorted, just exits, returning the current item position - otherwise, if needed, calls ItemFind() to find a new position.


C13: In TListChartSource, now all the following methods are functions (they return the new item position):
- SetColor(),
- SetText(),
- SetXList(),
- SetXValue(),
- SetYList(),
- SetYValue().
As a consequence, their results are now used in TListChartSource.AddXListYList() and TListChartSource.AddXYList() implementations.


C14: The functions listed in C13 make now calls to ItemModified(), to support sorting. As a consequence, slow sequential search implementation is removed from SetXValue().


C15: In TListChartSourceStrings.Insert(), ItemInsert() is called instead of FData.Insert(), to support sorting.


C16: In TListChartSource.Add(), ItemAdd() is called instead of FData.Insert(), to support sorting - so slow sequential search implementation is removed from Add().


C17: In TListChartSource.Add(), lacking try..except..end statement is added.


C18: In TListChartSource.CopyFrom(), copying is performed in a more efficient way.


C19: In TListChartSource.CopyFrom(), "HasSameSorting(ASource)" is called instead of just "ASource.IsSorted".


C20: In TListChartSource.SetXValue(), no-more-needed check for NaN values is removed.


C21: A completely new component - TSortedChartSource - is added.

Marcin Wiazowski

2019-05-20 00:04

reporter   ~0116269

Let's evaluate cost of changes (in the sense of execution time) for NOT SORTED TListChartSource objects:

C1: No significant impact: IsSortedByXAsc() executes only IsSorted() in this case, and exits - which is almost same as calling IsSorted() directly.

C2: No impact at all: these methods are overridden and are never called.
 
C3: Tiny impact: a new call to IsSorted() is made, but no other new code is executed.

C4: No impact at all: adding the HasSameSorting() method does not itself change anything - and its usage will be considered below.

C5: No impact at all: no code is modified.

C6: No impact at all: Sorted property is not used.

C7: No significant impact: additional "case" statement must be executed - also FSorted field must be read in this case, but this was also before; no other code is executed.

C8: No impact at all: DoCompare() and DoSort() are never called.

C9: No impact at all: OnCompare property is not used.

C10: No impact at all: Sort() is never called.

C11: No impact at all: SortBy, SortDir and SortIndex properties are not used.

C12: No impact at all: adding these methods does not change anything - and their usage will be considered below.

C13: No significant impact: just few assembler instructions are added.

C14: Tiny impact: a new call to ItemModified() is made, where IsSorted() is called - but no other new code is executed.

C15: Tiny impact: in ItemInsert(), a call to IsSorted() is made - but no other new code is executed.

C16: Tiny impact: in ItemAdd(), a call to IsSorted() is made - but no other new code is executed.

C17: Noticeable impact: as I checked separately, and as it can be seen in the comparison of execution times, in my post 0035356:0116200 above - the SpeedTest application slows down from 187 ms to 234 ms. However, adding the try..except..end clause is just a bugfix, so it should be applied in any case; similar code is already in TListChartSourceStrings.Insert(). The mentioned execution times are for one million points, so there should be no practical difference for more usual cases.

C18: Improvement for cases using XList and/or YList, and no change for other cases.

C19: No impact at all: HasSameSorting() is never called.

C20: Tiny improvement.

C21: Does not concern.



============



Let's evaluate cost of changes (in the sense of execution time) for SORTED TListChartSource objects:

C1: Tiny impact: additional three integer comparisons are made.

C2: As in C2 above.
 
C3: Significant improvement in most cases.

C4: As in C4 above.

C5: As in C5 above.

C6: Tiny impact: only an additional Notify() call is made.

C7: Tiny impact: additional "case" statement must be executed, plus one or two additional integer comparisons are made.

C8: This was discussed in the post 0035356:0116266 above.

C9: Tiny impact: assuming, that OnCompare property is set BEFORE setting Sorted to True, only Notify() call is made.

C10: This leads us basically to DoSort() - so see C8.

C11: Tiny impact: assuming, that SortBy, SortDir and SortIndex properties are set BEFORE setting Sorted to True, only Notify() calls are made.

C12: As in C12 above.

C13: As in C13 above.

C14: Only sorting in SetXValue() can be compared to previous implementation (because there was no sorting in the other methods before): there is a noticeable improvement in most cases, due to using binary search, instead of sequential search.

C15: This code is only used when loading data points from LFM stream, so introduced change makes no practical difference for such numbers of points, that can be stored in LFM.

C16: There is a noticeable improvement in most cases, due to using binary search, instead of sequential search.

C17: As in C17 above.

C18: As in C18 above.

C19: No significant impact: HasSameSorting() is called only once, and it contains only very simple code.

C20: Tiny improvement.

C21: Does not concern.



============



Conclusion: with the exception of the C17 case (which is a bugfix, so it should be applied in any case), there is tiny or no impact in most cases, and a significant improvement in some cases. Also SpeedTest application confirms this - see comparison in the post 0035356:0116200 above.

So, fortunately, the overall cost of the introduced changes - in the sense of execution time - is negative: there is a small slow-down in some cases, but significant improvement in other cases.

Marcin Wiazowski

2019-05-20 00:05

reporter   ~0116270

Let's evaluate cost of changes (in the sense of code size):

C1: Very small: IsSortedByXAsc() has a 1-line implementation.

C2: Very small: the setters have 2-line implementations.
 
C3: Very small: 6 new lines of simple code.

C4: Very small: few new lines of simple code.

C5: No cost at all: no code is modified.

C6: Very small: one function call is added.

C7: Very small: few new lines of simple code.

C8: Noticeable: DoCompare() has about 40 lines of code. DoSort() also has about 40 lines of code, but if contains a bugfix for exchanging equal items, so it should be applied in any case.

C9: Very small: the setter has 3-line implementation.

C10: Very small: 7 new lines of simple code.

C11: Very small: the setters have 4-line implementations.

C12: Noticeable: ItemAdd(), ItemInsert(), ItemFind() and ItemModified() add about 40 lines of code in total - although about 12 lines of no longer needed code is removed from TListChartSource.Add() and TListChartSource.SetXValue().

C13: Very small: just few assembler instructions are added.

C14: Very small: 6 function calls are added in total.

C15: No cost: one function call is replaced with another one.

C16: No cost: one function call is replaced with another one.

C17: Very small: an exception handler adds several assembler instructions.

C18: Very small: both old and new code have similar size.

C19: No cost: one function call is replaced with another one.

C20: Tiny improvement.

C21: Noticeable: the new TSortedChartSource class adds about 90 lines of code.



Summary: The largest code pieces are:
- DoCompare() adds about 40 lines of code,
- DoSort() adds about 40 lines of code (but it's a bugfix, so it's required),
- TSortedChartSource class adds about 90 lines of code (but it's a completely new functionality).

All the other changes give about 50 lines of new code in total (more lines are added, but some other are removed).



In my opinion, this is an acceptable cost when comparing to advantages:
- full built-in sorting support in TListChartSource, which can be further enhanced by providing custom OnCompare handler or inheritance,
- speed gain in many cases,
- new TSortedChartSource class,
- same comparison function is now used in all cases - while old TListChartSource implementation used CompareDataItemX() in TListChartSource.Sort(), and "<" or ">" in TListChartSource.Add() and TListChartSource.SetXValue() - with improper NaN handling in TListChartSource.SetXValue(),
- calling Sort() twice does not resort exact items.

Marcin Wiazowski

2019-05-20 00:06

reporter   ~0116271

Here is my evaluation of regression / new bug probability:

C1: Very low: IsSortedByXAsc() has trivial implementation.

C2: Very low: the setters have trivial implementations.
 
C3: Very low: the new code is trivial.

C4: Very low: HasSameSorting() has trivial implementation.

C5: Very low: no code is modified.

C6: Very low: one function call is added.

C7: Very low: IsSorted() has trivial implementation.

C8:
  a) Low: DoCompare() has simple implementation,
  b) Very low: DoSort() has been copied from the Classes unit, so the code has been working for years; the only introduced change is trivial.

C9: Very low: the setter has trivial implementation.

C10: Very low: Sort() has trivial implementation.

C11: Very low: the setters have trivial implementations.

C12:
  a) Very low: ItemAdd() has trivial implementation,
  b) Very low: ItemInsert() has trivial implementation,
  c) Low: ItemFind() - the algorithm has been copied from tacustomsource.pas, from FindUB() function; the added optimization is trivial; the result is assigned not from R, but from L, to place the new data at the end (this has been thoroughly tested).
  d) Moderate: ItemModified() - this is a completely new algorithm, although it has been thoroughly tested.

C13: Very low: just results are assigned.

C14: See C12 d).

C15: Very low: change is trivial.

C16: Very low: change is trivial.

C17: Very low: change is trivial.

C18: Very low: the new code is trivial.

C19: Very low: change is trivial.

C20: Very low: change is trivial.

C21: Moderate/low: the new TSortedChartSource class is based on the already existing TCalculatedChartSource.



Summary: Although many changes in code are made, most of them are simple. I assume the risk of potential problems to be low.

Marcin Wiazowski

2019-05-20 00:07

reporter   ~0116272

It's time for the overall summary.



It turns out, that patches introduce many changes, however:
- there are many similar changes (like adding ItemModified() or Notify() calls in many places, or changing procedures to functions),
- many of changes are trivial.

As a consequence, I finally evaluated the risk of potential regressions or other problems to be relatively low.



As for me, the advantages to added code size ratio is acceptable. In particular, cost of adding all the new sorting possibilities is in fact only the size of DoCompare() method, plus SetSortBy() / SetSortDir() / SetSortIndex() / SetOnCompare() setters. And, by the way, some problems and inconsistencies in sorting are solved.



Moving sorting code to another, helper class, doesn't make sense, because it won't allow to reduce the amount of code added to the compiled executable (but it would introduce an additional indirection level).



After the changes, execution time is a bit longer in some cases, but much shorter in other cases - which is good.



So, finally, I find the sorting functionality not to be as expensive, as it could initially look like.

Marcin Wiazowski

2019-05-20 00:09

reporter   ~0116273

Additional note I:

> I wonder if I ever used sorting in any chart with real data, even sorting by x

Setting Sorted to True for time-series data (either for series using external data source, or built-in source) should be advised in the documentation. This is because chart drawing is much faster in these cases (thanks to all these IsSortedByXAsc() calls in the TAChart's code). I just made some quick test: for some large data set, when only small subset is displayed (when we have data from few years, but display only one week) - setting Sorted to True (which should be performed BEFORE adding data points to the series/source - see results in 0035356:0116030) made chart drawing 2x faster in my case.



But - maybe - TListChartSource could automatically detect, that data is added in the ascending order, and make IsSorted() returning True in such case? I'll take a look at this and let you know here.

Marcin Wiazowski

2019-05-20 00:13

reporter   ~0116274

Additional side note II: as I wrote above, TSortChartSource can only react to "change" notifications, and the whole sorting algorithm must be executed after adding/modifying each point.


I thought about adding some additional information for notifications. Since Notify() method is implemented in TBasicChartSource, some additional data record could be implemented there, which I can imagine as:

  TBasicChartSource = class(TComponent)
  protected
    FNotifyInfo: TNotifyInfo;
  public
    property NotifyInfo: TNotifyInfo read FNotifyInfo;
  end;

where:

  TNotifyKind = (nkEmpty, nkUnknown, nkItemModified, nkItemMoved, nkItemDeleted);

  TNotifyInfo = record
    NotifyKind: TNotifyKind;
    Index: Integer;
    IndexNew: Integer;
  end;

so:
- for new item added, or item modified, we could have NotifyKind = nkItemModified, with Index pointing to the item,
- for item moved (or first modified, and then moved), we could have NotifyKind = nkItemMoved, with Index pointing to the old index, and IndexNew pointing to the new index,
- for item deleted, we could have NotifyKind = nkItemDeleted, with Index pointing to the deleted item.


A care should be taken, to avoid making the notification data lost: for NotifyKind = nkEmpty, the record can be freely written, but for NotifyKind <> nkEmpty (which means, that notification data has been set, but Notify() has not been called - for example when we are between the BeginUpdate .. EndUpdate calls) NotifyKind must be set to nkUnknown, so the notification client will have to make the full refresh.


Thanks to this solution, TSortChartSource could read NotifyInfo and reflect the change, instead of resorting all its data at each change. I can also imagine a situation, when the series - being a source's client - checks if the modified data point is in the current chart's viewport - if not, there would be no need to repaint the chart.


I don't have currently time to experiment with this idea, but please feel free to implement it, or adapt, if you find it useful.

wp

2019-05-20 20:08

reporter   ~0116289

Thank for this thorough study.

Applied phase3.diff -> r61248.

Trying to modify the "Test-component" project to something more useful for the TAChart examples folder (putting a SortedSource behind a ListSource is not very interesting because ListSource can do the sorting on its own) by means of a TUserDefinedChartSource I noticed that the SortedSource does not work any more. This is because the ChartDataItems made available of this source are always in the same buffer, this means that the SortedSource always compares the same items. The SortedSource must not take the ChartDataItem pointers provided by the Origin directly, but must create local copies of the items for the compare procedure. The same issue probably happens with the DBChartSource.
See "sorted_source_userdef_source.zip".

sorted_source_userdef_source.zip (3,057 bytes)

Marcin Wiazowski

2019-05-20 21:12

reporter   ~0116290

I'm attaching phase4_updated.diff, which fixes the problem that you found.

phase4_updated.diff (8,733 bytes)
Index: components/tachart/tacustomsource.pas
===================================================================
--- components/tachart/tacustomsource.pas	(revision 61248)
+++ components/tachart/tacustomsource.pas	(working copy)
@@ -99,6 +99,7 @@
     procedure SetY(const AValue: Double);
     procedure MultiplyY(const ACoeff: Double);
     function Point: TDoublePoint; inline;
+    procedure MakeUnique;
   end;
   PChartDataItem = ^TChartDataItem;
 
@@ -237,7 +238,7 @@
     procedure EndUpdate; override;
   public
     class procedure CheckFormat(const AFormat: String);
-    function BasicExtent: TDoubleRect;
+    function BasicExtent: TDoubleRect; virtual;
     function Extent: TDoubleRect; virtual;
     function ExtentCumulative: TDoubleRect; virtual;
     function ExtentList: TDoubleRect; virtual;
@@ -538,6 +539,15 @@
     Result := YList[AIndex - 1];
 end;
 
+procedure TChartDataItem.MakeUnique;
+begin
+  // using SetLength() is a documented way of making the dynamic array unique:
+  // "the reference count after a call to SetLength will be 1"
+  UniqueString(Text);
+  SetLength(XList, Length(XList));
+  SetLength(YList, Length(YList));
+end;
+
 procedure TChartDataItem.MultiplyY(const ACoeff: Double);
 var
   i: Integer;
Index: components/tachart/tasources.pas
===================================================================
--- components/tachart/tasources.pas	(revision 61248)
+++ components/tachart/tasources.pas	(working copy)
@@ -75,6 +75,41 @@
     property OnCompare;
   end;
 
+  { TSortedChartSource }
+
+  TSortedChartSource = class(TCustomSortedChartSource)
+  strict private
+    FListener: TListener;
+    FListenerSelf: TListener;
+    FOrigin: TCustomChartSource;
+    procedure Changed(ASender: TObject);
+    procedure SetOrigin(AValue: TCustomChartSource);
+  protected
+    function DoCompare(AItem1, AItem2: Pointer): Integer; override;
+    function GetCount: Integer; override;
+    function GetItem(AIndex: Integer): PChartDataItem; override;
+    procedure ResetTransformation(ACount: Integer);
+    procedure SetXCount(AValue: Cardinal); override;
+    procedure SetYCount(AValue: Cardinal); override;
+  public
+    constructor Create(AOwner: TComponent); override;
+    destructor Destroy; override;
+  published
+    function BasicExtent: TDoubleRect; override;
+    function Extent: TDoubleRect; override;
+    function ExtentCumulative: TDoubleRect; override;
+    function ExtentList: TDoubleRect; override;
+    function ExtentXYList: TDoubleRect; override;
+    function ValuesTotal: Double; override;
+    property Origin: TCustomChartSource read FOrigin write SetOrigin;
+    // Sorting
+    property SortBy;
+    property SortDir;
+    property Sorted;
+    property SortIndex;
+    property OnCompare;
+  end;
+
   { TMWCRandomGenerator }
 
   // Mutliply-with-carry random number generator.
@@ -276,12 +311,11 @@
 begin
   RegisterComponents(
     CHART_COMPONENT_IDE_PAGE, [
-      TListChartSource, TRandomChartSource, TUserDefinedChartSource,
-      TCalculatedChartSource
+      TListChartSource, TSortedChartSource, TRandomChartSource,
+      TUserDefinedChartSource, TCalculatedChartSource
     ]);
 end;
 
-
 { TListChartSourceStrings }
 
 procedure TListChartSourceStrings.Clear;
@@ -481,7 +515,6 @@
     end;
 end;
 
-
 { TListChartSource }
 
 function TListChartSource.Add(
@@ -841,6 +874,196 @@
   (FDataPoints as TListChartSourceStrings).LoadingFinished;
 end;
 
+{ TSortedChartSource }
+
+constructor TSortedChartSource.Create(AOwner: TComponent);
+begin
+  inherited Create(AOwner);
+  FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
+  FYCount := MaxInt;
+  FListener := TListener.Create(@FOrigin, @Changed);
+  FListenerSelf := TListener.Create(nil, @Changed);
+  Broadcaster.Subscribe(FListenerSelf);
+end;
+
+destructor TSortedChartSource.Destroy;
+begin
+  ResetTransformation(0);
+  FreeAndNil(FListenerSelf);
+  FreeAndNil(FListener);
+  inherited;
+end;
+
+procedure TSortedChartSource.Changed(ASender: TObject);
+begin
+  if ASender = Self then begin
+    // We can get here only due to FListenerSelf's notification.
+    // If some of our own (not Origin's) sorting properties was changed and we
+    // are sorted, then our Sort() method has been called, so the transformation
+    // is valid; but if we are no longer sorted, only notification is sent (so
+    // we are here), so we must reinitialize the transformation to return to
+    // the transparent (i.e. unsorted) state.
+    if not IsSorted then
+      ResetTransformation(Count);
+    exit;
+  end;
+
+  if FOrigin <> nil then begin
+    FXCount := Origin.XCount;
+    FYCount := Origin.YCount;
+    ResetTransformation(Origin.Count);
+    if IsSorted and (not HasSameSorting(Origin)) then Sort else Notify;
+  end else begin
+    FXCount := MaxInt;    // Allow source to be used by any series while Origin = nil
+    FYCount := MaxInt;
+    ResetTransformation(0);
+    Notify;
+  end;
+end;
+
+function TSortedChartSource.DoCompare(AItem1, AItem2: Pointer): Integer;
+var
+  item1, item2: TChartDataItem;
+begin
+  // some data sources use same memory buffer for every item read,
+  // so local copies must be made before comparing two items
+  item1 := Origin.Item[PInteger(AItem1)^]^;
+
+  // avoid sharing same memory by item1's and item2's reference-
+  // counted variables
+  item1.MakeUnique;
+
+  item2 := Origin.Item[PInteger(AItem2)^]^;
+
+  Result := inherited DoCompare(@item1, @item2);
+end;
+
+function TSortedChartSource.GetCount: Integer;
+begin
+  if Origin <> nil then
+    Result := Origin.Count
+  else
+    Result := 0;
+end;
+
+function TSortedChartSource.GetItem(AIndex: Integer): PChartDataItem;
+begin
+  if Origin <> nil then
+    Result := PChartDataItem(Origin.Item[PInteger(FData.Items[AIndex])^])
+  else
+    Result := nil;
+end;
+
+procedure TSortedChartSource.ResetTransformation(ACount: Integer);
+var
+  i: Integer;
+  pint: PInteger;
+begin
+  if ACount > FData.Count then begin
+    for i := 0 to FData.Count - 1 do
+      PInteger(FData.List^[i])^ := i;
+
+    FData.Capacity := ACount;
+
+    pint := nil;
+    try // optimization: don't execute try..except..end in a loop
+      for i := FData.Count to ACount - 1 do begin
+        New(pint);
+        pint^ := i;
+        FData.Add(pint); // don't use ItemAdd() here
+        pint := nil;
+      end;
+    except
+      if pint <> nil then
+        Dispose(pint);
+      raise;
+    end;
+  end else
+  begin
+    for i := ACount to FData.Count - 1 do
+      Dispose(PInteger(FData.List^[i]));
+
+    FData.Count := ACount;
+    FData.Capacity := ACount; // release needless memory
+
+    for i := 0 to FData.Count - 1 do
+      PInteger(FData.List^[i])^ := i;
+  end;
+end;
+
+procedure TSortedChartSource.SetOrigin(AValue: TCustomChartSource);
+begin
+  if AValue = Self then
+    AValue := nil;
+  if FOrigin = AValue then exit;
+  if FOrigin <> nil then
+    FOrigin.Broadcaster.Unsubscribe(FListener);
+  FOrigin := AValue;
+  if FOrigin <> nil then
+    FOrigin.Broadcaster.Subscribe(FListener);
+  Changed(nil);
+end;
+
+procedure TSortedChartSource.SetXCount(AValue: Cardinal);
+begin
+  Unused(AValue);
+  raise EXCountError.Create('Cannot set XCount');
+end;
+
+procedure TSortedChartSource.SetYCount(AValue: Cardinal);
+begin
+  Unused(AValue);
+  raise EYCountError.Create('Cannot set YCount');
+end;
+
+function TSortedChartSource.BasicExtent: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.BasicExtent;
+end;
+
+function TSortedChartSource.Extent: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.Extent;
+end;
+
+function TSortedChartSource.ExtentCumulative: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.ExtentCumulative;
+end;
+
+function TSortedChartSource.ExtentList: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.ExtentList;
+end;
+
+function TSortedChartSource.ExtentXYList: TDoubleRect;
+begin
+  if Origin = nil then
+    Result := EmptyExtent
+  else
+    Result := Origin.ExtentXYList;
+end;
+
+function TSortedChartSource.ValuesTotal: Double;
+begin
+  if Origin = nil then
+    Result := 0
+  else
+    Result := Origin.ValuesTotal;
+end;
+
 { TMWCRandomGenerator }
 
 function TMWCRandomGenerator.Get: LongWord;
@@ -1452,7 +1675,7 @@
 procedure TCalculatedChartSource.SetOrigin(AValue: TCustomChartSource);
 begin
   if AValue = Self then
-      AValue := nil;
+    AValue := nil;
   if FOrigin = AValue then exit;
   if FOrigin <> nil then
     FOrigin.Broadcaster.Unsubscribe(FListener);
phase4_updated.diff (8,733 bytes)

wp

2019-05-20 23:11

reporter   ~0116294

Thank you. Now I applied also "phase4-updated.diff" (-> r61251) and "phase5.diff" (-> r61252).

I think it's time now to close this monster thread. Please open new reports for any extensions, re-open the current report only if this particular current code has issues.

Marcin Wiazowski

2019-05-21 00:19

reporter   ~0116295

Oook.... I reviewed the whole thread - and all the problems described here are solved now; one discussion was moved to 0035535.



I'd like to correct one my statement: in 0035356:0116030 I wrote:

"quick sort algorithm - which is highly time-consuming even for sorted data (and much more time-consuming for random data)"

As can be read in Wikipedia, sorting the already sorted data is the worst case for the quick sort algorithm; some modifications in the algorithm are possible to avoid slowdown, but Lazarus implementations don't use such improvements.



Thank you for your valuable notices.

Issue History

Date Modified Username Field Change
2019-04-10 16:20 Marcin Wiazowski New Issue
2019-04-10 16:20 Marcin Wiazowski File Added: patch.diff
2019-04-10 17:02 wp Note Added: 0115396
2019-04-10 17:02 wp Assigned To => wp
2019-04-10 17:02 wp Status new => assigned
2019-04-10 17:07 wp Note Edited: 0115396 View Revisions
2019-04-10 17:07 wp Note Edited: 0115396 View Revisions
2019-04-10 17:45 Marcin Wiazowski Note Added: 0115398
2019-04-10 18:38 wp Note Added: 0115399
2019-04-10 19:59 Marcin Wiazowski Note Added: 0115401
2019-04-10 22:51 wp Note Added: 0115411
2019-04-10 23:19 Marcin Wiazowski Note Added: 0115413
2019-04-10 23:48 wp Note Added: 0115414
2019-04-12 01:42 Marcin Wiazowski Note Added: 0115433
2019-04-12 23:15 Marcin Wiazowski File Added: Test.zip
2019-04-12 23:15 Marcin Wiazowski File Added: Test.png
2019-04-12 23:16 Marcin Wiazowski File Added: patch_ver2.diff
2019-04-12 23:19 Marcin Wiazowski Note Added: 0115455
2019-04-13 01:07 wp Note Added: 0115459
2019-04-13 01:09 wp File Added: Test-wp.zip
2019-04-13 10:27 wp Note Edited: 0115459 View Revisions
2019-04-13 10:28 wp Note Edited: 0115459 View Revisions
2019-04-13 16:36 Marcin Wiazowski Note Added: 0115474
2019-04-13 18:37 wp Note Added: 0115475
2019-04-13 19:45 Marcin Wiazowski Note Added: 0115476
2019-04-13 20:01 wp Note Added: 0115477
2019-04-13 20:24 Marcin Wiazowski Note Added: 0115478
2019-04-13 23:43 wp Note Added: 0115480
2019-04-14 03:53 Marcin Wiazowski File Added: patch_ver3.diff
2019-04-14 03:58 Marcin Wiazowski Note Added: 0115487
2019-04-14 04:12 Marcin Wiazowski Note Added: 0115488
2019-04-14 22:54 wp File Added: BubbleSortTest.zip
2019-04-14 23:07 wp Note Added: 0115509
2019-04-14 23:08 wp Note Edited: 0115509 View Revisions
2019-04-14 23:09 wp Note Edited: 0115509 View Revisions
2019-04-15 04:14 Marcin Wiazowski File Added: patch_new_update.diff
2019-04-15 04:18 Marcin Wiazowski Note Added: 0115513
2019-04-15 16:37 Marcin Wiazowski Note Added: 0115523
2019-04-15 17:27 Marcin Wiazowski Note Added: 0115526
2019-04-15 18:52 wp Note Added: 0115527
2019-04-15 23:42 Marcin Wiazowski Note Added: 0115532
2019-04-17 04:07 Marcin Wiazowski Note Added: 0115576
2019-04-17 09:12 wp Note Added: 0115580
2019-04-17 12:27 Marcin Wiazowski Note Added: 0115588
2019-04-17 16:45 wp Note Added: 0115594
2019-04-17 17:25 Marcin Wiazowski Note Added: 0115599
2019-04-17 18:14 wp Note Added: 0115602
2019-04-17 23:51 Marcin Wiazowski Note Added: 0115612
2019-04-18 01:04 Marcin Wiazowski Note Added: 0115617
2019-04-18 01:06 Marcin Wiazowski Note Edited: 0115617 View Revisions
2019-04-18 01:15 wp Note Added: 0115618
2019-04-18 02:36 Marcin Wiazowski Note Added: 0115622
2019-04-18 15:19 Marcin Wiazowski Note Added: 0115649
2019-04-18 16:28 wp Note Added: 0115653
2019-04-19 14:38 wp Note Added: 0115674
2019-04-20 11:28 wp Note Edited: 0115674 View Revisions
2019-04-27 23:30 Marcin Wiazowski Note Added: 0115861
2019-04-27 23:45 Marcin Wiazowski Note Added: 0115862
2019-04-29 10:59 wp Note Added: 0115887
2019-04-29 12:20 Marcin Wiazowski Note Added: 0115889
2019-04-30 10:34 wp Note Added: 0115914
2019-05-01 15:02 Marcin Wiazowski Note Added: 0115944
2019-05-05 22:55 Marcin Wiazowski File Added: SpeedTest.zip
2019-05-05 22:55 Marcin Wiazowski File Added: exchange_test.diff
2019-05-05 22:55 Marcin Wiazowski Note Added: 0116030
2019-05-06 01:08 wp Note Added: 0116033
2019-05-06 01:53 Marcin Wiazowski Note Added: 0116035
2019-05-07 11:41 wp Note Added: 0116060
2019-05-07 14:47 Marcin Wiazowski Note Added: 0116064
2019-05-08 23:50 Marcin Wiazowski File Added: phase1.diff
2019-05-08 23:50 Marcin Wiazowski Note Added: 0116085
2019-05-09 14:52 wp Note Added: 0116101
2019-05-11 18:09 Marcin Wiazowski File Added: const_rec.png
2019-05-11 18:09 Marcin Wiazowski File Added: const_str.png
2019-05-11 18:09 Marcin Wiazowski File Added: phase2.diff
2019-05-11 18:09 Marcin Wiazowski Note Added: 0116138
2019-05-12 22:21 wp Note Added: 0116150
2019-05-14 23:53 Marcin Wiazowski File Added: phase3.diff
2019-05-14 23:53 Marcin Wiazowski Note Added: 0116200
2019-05-15 19:40 Marcin Wiazowski File Added: phase4.diff
2019-05-15 19:40 Marcin Wiazowski File Added: Test-component.zip
2019-05-15 19:40 Marcin Wiazowski Note Added: 0116211
2019-05-16 10:07 wp Note Added: 0116216
2019-05-16 10:08 wp Note Edited: 0116216 View Revisions
2019-05-19 23:59 Marcin Wiazowski File Added: phase5.diff
2019-05-19 23:59 Marcin Wiazowski Note Added: 0116266
2019-05-20 00:00 Marcin Wiazowski Note Added: 0116267
2019-05-20 00:02 Marcin Wiazowski Note Added: 0116268
2019-05-20 00:04 Marcin Wiazowski Note Added: 0116269
2019-05-20 00:05 Marcin Wiazowski Note Added: 0116270
2019-05-20 00:06 Marcin Wiazowski Note Added: 0116271
2019-05-20 00:07 Marcin Wiazowski Note Added: 0116272
2019-05-20 00:09 Marcin Wiazowski Note Added: 0116273
2019-05-20 00:13 Marcin Wiazowski Note Added: 0116274
2019-05-20 20:08 wp File Added: sorted_source_userdef_source.zip
2019-05-20 20:08 wp Note Added: 0116289
2019-05-20 21:12 Marcin Wiazowski File Added: phase4_updated.diff
2019-05-20 21:12 Marcin Wiazowski Note Added: 0116290
2019-05-20 23:11 wp Status assigned => resolved
2019-05-20 23:11 wp Resolution open => fixed
2019-05-20 23:11 wp Fixed in Revision => 60972, 61189, 61190, 61211, 61248, 61251. 61252
2019-05-20 23:11 wp LazTarget => -
2019-05-20 23:11 wp Widgetset Win32/Win64 => Win32/Win64
2019-05-20 23:11 wp Note Added: 0116294
2019-05-21 00:19 Marcin Wiazowski Status resolved => closed
2019-05-21 00:19 Marcin Wiazowski Note Added: 0116295
2019-05-25 11:13 Juha Manninen Relationship added related to 0035630
2019-06-02 11:52 wp Relationship added related to 0035664
2019-06-02 11:54 wp Relationship added related to 0035666
2019-06-14 12:27 wp Relationship added related to 0035681