View Issue Details

IDProjectCategoryView StatusLast Update
0025045LazarusIDEpublic2017-04-25 11:53
ReporterPaul W Assigned ToMattias Gaertner  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
PlatformWindows 32bit 
Product Version1.0.12 
Summary0025045: Codetools incorrectly handles multi-dimensional arrays
DescriptionWith some correct pascal code related to arrays, codetools reject to work.
Possibly I try to report two issues at once, if Access violation is other issue, I can report it separately, but for now I'm not sure.
Steps To Reproduce1.Open attached Lazarus project (ctrlspacerror.zip).
2.Go to any line reported as not working and place cursor after dot.
3.Press Ctrl+Space by default to make Lazarus open Identifier completion.
4.Unexpectedly Lazarus rejects to do so, while it shouldn't.
Tagspatch
Fixed in Revision54719 54729
LazTarget-
WidgetsetWin32/Win64
Attached Files

Relationships

related to 0022627 assignedMattias Gaertner access violation at variable declaration completion with 2d array 
related to 0016711 assignedMattias Gaertner Error with code Completion for 2D or higher dimension array's 

Activities

Paul W

2013-09-17 15:39

reporter  

ctrlspaceerror.zip (1,419 bytes)

Anton

2015-01-27 19:10

reporter  

multidim_array.patch (9,311 bytes)   
Index: components/codetools/finddeclarationtool.pas
===================================================================
--- components/codetools/finddeclarationtool.pas	(revision 47519)
+++ components/codetools/finddeclarationtool.pas	(working copy)
@@ -7614,7 +7614,7 @@
     {$ENDIF}
     if fdfExtractOperand in Params.Flags then begin
       // simple copying, todo: expand argument
-      Params.AddOperandPart(Copy(Src, CurPos.StartPos, CurAtomBracketEndPos-CurPos.StartPos));
+      Params.AddOperandPart(ExtractBrackets(CurPos.StartPos,[]));
     end;
     if (not (NextAtomType in [vatSpace,vatPoint,vatAs,vatUp,vatRoundBracketClose,
       vatRoundBracketOpen,vatEdgedBracketClose,vatEdgedBracketOpen]))
@@ -7641,6 +7641,27 @@
       RaiseIllegalQualifierFound;
     end;
 
+    if ExprType.Context.Node.Desc in [ctnRangedArrayType,ctnOpenArrayType] then
+    begin
+      MoveCursorToCleanPos(CurAtom.StartPos);
+      ReadNextAtom; // "["
+      ReadNextAtom;
+      repeat
+        case CurPos.Flag of
+        cafRoundBracketClose: SaveRaiseBracketCloseExpectedButAtomFound;
+        cafRoundBracketOpen,
+        cafEdgedBracketOpen: ReadTilBracketClose(true);
+        cafComma:
+          with ExprType, Context do begin
+            Context:=Tool.FindBaseTypeOfNode(Params,Node.LastChild);
+            if not (Node.Desc in [ctnRangedArrayType,ctnOpenArrayType]) then
+              RaiseIllegalQualifierFound;
+          end;
+        end;
+        ReadNextAtom;
+      until CurPos.Flag=cafEdgedBracketClose;
+    end;
+
     {$IFDEF ShowExprEval}
     DebugLn(['  FindExpressionTypeOfTerm ResolveEdgedBracketOpen ExprType=',ExprTypeToString(ExprType)]);
     {$ENDIF}
@@ -9526,6 +9547,16 @@
         then begin
           // parameter list ended in front of Variable => continue search
           {$IFDEF VerboseCPS}DebugLn('CheckIdentifierAndParameterList parameter list ended in front of cursor');{$ENDIF}
+          if CurPos.Flag=cafEdgedBracketClose then begin
+            ReadNextAtom;
+            if CurPos.Flag=cafEdgedBracketOpen then begin
+              // [][] is equal to [,]
+              ParameterStart:=CurPos.EndPos;
+              inc(CurParameterIndex);
+              continue;
+            end else
+              UndoReadNextAtom;
+          end;
           exit;
         end else begin
           // invalid closing bracket found
Index: components/codetools/pascalparsertool.pas
===================================================================
--- components/codetools/pascalparsertool.pas	(revision 47519)
+++ components/codetools/pascalparsertool.pas	(working copy)
@@ -4398,42 +4398,61 @@
     array[SubRange,SubRange,...] of ...
     array[Subrange];  // without "of" means array of byte
 }
-begin
-  CreateChildNode;
-  // first set the type to open array (an array type without brackets)
-  CurNode.Desc:=ctnOpenArrayType;
-  ReadNextAtom;
-  if (CurPos.Flag=cafEdgedBracketOpen) then begin
-    repeat
-      ReadNextAtom;
-      // this is a ranged array -> change type
-      CurNode.Desc:=ctnRangedArrayType;
-      CreateChildNode;
-      CurNode.Desc:=ctnRangeType;
-      ReadSubRange(true);
-      CurNode.EndPos:=LastAtoms.GetValueAt(0).EndPos;
-      EndChildNode; // close ctnRangeType
-      if (CurPos.Flag=cafEdgedBracketClose) then break;
-      if (CurPos.Flag<>cafComma) then
-        SaveRaiseCharExpectedButAtomFound(']');
-    until false;
-    ReadNextAtom;
+
+  function ReadElemType: boolean;
+  begin
     if CurPos.Flag in [cafSemicolon,cafRoundBracketClose,cafEdgedBracketClose]
     then begin
       // array[] without "of" means array[] of byte
       CurNode.EndPos:=CurPos.StartPos;
       EndChildNode; // close array
-      exit(true);
+      Result:=true;
+    end else begin
+      if not UpAtomIs('OF') then
+        SaveRaiseStringExpectedButAtomFound('"of"');
+      ReadNextAtom;
+      Result:=ParseType(CurPos.StartPos);
+      CurNode.EndPos:=CurPos.StartPos;
+      EndChildNode; // close array
     end;
   end;
-  if not UpAtomIs('OF') then
-    SaveRaiseStringExpectedButAtomFound('"of"');
+
+  function ReadIndexType: boolean;
+  begin
+    ReadNextAtom;
+    CreateChildNode;
+    CurNode.Desc:=ctnRangeType;
+    ReadSubRange(true);
+    CurNode.EndPos:=LastAtoms.GetValueAt(0).EndPos;
+    EndChildNode; // close ctnRangeType
+    if CurPos.Flag=cafComma then begin
+      // "array [T1,T2]" is equal to "array [T1] of array [T2]"
+      // so they should be parsed to the same CodeTree
+      CreateChildNode;
+      CurNode.Desc:=ctnRangedArrayType;
+      Result:=ReadIndexType();
+      CurNode.EndPos:=LastAtoms.GetValueAt(0).EndPos;
+      EndChildNode; // close ctnRangedArrayType
+    end else begin
+      if CurPos.Flag<>cafEdgedBracketClose then
+        SaveRaiseCharExpectedButAtomFound(']');
+      ReadNextAtom;
+      CurNode.EndPos:=LastAtoms.GetValueAt(0).EndPos;
+      Result:=ReadElemType;
+    end;
+  end;
+
+begin
+  CreateChildNode;
+  // first set the type to open array (an array type without brackets)
+  CurNode.Desc:=ctnOpenArrayType;
   ReadNextAtom;
-  Result:=ParseType(CurPos.StartPos);
-  CurNode.EndPos:=CurPos.StartPos;
-  EndChildNode; // close array
-  //debugln(['TPascalParserTool.KeyWordFuncTypeArray END Atom=',GetAtom,' CurNode=',CurNode.DescAsString]);
-  Result:=true;
+  if (CurPos.Flag=cafEdgedBracketOpen) then begin
+    CurNode.Desc:=ctnRangedArrayType;
+    Result:=ReadIndexType;
+    exit;
+  end;
+  Result:=ReadElemType;
 end;
 
 function TPascalParserTool.KeyWordFuncTypeProc: boolean;
Index: components/codetools/pascalreadertool.pas
===================================================================
--- components/codetools/pascalreadertool.pas	(revision 47519)
+++ components/codetools/pascalreadertool.pas	(working copy)
@@ -225,6 +225,8 @@
     // arrays
     function ExtractArrayRange(ArrayNode: TCodeTreeNode;
         Attr: TProcHeadAttributes): string;
+    function ExtractArrayRanges(ArrayNode: TCodeTreeNode;
+        Attr: TProcHeadAttributes): string;
 
     // module sections
     function ExtractSourceName: string;
@@ -2847,6 +2849,32 @@
   Result:=ExtractBrackets(CurPos.StartPos,Attr);
 end;
 
+function TPascalReaderTool.ExtractArrayRanges(ArrayNode: TCodeTreeNode;
+  Attr: TProcHeadAttributes): string;
+const
+  AllArrays = [ctnRangedArrayType, ctnOpenArrayType];
+var
+  i: Integer;
+begin
+  Result:='';
+  if (ArrayNode=nil) or not (ArrayNode.Desc in AllArrays) then exit;
+  while Assigned(ArrayNode.Parent) and (ArrayNode.Parent.Desc in AllArrays) do
+    ArrayNode:=ArrayNode.Parent;
+  MoveCursorToNodeStart(ArrayNode);
+  if not ReadNextUpAtomIs('ARRAY') then exit;
+  repeat
+    if Result<>'' then Result:=Result+',';
+    if ArrayNode.Desc=ctnRangedArrayType then begin
+      MoveCursorToNodeStart(ArrayNode.FirstChild); // FirstChild=Index type
+      Result:=Result+ExtractNode(ArrayNode.FirstChild,Attr);
+    end else begin
+      Result:=Result+'PtrUInt';
+    end;
+    ArrayNode:=ArrayNode.LastChild;
+  until not (ArrayNode.Desc in AllArrays);
+  Result:='['+Result+']';
+end;
+
 function TPascalReaderTool.PropertyIsDefault(PropertyNode: TCodeTreeNode): boolean;
 begin
   Result:=false;
Index: ide/codecontextform.pas
===================================================================
--- ide/codecontextform.pas	(revision 47519)
+++ ide/codecontextform.pas	(working copy)
@@ -337,8 +337,18 @@
         end;
       ')',']':
         begin
+          if BracketLevel=1 then begin
+            if Code[TokenStart]=']' then begin
+              ReadRawNextPascalAtom(Code,TokenEnd,TokenStart);
+              if TokenEnd=TokenStart then exit;
+              if Code[TokenStart]='[' then begin
+                inc(NewParameterIndex);
+                continue; // [][] is full version of [,]
+              end
+            end else
+              exit;// cursor behind procedure call
+          end;
           dec(BracketLevel);
-          if BracketLevel=0 then exit;// cursor behind procedure call
         end;
       ',':
         if BracketLevel=1 then inc(NewParameterIndex);
@@ -394,16 +404,11 @@
                  phpWithResultType]);
               Result:=true;
             end;
-          ctnOpenArrayType:
+          ctnOpenArrayType,ctnRangedArrayType:
             begin
-              s:=s+'[Index: PtrUInt]';
+              s:=s+ExprTool.ExtractArrayRanges(ExprNode,[]);
               Result:=true;
             end;
-          ctnRangedArrayType:
-            begin
-              s:=s+ExprTool.ExtractArrayRange(ExprNode,[]);
-              Result:=true;
-            end;
           end;
         end else if Expr.Desc in (xtAllStringTypes+xtAllWideStringTypes-[xtShortString])
         then begin
@@ -624,10 +629,10 @@
         repeat
           inc(p);
         until (p>=length(Result)) or (Result[p]='''');
-      'a'..'z','A'..'Z','_':
+      'a'..'z','A'..'Z','_','0'..'9':
         if (BracketLevel=1) and (not ReadingType) then begin
           WordStart:=p;
-          while (p<=length(Result)) and (IsIdentChar[Result[p]]) do
+          while (p<=length(Result)) and IsDottedIdentChar[Result[p]] do
             inc(p);
           WordEnd:=p;
           //DebugLn('MarkCurrentParameterInHint Word=',copy(Result,WordStart,WordEnd-WordStart));
multidim_array.patch (9,311 bytes)   

Anton

2015-01-27 19:13

reporter   ~0080615

Possible patch is attached.
I have noticed that "(." is not equal (from the CT point of view) to "[" (and ".)" is not equal to "]"). ReadNextAtom() should be slightly improved.

Juha Manninen

2017-04-24 18:33

developer   ~0099869

Ping Mattias. What do you think about the patch? It has been ignored for well over 2 years.

Mattias Gaertner

2017-04-25 09:09

manager   ~0099895

Thank You!

Ondrej Pokorny

2017-04-25 11:26

developer   ~0099912

The patch broke array field parsing:

TMyClass = class
private
  MyField: array of Integer;
end;


You can observe the error e.g. on TCompositeCellEditor in grids.pas.

Issue History

Date Modified Username Field Change
2013-09-17 15:39 Paul W New Issue
2013-09-17 15:39 Paul W File Added: ctrlspaceerror.zip
2013-09-17 16:33 Mattias Gaertner Assigned To => Mattias Gaertner
2013-09-17 16:33 Mattias Gaertner Status new => assigned
2014-10-28 23:18 Juha Manninen Relationship added related to 0022627
2014-10-28 23:18 Juha Manninen Relationship added related to 0016711
2014-10-28 23:19 Juha Manninen LazTarget => -
2014-10-28 23:19 Juha Manninen Summary Codetools incorrectly handle arrays => Codetools incorrectly handles multi-dimensional arrays
2015-01-27 19:10 Anton File Added: multidim_array.patch
2015-01-27 19:11 Anton Tag Attached: patch
2015-01-27 19:13 Anton Note Added: 0080615
2017-04-24 18:33 Juha Manninen Note Added: 0099869
2017-04-25 09:09 Mattias Gaertner Fixed in Revision => 54719
2017-04-25 09:09 Mattias Gaertner Note Added: 0099895
2017-04-25 09:09 Mattias Gaertner Status assigned => resolved
2017-04-25 09:09 Mattias Gaertner Resolution open => fixed
2017-04-25 11:26 Ondrej Pokorny Note Added: 0099912
2017-04-25 11:26 Ondrej Pokorny Status resolved => assigned
2017-04-25 11:26 Ondrej Pokorny Resolution fixed => reopened
2017-04-25 11:53 Mattias Gaertner Fixed in Revision 54719 => 54719 54729
2017-04-25 11:53 Mattias Gaertner Status assigned => resolved
2017-04-25 11:53 Mattias Gaertner Resolution reopened => fixed