View Issue Details

IDProjectCategoryView StatusLast Update
0019650LazarusLCLpublic2013-06-06 07:35
ReporterAnton Kavalenka Assigned ToMattias Gaertner  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Platformall 
Product Version0.9.31 (SVN) 
Summary0019650: lresources.pas: r31040 broke usage of WIdeString properties in componets
DescriptionThe revision has removed storage of widestring properties into binary resource or LRS resource.

The problem made by assuming all the string properties are UTF-8 encoded widestring.
Steps To ReproduceUse .LRS form storage

Install component from 0012981 and try to examine what is stored into stream.
TagsNo tags attached.
Fixed in Revision37642
LazTarget-
WidgetsetGTK 2, Win32/Win64
Attached Files

Relationships

related to 0019467 closedMattias Gaertner Delphi compatible Widestring syntax does not show correctly on Windows 

Activities

Anton Kavalenka

2011-07-30 14:14

reporter   ~0050284

Last edited: 2011-07-30 14:42

Both String and WString data is written to binary stream without any indication that strings are UTF-8 encoded.

All the strings are written as vaString or vaLString but not vaUTF8String, which would be in all cases CORRECT and TRUE.

see lresorces.pas line 2767 and further

!!! Output.WriteByte(Ord(vaLString));
!!! WriteLongString(toStringBuf);


When the destination component has property of String - nothing special happens.
When the destination component has Widestring property - the Unicode is lost
I.E the following occurs:

see reader.inc line 1504
function TReader.ReadWideString: WideString;
var
 s: String;
 i: Integer;
 vt:TValueType;
begin
  if NextValue in [vaWString,vaUString,vaUTF8String] then
    //vaUTF8String needs conversion? 2008-09-06 mse, YES!! AntonK
    begin
      vt:=ReadValue;
      if vt=vaUTF8String then
        Result := utf8decode(fDriver.ReadString(vaLString))
      else
        Result := FDriver.ReadWideString
    end
  else
    begin
      //data probable from ObjectTextToBinary
      s := ReadString;
      setlength(result,length(s));
      for i:= 1 to length(s) do begin
        result[i]:= widechar(ord(s[i])); //no code conversion
    end;
  end;
end;

Anton Kavalenka

2011-07-30 14:39

reporter   ~0050287

Unfortunately TLRSObjectReader does not handle vaUTF8String

IMO the deletion of vaWstring writing should be reverted.

Anton Kavalenka

2012-06-14 11:06

reporter   ~0060517

Patch provided to allow handing WideString properties.
Currently components with WideString/UnicodeString properties cannot be properly converted to text representations and back (LRSObjectTextToBinary<->LRSObjectToText).

2012-06-14 11:07

 

lresources.diff (1,873 bytes)   
Index: lcl/lresources.pp
===================================================================
--- lcl/lresources.pp	(revision 37641)
+++ lcl/lresources.pp	(working copy)
@@ -2675,6 +2675,13 @@
       Output.Write(s[1], Length(s));
   end;
 
+  procedure WriteWideString(const s: WideString);
+  begin
+    WriteLRSInteger(Output,Length(s));
+    if Length(s) > 0 then
+      Output.Write(s[1], Length(s)*2);
+  end;
+
   procedure WriteInteger(value: LongInt);
   begin
     if (value >= -128) and (value <= 127) then begin
@@ -2779,7 +2786,7 @@
           WriteLRSExtended(Output,flt);
           ParserNextToken;
         end;
-      toString,toWString:
+      toString:
         begin
           toStringBuf := parser.TokenString;
           //DebugLn(['ProcessValue toStringBuf="',toStringBuf,'" ',dbgstr(toStringBuf)]);
@@ -2800,6 +2807,20 @@
             WriteLongString(toStringBuf);
           end;
         end;
+      toWString:
+        begin
+          toStringBuf := parser.TokenString;
+          //DebugLn(['ProcessValue toStringBuf="',toStringBuf,'" ',dbgstr(toStringBuf)]);
+          while ParserNextToken = '+' do
+          begin
+            ParserNextToken;   // Get next string fragment
+            if not (parser.Token in [toString,toWString]) then
+              parser.CheckToken(toString);
+            toStringBuf := toStringBuf + parser.TokenString;
+          end;
+          Output.WriteByte(Ord(vaWString));
+          WriteWideString(UTF8Decode(toStringBuf));
+        end;
       toSymbol:
         begin
           if CompareText(parser.TokenString, 'True') = 0 then
@@ -5321,7 +5342,7 @@
 
 procedure TUTF8Parser.LoadBuffer;
 var newread : integer;
-begin
+  begin
   newread:=fStream.Read(fBuf[0],ParseBufSize);
   fBuf[newread]:=#0;
   fLineStart:=fLineStart-fPos; // column = fPos - fLineStart + 1
@@ -5754,3 +5775,5 @@
 
 end.
 
+
+
lresources.diff (1,873 bytes)   

Mattias Gaertner

2012-06-14 11:52

manager   ~0060520

Thanks. Applied.

Issue History

Date Modified Username Field Change
2011-06-28 17:06 Anton Kavalenka New Issue
2011-06-28 17:06 Anton Kavalenka Widgetset => GTK 2, Win32/Win64
2011-06-28 19:40 Mattias Gaertner Status new => assigned
2011-06-28 19:40 Mattias Gaertner Assigned To => Mattias Gaertner
2011-07-30 14:14 Anton Kavalenka Note Added: 0050284
2011-07-30 14:39 Anton Kavalenka Note Added: 0050287
2011-07-30 14:42 Anton Kavalenka Note Edited: 0050284
2012-06-14 11:06 Anton Kavalenka Note Added: 0060517
2012-06-14 11:07 Anton Kavalenka File Added: lresources.diff
2012-06-14 11:52 Mattias Gaertner Fixed in Revision => 37642
2012-06-14 11:52 Mattias Gaertner LazTarget => -
2012-06-14 11:52 Mattias Gaertner Note Added: 0060520
2012-06-14 11:52 Mattias Gaertner Status assigned => resolved
2012-06-14 11:52 Mattias Gaertner Resolution open => fixed
2012-06-14 12:18 Anton Kavalenka Status resolved => closed
2013-06-06 07:35 Juha Manninen Relationship added related to 0019467