View Issue Details

IDProjectCategoryView StatusLast Update
0032185FPCFCLpublic2017-07-29 08:49
ReporterRolf WetjenAssigned ToMichael Van Canneyt 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
PlatformWindowsOSWindows 10 x64OS Version1703
Product Version3.1.1Product Build36763 
Target Version3.2.0Fixed in Version3.1.1 
Summary0032185: TRegistry can't handle UTF8/Unicode strings
DescriptionTRegistry uses the "A" version of the windows library functions.
Additional InformationI've attached a patch for this including a small test program.
TagsNo tags attached.
Fixed in Revision36765
FPCOldBugId
FPCTarget
Attached Files

Activities

Rolf Wetjen

2017-07-22 09:16

reporter  

TRegistry_UTF8.zip (148,341 bytes)

Michael Van Canneyt

2017-07-22 11:02

administrator   ~0101846

Your change in TRegistry is invalid for unix, where the returned data is still a 1-byte string.

I think the proper solution is probably to introduce ReadUnicodeString, and to change the ReadString return type to TUTF8String.

Thaddy de Koning

2017-07-22 11:10

reporter   ~0101847

Imo limit it to Ansi and Unicode16.

Rolf Wetjen

2017-07-22 15:25

reporter   ~0101850

Hi Michael,
please help me to understand your answer:
TRegistry.OpenKey uses RegCreateKeyEx for example. Searching for RegCreateKeyExA in the whole FPC folder finds only occurences in ...\RTL\Win, ...\RTL\Wince and in jwawinreg.pas. So, where is unix?
Rolf

Michael Van Canneyt

2017-07-22 18:29

administrator   ~0101853

The unix implementation is completely different, as the registry does not exist in unit. The unix implementation uses a TXMLRegistry class, which is used in xmlreg.inc

I changed your program to a console program.

Output on windows:
TestKey passed: Software\zzz_test\ASCII
TestValue passed: 000_ASCII; これは値です;
TestValue passed: 001_ASCII; ASCII;
TestValue passed: 010_പേര്; これは値です;
TestValue passed: 011_പേര്; ASCII;
TestValue passed: 020_ഇതാണ്ASCII; これは値です;
TestValue passed: 021_ഇതാണ്ASCII; ASCII;
TestValue passed: 030_这是一个测试; これは値です;
TestValue passed: 031_这是一个测试; ASCII;
TestGetValueNames passed: Key: "Software\zzz_test\ASCII"; Expected "000_ASCII;001_ASCII;010_പേര്;011_പേര്;020_ഇതാണ്ASCII;021_ഇതാണ്ASCII;030_这是一个测试;031_这是一个测试";
TestKey passed: Software\zzz_test\这是一个测试
TestValue passed: 100_ASCII; これは値です;
TestValue passed: 101_ASCII; ASCII;
TestValue passed: 110_പേര്; これは値です;
TestValue passed: 111_പേര്; ASCII;
TestValue passed: 120_ഇതാണ്ASCII; これは値です;
TestValue passed: 121_ഇതാണ്ASCII; ASCII;
TestValue passed: 130_这是一个测试; これは値です;
TestValue passed: 131_这是一个测试; ASCII;
TestGetValueNames passed: Key: "Software\zzz_test\这是一个测试"; Expected "100_ASCII;101_ASCII;110_പേര്;111_പേര്;120_ഇതാണ്ASCII;121_ഇതാണ്ASCII;130_这是一个测试;131_这是一个测
TestGetKeyNames passed: Key: "Software\zzz_test"; Expected: "ASCII;这是一个测试";

Output on Linux:

TestKey passed: Software\zzz_test\ASCII
TestValue-02 failed: WriteString(000_ASCII_wrong,これは値です)
TestValue-02 failed: WriteString(001_ASCII_wrong,ASCII)
TestValue-02 failed: WriteString(010_പേര്_wrong,これは値です)
TestValue-02 failed: WriteString(011_പേര്_wrong,ASCII)
TestValue-02 failed: WriteString(020_ഇതാണ്ASCII_wrong,これは値です)
TestValue-02 failed: WriteString(021_ഇതാണ്ASCII_wrong,ASCII)
TestValue-02 failed: WriteString(030_这是一个测试_wrong,これは値です)
TestValue-02 failed: WriteString(031_这是一个测试_wrong,ASCII)
TestGetValueNames-01 failed: Key ""; Got: "000_ASCII_wrong;001_ASCII_wrong;010_പേര്_wrong;011_പേര്_wrong;020_ഇതാണ്ASCII_wrong;021_ഇതാണ്ASCII_wrong;030_这是一个测试_wrong;031_这是一个测试_wrong"; Expected: "000_ASCII;001_ASCII;010_പേര്;011_പേര്;020_ഇതാണ്ASCII;021_ഇതാണ്ASCII;030_这是一个测试;031_这是一个测试";
TestKey passed: Software\zzz_test\这是一个测试
TestValue-02 failed: WriteString(100_ASCII_wrong,これは値です)
TestValue-02 failed: WriteString(101_ASCII_wrong,ASCII)
TestValue-02 failed: WriteString(110_പേര്_wrong,これは値です)
TestValue-02 failed: WriteString(111_പേര്_wrong,ASCII)
TestValue-02 failed: WriteString(120_ഇതാണ്ASCII_wrong,これは値です)
TestValue-02 failed: WriteString(121_ഇതാണ്ASCII_wrong,ASCII)
TestValue-02 failed: WriteString(130_这是一个测试_wrong,これは値です)
TestValue-02 failed: WriteString(131_这是一个测试_wrong,ASCII)
TestGetValueNames-01 failed: Key ""; Got: "100_ASCII_wrong;101_ASCII_wrong;110_പേര്_wrong;111_പേര്_wrong;120_ഇതാണ്ASCII_wrong;121_ഇതാണ്ASCII_wrong;130_这是一个测试_wrong;131_这是一个测试_wrong"; Expected: "100_ASCII;101_ASCII;110_പേര്;111_പേര്;120_ഇതാണ്ASCII;121_ഇതാണ്ASCII;130_这是一个测试;131_这是一个测试";
TestGetKeyNames passed: Key: "Software\zzz_test"; Expected: "ASCII;这是一个测试";


I am working on a solution.

Michael Van Canneyt

2017-07-22 19:10

administrator   ~0101855

Fixed patch, and applied. Added console version of your test to examples.

Rolf Wetjen

2017-07-27 14:42

reporter   ~0101945

I see today that someone is heavily working on TRegistry.
I think that it's not a good idea to store unicode data in an xml file although I'm not an expert. So, maybe all string read and write routine can be moved from regitry.pp into the include files. Then windows can do unicode and xml can do utf8.

Michael Van Canneyt

2017-07-27 16:38

administrator   ~0101947

I don't see the problem. XML is by definition unicode.
One just needs to agree on the interface exposed by XMLreg.

Rolf Wetjen

2017-07-28 07:42

reporter   ~0101960

Hi Michael,
I've tested TRegistry with my test app and the modification to store strings in UFT8. The resulting reg.xml is UTF8. The first line is "<?xml version="1.0" encoding="utf-8" ?>"

Rolf

Rolf Wetjen

2017-07-28 07:43

reporter  

reg.xml (1,303 bytes)

Michael Van Canneyt

2017-07-28 08:00

administrator   ~0101962

@Rolf,

XML is unicode. Whether this is UTF16 ur UTF8 is irrelevant. The conversion is handled automatically.

And please stop reopening the issue if there is no bug.
he bugtracker is not the place to discuss.
If you want to discuss design, please mail me.
If there is an actual bug, then report it in a new bug.

Issue History

Date Modified Username Field Change
2017-07-22 09:16 Rolf Wetjen New Issue
2017-07-22 09:16 Rolf Wetjen File Added: TRegistry_UTF8.zip
2017-07-22 10:07 Michael Van Canneyt Assigned To => Michael Van Canneyt
2017-07-22 10:07 Michael Van Canneyt Status new => assigned
2017-07-22 11:02 Michael Van Canneyt Note Added: 0101846
2017-07-22 11:10 Thaddy de Koning Note Added: 0101847
2017-07-22 15:25 Rolf Wetjen Note Added: 0101850
2017-07-22 18:29 Michael Van Canneyt Note Added: 0101853
2017-07-22 19:10 Michael Van Canneyt Fixed in Revision => 36765
2017-07-22 19:10 Michael Van Canneyt Note Added: 0101855
2017-07-22 19:10 Michael Van Canneyt Status assigned => resolved
2017-07-22 19:10 Michael Van Canneyt Fixed in Version => 3.1.1
2017-07-22 19:10 Michael Van Canneyt Resolution open => fixed
2017-07-22 19:10 Michael Van Canneyt Target Version => 3.2.0
2017-07-27 14:42 Rolf Wetjen Note Added: 0101945
2017-07-27 14:42 Rolf Wetjen Status resolved => feedback
2017-07-27 14:42 Rolf Wetjen Resolution fixed => reopened
2017-07-27 16:38 Michael Van Canneyt Note Added: 0101947
2017-07-27 16:38 Michael Van Canneyt Status feedback => resolved
2017-07-27 16:38 Michael Van Canneyt Resolution reopened => fixed
2017-07-28 07:42 Rolf Wetjen Note Added: 0101960
2017-07-28 07:42 Rolf Wetjen Status resolved => feedback
2017-07-28 07:42 Rolf Wetjen Resolution fixed => reopened
2017-07-28 07:43 Rolf Wetjen File Added: reg.xml
2017-07-28 08:00 Michael Van Canneyt Note Added: 0101962
2017-07-28 08:00 Michael Van Canneyt Status feedback => resolved
2017-07-28 08:00 Michael Van Canneyt Resolution reopened => fixed
2017-07-29 08:49 Rolf Wetjen Status resolved => closed