View Issue Details

IDProjectCategoryView StatusLast Update
0018193FPCRTLpublic2021-04-22 16:01
Reporter周勇 Assigned ToMarco van de Voort  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
PlatformwindowsOSwindows7 
Summary0018193: Initialization of CurrencyString In sysutils has a bug
Descriptionsource code(line no 688):
CurrencyString:=GetLocaleStr(LID, LOCALE_SCURRENCY, '');

"GetLocaleStr(LID, LOCALE_SCURRENCY, '');" return UTF16 string, so it's error when assigned to string. I'am chinese,CurrencyString must be '¥', but CurrencyString display '?'. If Source code change to " CurrencyString:=UTF16toUTF8(GetLocaleStr(LID, LOCALE_SCURRENCY, '')); " then CurrencyString is '¥'.
TDBEdit is not displayed correctly when 'FieldType' is currency;
TagsNo tags attached.
Fixed in Revision49244
FPCOldBugId
FPCTarget-
Attached Files

Relationships

related to 0015914 closedFelipe Monteiro de Carvalho Lazarus FloatToStrF(12345.67,ffCurrency,10,2) produces "?" instead of ThousandSeparator [renamed] 

Activities

周勇

2010-12-11 17:18

reporter   ~0044179

lazarus:0.9.29 fpc:2.4.2

周勇

2010-12-11 17:33

reporter   ~0044181

It can't invoke UTF16toUTF8 In fcl

Vincent Snijders

2010-12-13 13:10

manager   ~0044201

While Jonas thinks this is an issue with the TDBEdit, I don't think this issue can be fixed without help in the sysutils (support for UTF8 encoded strings) or the db units.

Vincent Snijders

2010-12-13 13:12

manager   ~0044202

Can you create an example application? I think that it won't work for euro either, because the euro sign is not a ascii character either.

LacaK

2010-12-14 07:43

developer   ~0044216

Similar problem was discussed also in http://bugs.freepascal.org/view.php?id=15914

IMHO problem exists for all data-aware controls (TDBGrid, TDBEdit), becasue they expect UTF8 encoded strings, but CurrencyString (and also ThousandSeparator etc.) are strings encoded in current system locale - ansistrings (case of GetLocaleInfoA) or are encoded in UNICODE-UTF16 - widestrings (case GetLocaleInfoW).

So some kind of translation must be done!

And yes, for Euro symbol (ord=128) it does not work also.

2010-12-14 08:37

 

lazarus test.zip (135,090 bytes)

周勇

2010-12-14 08:38

reporter   ~0044219

1.jpg in test.zip is result of runing.

Marco van de Voort

2011-03-23 14:59

manager   ~0046767

(The RTL/FCL defines ansistrings to be the default codepage of the system. The UTF-8 convention is a Lazarus convention. So the FPC code is correct from a FPC view)

Zeljan Rikalo

2012-02-04 13:36

reporter   ~0056404

Not blocker, so postponed.

Ihor Lavrynyuk

2012-05-05 12:38

reporter   ~0059255

Last edited: 2012-05-05 12:40

have some problem with "?"
win 7
TDBEdit
Lazarus 1.1 Rev 37156 FPC 2.6.1
db sqlite3
Zeos Component ZEOSLIB_REV943

Ihor Lavrynyuk

2012-05-08 19:13

reporter   ~0059409

look what I found ... http://www.youtube.com/watch?v=69xekZK3gKc

Juha Manninen

2021-03-23 10:02

reporter   ~0129833

Ihor, what does it mean? Is the problem gone?
The report is over 10 years old and was made for Lazarus 0.9.29.

Marco van de Voort

2021-03-23 10:16

manager   ~0129834

Last edited: 2021-03-23 10:16

View 2 revisions

His comment is also 8 years old? You probably got an event because Lacak stopped monitoring

LacaK

2021-03-23 13:26

developer   ~0129840

Yes right, I stopped monitoring. Still this issue is "strange". What I can say is:
In Initialization section of SysUtils is called -> InitInternational -> GetFormatSettings -> GetLocaleFormatSettings -> GetLocaleStr(LID, LOCALE_SCURRENCY, '') -> GetLocaleInfoA

Small test application in Lazarus under Windows, where I use:
  showmessageFmt('CurrencyString CP=%d %s',[StringCodePage(CurrencyString), CurrencyString]);
  GetFormatSettings;
  showmessageFmt('CurrencyString CP=%d %s',[StringCodePage(CurrencyString), CurrencyString]);

Gives results:
  CP=65001 € <-- right result
  CP=65001 <-- wrong result (Euro symbol is missing)

Why after initialization is CurrencyString okay and after call to GetFormatSettings is no more okay?

LacaK

2021-03-23 14:15

developer   ~0129842

Response to my question:
There is called in TApplication.IntfSettingsChange -> GetFormatSettingsUTF8 which calls GetLocaleInfoW and handles WideChar to UTF-8 conversion.

So in Lazarus is it from my POV fixed.
Question is: can it be fixed also in SysUtils (for example use GetLocaleInfoW instead of GetLocaleInfoA and do correct translation to DefaultSystemCodePage) ?

Juha Manninen

2021-03-23 19:31

reporter   ~0129843

> His comment is also 8 years old?

Oops yes. I should be more careful.
If the remaining issue is in SysUtils, should we move this to FPC project or maybe resolve?

LacaK

2021-03-24 07:49

developer   ~0129846

I think that yes -> move to FPC (original report is also about SysUtils)

Juha Manninen

2021-03-24 07:59

reporter   ~0129847

Moved.

LacaK

2021-03-24 08:22

developer   ~0129848

Last edited: 2021-03-24 08:27

View 2 revisions

If I change function GetLocaleStr in rtl/win/SysUtils.pp as follows:

function GetLocaleStr(LID, LT: Longint; const Def: string): AnsiString;
var
  L: Integer;
  Buf: array[0..255] of WideChar;
begin
  L := GetLocaleInfoW(LID, LT, Buf, Length(Buf));
  if L > 0 then
    widestringmanager.Wide2AnsiMoveProc(Buf, Result, DefaultSystemCodePage, L-1)
  else
    Result := Def;
end;

then I get correct result.
(I am not sure if it is acceptable solution, but it works for me)

Marco van de Voort

2021-04-17 21:15

manager   ~0130436

Hmm. Then maybe we skip the whole fixed buffer stuff ? This is not called constantly, so better do two calls for the buffer size. The question is only if it is allowed to overwrite the final zero character

function GetLocaleStr(LID, LT: Longint; const Def: string): AnsiString;
var
  L: Integer;
  Buf: unicodestring;
begin
  L := GetLocaleInfoW(LID, LT, nil, 0);
  if L > 0 then
    begin
      SetLength(Buf,L-1);
      L := GetLocaleInfoW(LID, LT, @Buf[1], L); // writes additional zero
      result:=L; // convert to defaultcodepage.
    end
  else
    Result := Def;
end;

LacaK

2021-04-20 09:06

developer   ~0130467

@Marco, I took code snippet from LazUtf8 (winlazutf8.inc), where is used WideChar buffer ... is any problem with this buffer (is allocated on stack, does not?)

In your code result:=L; should be Result:=Buf; does not? :-)
Otherwise I don't see a problem if unicode strings are terminated always by NULL widechar (two 0 bytes), so SetLength allocates L*2 bytes.

Marco van de Voort

2021-04-20 09:07

manager   ~0130468

result:=buff; :yes.

static buffer: maximum length. It is not good to build in all kinds of fixed lengths into the RTL.

Thaddy de Koning

2021-04-20 09:14

reporter   ~0130469

Isn't this the same recurring issue?
Given a UTF8 string and a char count, the buffer should be nbr of chars * four?

Sven Barth

2021-04-20 09:36

manager   ~0130470

@Thaddy: GetLocaleInfoW returns the necessary amount of WideChars to fullfill the request if the provided buffer size is too small. No need to fiddle with UTF-8 or surrogate pairs.

LacaK

2021-04-20 10:36

developer   ~0130471

I am okay with any implementation which makes things better :-)

(btw: I still do not understand where is problem with 256 WideChar buffer? You think, that it may happens that requested LocaleInfo will be longer than 255 WideChars?)

Sven Barth

2021-04-21 09:29

manager   ~0130491

@LacaK: and what if? That's the point: Windows provides the ability to handle this gracefully, so it should be used. There is simply no need to use static buffers in this case.

ravi dion

2021-04-21 10:25

reporter   ~0130492

"Note For interoperability reasons, the application should prefer the GetLocaleInfoEx function to GetLocaleInfo because Microsoft is migrating toward the use of locale names instead of locale identifiers for new locales. Any application that runs only on Windows Vista and later should use GetLocaleInfoEx."
https://docs.microsoft.com/en-us/windows/win32/api/winnls/nf-winnls-getlocaleinfow

Thaddy de Koning

2021-04-21 13:21

reporter   ~0130495

GetLocaleInfoW is platfform, Sven......

Sven Barth

2021-04-21 13:59

manager   ~0130496

@Thaddy: And? We're talking about platform specific functionality anyway.

LacaK

2021-04-22 12:02

developer   ~0130509

@Sven, okay I agree - let's implement it ...

(Only last small note: write L widechars (null terminated) to to L-1 length unicode string is for me not very intuitive. It depends on fact, that allocated memory for dynamic string always includes +1 char for zero termination, which is from my POV implementation detail)

Sven Barth

2021-04-22 13:42

manager   ~0130512

It's documented that Ansi- and UnicodeString end in #0.

Marco van de Voort

2021-04-22 16:01

manager   ~0130517

Ok, committed the earlier code, please test

Issue History

Date Modified Username Field Change
2010-12-11 17:16 周勇 New Issue
2010-12-11 17:18 周勇 Note Added: 0044179
2010-12-11 17:33 周勇 Note Added: 0044181
2010-12-13 11:54 Jonas Maebe Project FPC => Lazarus
2010-12-13 13:10 Vincent Snijders Note Added: 0044201
2010-12-13 13:10 Vincent Snijders LazTarget => -
2010-12-13 13:10 Vincent Snijders Assigned To => Joost van der Sluis
2010-12-13 13:10 Vincent Snijders Status new => assigned
2010-12-13 13:12 Vincent Snijders Note Added: 0044202
2010-12-13 13:12 Vincent Snijders Status assigned => feedback
2010-12-14 07:43 LacaK Note Added: 0044216
2010-12-14 08:37 周勇 File Added: lazarus test.zip
2010-12-14 08:38 周勇 Note Added: 0044219
2010-12-14 15:36 Vincent Snijders Relationship added related to 0015914
2010-12-14 15:37 Vincent Snijders LazTarget - => 1.0
2010-12-14 15:37 Vincent Snijders Status feedback => acknowledged
2010-12-14 15:37 Vincent Snijders Target Version => 1.0.0
2011-03-23 14:59 Marco van de Voort Note Added: 0046767
2012-02-04 13:36 Zeljan Rikalo LazTarget 1.0 => 1.2
2012-02-04 13:36 Zeljan Rikalo Note Added: 0056404
2012-03-13 07:50 Vincent Snijders Target Version 1.0.0 => 1.2.0
2012-05-05 12:38 Ihor Lavrynyuk Note Added: 0059255
2012-05-05 12:38 Ihor Lavrynyuk Note Edited: 0059255
2012-05-05 12:39 Ihor Lavrynyuk Note Edited: 0059255
2012-05-05 12:40 Ihor Lavrynyuk Note Edited: 0059255
2012-05-08 19:13 Ihor Lavrynyuk Note Added: 0059409
2014-01-14 15:10 Martin Friebe LazTarget 1.2 => 1.4
2014-01-14 15:12 Martin Friebe Target Version 1.2.0 => 1.4
2014-09-10 11:41 Juha Manninen LazTarget 1.4 => -
2014-09-10 11:41 Juha Manninen Target Version 1.4 =>
2014-12-02 09:48 Joost van der Sluis Assigned To Joost van der Sluis =>
2014-12-02 09:48 Joost van der Sluis Status acknowledged => new
2021-03-23 10:02 Juha Manninen Note Added: 0129833
2021-03-23 10:16 Marco van de Voort Note Added: 0129834
2021-03-23 10:16 Marco van de Voort Note Edited: 0129834 View Revisions
2021-03-23 13:26 LacaK Note Added: 0129840
2021-03-23 14:15 LacaK Note Added: 0129842
2021-03-23 19:31 Juha Manninen Note Added: 0129843
2021-03-24 07:49 LacaK Note Added: 0129846
2021-03-24 07:58 Juha Manninen Project Lazarus => FPC
2021-03-24 07:59 Juha Manninen Note Added: 0129847
2021-03-24 08:19 LacaK Category FCL => RTL
2021-03-24 08:19 LacaK FPCTarget => -
2021-03-24 08:22 LacaK Note Added: 0129848
2021-03-24 08:27 LacaK Note Edited: 0129848 View Revisions
2021-04-17 21:15 Marco van de Voort Note Added: 0130436
2021-04-20 09:06 LacaK Note Added: 0130467
2021-04-20 09:07 Marco van de Voort Note Added: 0130468
2021-04-20 09:14 Thaddy de Koning Note Added: 0130469
2021-04-20 09:36 Sven Barth Note Added: 0130470
2021-04-20 10:36 LacaK Note Added: 0130471
2021-04-21 09:29 Sven Barth Note Added: 0130491
2021-04-21 10:25 ravi dion Note Added: 0130492
2021-04-21 13:21 Thaddy de Koning Note Added: 0130495
2021-04-21 13:59 Sven Barth Note Added: 0130496
2021-04-22 12:02 LacaK Note Added: 0130509
2021-04-22 13:42 Sven Barth Note Added: 0130512
2021-04-22 16:01 Marco van de Voort Fixed in Revision => 49244
2021-04-22 16:01 Marco van de Voort Assigned To => Marco van de Voort
2021-04-22 16:01 Marco van de Voort Status new => resolved
2021-04-22 16:01 Marco van de Voort Resolution open => fixed
2021-04-22 16:01 Marco van de Voort Note Added: 0130517