View Issue Details

IDProjectCategoryView StatusLast Update
0027099FPCRTLpublic2015-02-18 20:44
ReporterMattias Gaertner Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
Product Version2.7.1 
Summary0027099: unit clocale creates invalid characters on Russian Linux
Descriptionunit clocale sets FormatSettings by asking nl_langinfo.
The "char" variables are set by using the first byte of the locale strings.

On Linux the thousand separator for Russian ru_RU.UTF-8 (LC_NUMERIC=ru_RU.utf8) is #$C2$A0 - a two byte in UTF-8. This results in the invalid character #$C2.

The attached patch checks for non ASCII character and CP_UTF8, so that the default is used.
Steps To Reproduceexport LC_NUMERIC=ru_RU.utf8

{$mode ObjFPC}
uses Classes, SysUtils, clocale;
begin
  writeln(ThousandSeparator);
end;
TagsNo tags attached.
Fixed in Revision
FPCOldBugId0
FPCTarget
Attached Files

Relationships

related to 0027086 new FormatSettings do not support changing DefaultSystemCodePage 

Activities

Zeljan Rikalo

2014-12-04 13:33

reporter   ~0079639

@Mattias, patch is for lazarus , not for fpc as I can see.

Mattias Gaertner

2014-12-04 13:40

manager  

clocale.patch (519 bytes)   
Index: packages/rtl-extra/src/unix/clocale.pp
===================================================================
--- packages/rtl-extra/src/unix/clocale.pp	(revision 29116)
+++ packages/rtl-extra/src/unix/clocale.pp	(working copy)
@@ -108,6 +108,8 @@
   function GetLocaleChar(item: cint): char;
   begin
     GetLocaleChar := nl_langinfo(item)^;
+    if (ord(GetLocaleChar)>127) and (DefaultSystemCodePage=CP_UTF8) then
+      Result:=#0;
   end;
 
   function SkipModifiers(const s: string; var i: integer): string;
clocale.patch (519 bytes)   

Mattias Gaertner

2014-12-04 13:44

manager   ~0079640

Sorry. I uploaded the right patch.

Mattias Gaertner

2015-02-18 20:43

manager  

clocale_with_mapping.patch (1,476 bytes)   
Index: packages/rtl-extra/src/unix/clocale.pp
===================================================================
--- packages/rtl-extra/src/unix/clocale.pp	(revision 29753)
+++ packages/rtl-extra/src/unix/clocale.pp	(working copy)
@@ -106,8 +106,41 @@
   end;
 
   function GetLocaleChar(item: cint): char;
+  var
+    p: PChar;
   begin
-    GetLocaleChar := nl_langinfo(item)^;
+    p := nl_langinfo(item);
+    Result := p^;
+    if (ord(Result)>127) and (DefaultSystemCodePage=CP_UTF8) then begin
+      Result := #0;
+      case p^ of
+      #$C2:
+        case p[1] of
+        #$A0: Result:=' '; // non breakable space
+        #$B7: Result:='.'; // middle stop
+        end;
+      #$CB:
+        if p[1]=#$99 then Result:=''''; // dot above, italian handwriting
+      #$D9:
+        case p[1] of
+        #$AB: Result:=','; // arabic decimal separator, persian thousand separator
+        #$AC: Result:=''''; // arabic thousand separator
+        end;
+      #$E2:
+        case p[1] of
+        #$80:
+          case p[2] of
+          #$82, // long space
+          #$83, // long space
+          #$89, // thin space
+          #$AF: // narrow non breakable space
+            Result := ' ';
+          #$94: Result := '-'; // persian decimal mark
+          end;
+        #$8E: if p[2]=#$96 then Result := ''''; // codepoint 9110 decimal separator
+        end;
+      end;
+    end;
   end;
 
   function SkipModifiers(const s: string; var i: integer): string;
clocale_with_mapping.patch (1,476 bytes)   

Mattias Gaertner

2015-02-18 20:44

manager   ~0081222

I improved the patch with a mapping for common characters.

Issue History

Date Modified Username Field Change
2014-11-27 14:12 Mattias Gaertner New Issue
2014-11-27 14:12 Mattias Gaertner File Added: utf8argv.patch
2014-12-04 13:33 Zeljan Rikalo Note Added: 0079639
2014-12-04 13:39 Mattias Gaertner File Deleted: utf8argv.patch
2014-12-04 13:40 Mattias Gaertner File Added: clocale.patch
2014-12-04 13:44 Mattias Gaertner Note Added: 0079640
2014-12-08 12:15 Marco van de Voort Relationship added related to 0027086
2015-02-18 20:43 Mattias Gaertner File Added: clocale_with_mapping.patch
2015-02-18 20:44 Mattias Gaertner Note Added: 0081222