View Issue Details

IDProjectCategoryView StatusLast Update
0029494LazarusInstallerpublic2016-09-28 21:37
ReporterYuriAssigned ToBart Broersma 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
Platformx86, x64OSWindowsOS VersionXP, 7, 8.1
Product Version1.6RC1Product Build1.6RC2 
Target Version1.6.2Fixed in Version1.6.2 
Summary0029494: Lazarus can't start if settings folder contains cyrillic symbols
DescriptionAfter usual installing Lazarus can't start if username in cyrrilic (tested on Russian versions of Windows XP, 7 (x64), 8.1 (x64)), "Unable to create file: C:\....\enviromentoptions.xml". Lazarus 1.4.4 have not this problem.
If used a Secondary installation of Lazarus whith latin only alphabet for installation path and settings path, then Lazarus works. If settings path contains cyrillic symbols - the error is the same.

Also if installation path contains cyrillic symbols, then start dialog can't set path to compiler: when choose a compiler, dialog can't find "fpc.cfg".

P.S.: sorry for ugly english translate.
Steps To ReproduceInstall Lazarus in Windows, usual if username contains cyrillic symbols, secondary otherwise (settings path must contains cyrillic symbols).
Try to launch.
TagsNo tags attached.
Fixed in Revisionr53047
LazTarget1.6.2
WidgetsetWin32/Win64
Attached Files
  • 2016-02-18 20_49_03.png (26,119 bytes)
    2016-02-18 20_49_03.png (26,119 bytes)
  • 2016-02-18 20_49_46.png (8,397 bytes)
    2016-02-18 20_49_46.png (8,397 bytes)
  • debug.log (17,266 bytes)
  • idecmdline.diff (769 bytes)
    Index: ide/idecmdline.pas
    ===================================================================
    --- ide/idecmdline.pas	(revision 52737)
    +++ ide/idecmdline.pas	(working copy)
    @@ -127,7 +127,13 @@
         for i := 0 to Cfg.Count - 1 do begin
           s := Cfg[i];
           if (s <> '') and (s[1] = '-') then
    -        ParamsAndCfgFileContent.Add(Trim(Cfg[i]))
    +        begin
    +          s := Trim(s);
    +          //cfg file is made by Windows installer and probably is Windows default codepage
    +          if FindInvalidUTF8Character(PChar(s), Length(s), True) > 0 then
    +            s := WinCPToUtf8(s);
    +          ParamsAndCfgFileContent.Add(s)
    +        end
           else
           if (Trim(s) <> '') and (s[1] <> '#') then
             Warn := Warn + IntToStr(i)+': ' + s + LineEnding;
    
    idecmdline.diff (769 bytes)

Relationships

related to 0029157 closedMattias Gaertner Lazarus can't save configuration if default primary config path contains non ASCII characters 
related to 0026696 resolvedBart Broersma Unable to install / run Lazarus IDE 

Activities

Yuri

2016-01-24 08:01

reporter  

Yuri

2016-01-24 10:00

reporter   ~0089288

The revision is 51246 (1.6RC2).

Bart Broersma

2016-02-06 16:51

developer   ~0089817

I cannot reproduce this with 1.6RC3 (r51531) nor trunk (r51439).
Tested on Win7, with username being "บลูเบอ".
Note that these caracters fall outside of my current codepage (Western European).
Lazarus correctly creates the C:\Users\บลูเบอ\AppData\lazarus folder and writes it's config files there.

Q1: are cyrillic characters inside your current windows codepage?
Q2: did you use the DisableUtf8RTL feature?

Yuri

2016-02-16 20:16

reporter   ~0090082

Answer 1: I think, that the characters inside codepage 1251, because it's Windows default codepage (Russian). The folders was created by the system ('c:\users\Юрий') or in the Explorer (for testing secondary installation).

Answer 2: If I correctly understand, DisableUtf8RTL feature is used in a project options. But this error prevents Lazarus installation. Therefore, DisableUtf8RTL feature may not be affected by this bug. Correct me please if I am wrong.

If needed, I can provide remote access in a few days to test this error. I would be happy to help any way I can.

Bart Broersma

2016-02-16 23:14

developer   ~0090086

DisableUtf8RTL will probably make it impossible for Lazarus to create it's default config folder _if_ the usernamehas unicode characters outside the current codepage.
Note also that when username has unicodecharacters outside the current codepage, the compiler will not be able to compile the project,

As a (temporary) workaround you can set --pcp=somepathwithonlyasciicharacters.

Yuri

2016-02-18 19:37

reporter   ~0090118

Thank You for this workaround.

Lazarus 1.6 (r51630) is released! But the bug is stil here :-(
Tested on Windows 7 x86, secondary installation. Lazarus path = 'c:\Лаз\', config path = 'c:\Лаз\Сет\'.

Yuri

2016-02-18 19:38

reporter  

2016-02-18 20_49_03.png (26,119 bytes)
2016-02-18 20_49_03.png (26,119 bytes)

Yuri

2016-02-18 19:39

reporter  

2016-02-18 20_49_46.png (8,397 bytes)
2016-02-18 20_49_46.png (8,397 bytes)

Yuri

2016-02-27 22:52

reporter   ~0090418

I can give you remote access to the (virtual) machine for all those who would like to test this bug. Does this help anyone?

Bart Broersma

2016-02-27 23:37

developer   ~0090419

Last edited: 2016-02-27 23:37

View 2 revisions

@Yuri: not me.

Michl

2016-03-07 18:58

developer   ~0090741

I've tested it today too. I can't start a Lazarus installed in a directory C:\Lazö.

Windows 7 64bit, Lazarus 1.6.0 32bit.

Dubug.log added.

Michl

2016-03-07 18:59

developer  

debug.log (17,266 bytes)

Bart Broersma

2016-03-07 19:06

developer   ~0090743

It's not about the Lazarus directory, but about the config directory.
The compiler an tools (make etc.) may very well not like C:\Lazö, but Lazarus iteslef should have no diffuculty writing it's confi files to e.g. C:\Users\Lazöäïü\AppData\local\lazarus.

user268

2016-03-07 19:35

  ~0090746

Last edited: 2016-03-07 19:47

View 3 revisions

Bart, it seems to me that conversion from special dir to unicode string used for config file missing. You may check this first.

BTW, why conversion need to be forced anyway with SysToUTF8 or UTF8Encode? Global problem here made different main codepage by different OS. However, I do not see the reason to force programmer to do that. When use load or save method for any library, it should be assumed that is in UTF8 or widestring and make these methods overloaded dependent of used OS. For these OS not use any unicode, it is assumed to use ANSI string only.

And where such path info is stored? File have BOM sequence? If not, what content using BOM or not should be treated on different OS?

AnsiString, String, UnicodeString, WideString... + different OS unicode support - all makes quite a confusion by default.

I do not want to argue here at all - it is simply much more logical instead to force programmer to do such conversions and obviously similar errors are unavoidable.

Michl

2016-03-07 19:45

developer   ~0090747

> It's not about the Lazarus directory, but about the config directory.

OK, i missunderstood the topic. A config directory á la C:\Lazarus\ConfigÖ works fine here.

Bart Broersma

2016-03-08 00:09

developer   ~0090749

@sasa: see my note 0089817.
Lazarus can handle the C:\Users\บลูเบอ\AppData\Local\lazarus path were it stores it's config.
Even though this path contains unicode characters that are outside my current codepage.
So, it's a mystery to me why this does not work with Cyrillic or Danish characters.

user268

2016-03-08 04:53

  ~0090755

I have read it already and my answer is the same as Yuri's.

Old Windows (as 95) does not supported unicode and based purely on default codepage, thus programs are also made for specific codepage or English codepage only. That made quite a confusion. From Windows2000 (or so), they started with full unicode support, however, localization stays as codepage default settings.

That is: if OS default codepage is set to 1251 (Cyrillic) i.e. Russian localization, all files wrote by Russian cyrillic internally are recognized as such (one byte character). However, Lazarus seems to do or not some improper conversion or/and triggering use of ANSI or widestring API or similar calls...

And that may be reason why is not possible to install Lazarus in path uses specific codepage. Just speculations...

When test such condition, suggestion is to use full unicode text, mixed different codepage characters than OS is localized.

For instance: Юрий_ŠĐČĆ

Upper have CP1251 and CP1250 characters (you may add Greece chars, etc), ensure that conversion should be handled as unicode, not by specific codepage...

The cause of the problem then should be found fast and without doubts.

Bart Broersma

2016-04-24 12:10

developer   ~0092171

> all files wrote by Russian cyrillic internally are recognized as such (one byte character)

That's about the contents of the file.
The issue is about the filename, which AFAIK Windows internally only uses widestrings.

Bart Broersma

2016-04-24 12:25

developer   ~0092172

Lazarus retrieves the default location where it stores its configfiles using GetAppConfigDirUtf8 function (in LazFileUtils unit):

PrimaryConfigPath:= ExtractFilePath(ChompPathDelim(GetAppConfigDirUTF8(False))) + 'lazarus';

Can you inspect what the content is of GetAppConfigDirUtf8(False) on your system?
It should return a filename encoded in UTF8, so please inspect if it is properly encoded.

The wrting of the xml file in the end is done by Laz2_XMLWrite.WriteXMLFile which expects an UTF8 encoded filename and uses a TFileStreamUTF8 which then translates the UTF8 to WideString.

Yuri

2016-04-28 19:03

reporter   ~0092278

When I launch program from Lazarus LazFileUtils.GetAppConfigDirUtf8(False) return this text: "D:\mb\_Test_2\". This is the project path.
When I launch compiled program directly: "C:\Users\Юрий\AppData\Local\project1\". This is correct path.

Bart Broersma

2016-04-29 12:21

developer   ~0092287

> "C:\Users\Юрий\AppData\Local\project1\".
And exactly that C:\Users\Юрий\AppData\Local\ is used by the IDE to to determine where it stores it's config files.

Yuri

2016-04-30 11:47

reporter   ~0092312

Last edited: 2016-05-01 20:27

View 2 revisions

I think that the error is not in this function. The problem arises when I use the secondary installation, if I specify a path that contains Cyrillic letters.

Bart Broersma

2016-05-01 23:24

developer   ~0092327

> if I specify a path that contains Cyrillic letters.
How did you specify the path?
On commandline or in the .lnk file (shortcut file)

Yuri

2016-05-02 08:38

reporter   ~0092329

> How did you specify the path?
In Lazarus secondary installation dialog box (selecting path to config files).

Yuri

2016-05-04 09:27

reporter   ~0092387

Last edited: 2016-05-04 10:35

View 2 revisions

I found the problem. Partially. After secondary installation in the Lazarus folder created file "lazarus.cfg". This file contains the path to the settings file. The contents of my file:
--primary-config-path=C:\Лаз\Сет

The file is stored in the system default encoding. CP-1251 in my case. In this case Lazarus is started with an error. If the same content stored in the UTF8 encoding, then everything works fine. In these folders are stored configuration files. Specify the path through addind "--primary-config-path" to the .lnk is works fine. Through the commanline is also works.

I think that a similar problem arises in the normal installation when using the default path. Encoding perceived wrong.

Bart Broersma

2016-07-02 14:35

developer   ~0093481

The lazarus.cfg is created by the installer?
(I only build from sources and I have no lazarus.cfg anywhere)

Bart Broersma

2016-07-05 18:47

developer   ~0093559

Thoughts (assuming we can't get the installer to write UTF8);

Do we (Lazarus) ever write to lazarus.cfg file?
If not (only the installer writes), then on Windows we can simply change the encoding from WinCP to UTF8 upon reading the file (in IDECmdLine unit)?

If we also do write ourselves, then upon applying the read information we could check if the directory exists and if not assume the string is actually Win codepage, convert it to UTF8 and check again?

Or: read lazarus cfg, if the text contains any char > 127 and it is not valid UTF8, convert all the text to UTF8, save it as UTF8 and be done with it?

Bart Broersma

2016-07-30 16:06

developer  

idecmdline.diff (769 bytes)
Index: ide/idecmdline.pas
===================================================================
--- ide/idecmdline.pas	(revision 52737)
+++ ide/idecmdline.pas	(working copy)
@@ -127,7 +127,13 @@
     for i := 0 to Cfg.Count - 1 do begin
       s := Cfg[i];
       if (s <> '') and (s[1] = '-') then
-        ParamsAndCfgFileContent.Add(Trim(Cfg[i]))
+        begin
+          s := Trim(s);
+          //cfg file is made by Windows installer and probably is Windows default codepage
+          if FindInvalidUTF8Character(PChar(s), Length(s), True) > 0 then
+            s := WinCPToUtf8(s);
+          ParamsAndCfgFileContent.Add(s)
+        end
       else
       if (Trim(s) <> '') and (s[1] <> '#') then
         Warn := Warn + IntToStr(i)+': ' + s + LineEnding;
idecmdline.diff (769 bytes)

Bart Broersma

2016-07-30 16:07

developer   ~0093905

Can you please test with attached patch (idecmdline.diff)?

Juha Manninen

2016-09-14 18:13

developer   ~0094652

Yuri, ping ...
Also others please test the patch.

Bart Broersma

2016-09-28 16:10

developer   ~0094854

Please test and close if OK.

Issue History

Date Modified Username Field Change
2016-01-24 08:01 Yuri New Issue
2016-01-24 08:01 Yuri File Added: 2016-01-24 09_08_32-Lazarus.png
2016-01-24 09:22 Ondrej Pokorny Relationship added duplicate of 0029157
2016-01-24 10:00 Yuri Note Added: 0089288
2016-01-24 13:22 Ondrej Pokorny Relationship replaced related to 0029157
2016-02-06 16:51 Bart Broersma Note Added: 0089817
2016-02-06 16:52 Bart Broersma LazTarget => -
2016-02-06 16:52 Bart Broersma Status new => feedback
2016-02-16 20:16 Yuri Note Added: 0090082
2016-02-16 20:16 Yuri Status feedback => new
2016-02-16 23:14 Bart Broersma Note Added: 0090086
2016-02-18 19:37 Yuri Note Added: 0090118
2016-02-18 19:38 Yuri File Added: 2016-02-18 20_49_03.png
2016-02-18 19:39 Yuri File Added: 2016-02-18 20_49_46.png
2016-02-27 22:52 Yuri Note Added: 0090418
2016-02-27 23:37 Bart Broersma Note Added: 0090419
2016-02-27 23:37 Bart Broersma Note Edited: 0090419 View Revisions
2016-03-07 15:52 Bart Broersma Relationship added related to 0026696
2016-03-07 18:58 Michl Note Added: 0090741
2016-03-07 18:59 Michl File Added: debug.log
2016-03-07 19:06 Bart Broersma Note Added: 0090743
2016-03-07 19:35 user268 Note Added: 0090746
2016-03-07 19:45 Michl Note Added: 0090747
2016-03-07 19:46 user268 Note Edited: 0090746 View Revisions
2016-03-07 19:47 user268 Note Edited: 0090746 View Revisions
2016-03-08 00:09 Bart Broersma Note Added: 0090749
2016-03-08 04:53 user268 Note Added: 0090755
2016-04-24 12:10 Bart Broersma Note Added: 0092171
2016-04-24 12:25 Bart Broersma Note Added: 0092172
2016-04-24 12:25 Bart Broersma Status new => feedback
2016-04-28 19:03 Yuri Note Added: 0092278
2016-04-28 19:03 Yuri Status feedback => new
2016-04-29 12:21 Bart Broersma Note Added: 0092287
2016-04-30 11:47 Yuri Note Added: 0092312
2016-05-01 20:27 Yuri Note Edited: 0092312 View Revisions
2016-05-01 23:24 Bart Broersma Note Added: 0092327
2016-05-02 08:38 Yuri Note Added: 0092329
2016-05-04 09:27 Yuri Note Added: 0092387
2016-05-04 10:35 Yuri Note Edited: 0092387 View Revisions
2016-05-04 11:12 Juha Manninen Category IDE => Installer
2016-07-02 14:35 Bart Broersma Note Added: 0093481
2016-07-05 18:47 Bart Broersma Note Added: 0093559
2016-07-30 16:06 Bart Broersma File Added: idecmdline.diff
2016-07-30 16:07 Bart Broersma Note Added: 0093905
2016-07-30 16:07 Bart Broersma Status new => feedback
2016-09-14 18:13 Juha Manninen Note Added: 0094652
2016-09-28 16:10 Bart Broersma Fixed in Revision => r53047
2016-09-28 16:10 Bart Broersma LazTarget - => 1.6.2
2016-09-28 16:10 Bart Broersma Note Added: 0094854
2016-09-28 16:10 Bart Broersma Status feedback => resolved
2016-09-28 16:10 Bart Broersma Fixed in Version => 1.6.2
2016-09-28 16:10 Bart Broersma Resolution open => fixed
2016-09-28 16:10 Bart Broersma Assigned To => Bart Broersma
2016-09-28 16:10 Bart Broersma Target Version => 1.6.2