View Issue Details

IDProjectCategoryView StatusLast Update
0012676LazarusLCLpublic2009-06-13 12:09
ReporterWD Assigned ToFelipe Monteiro de Carvalho  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionno change required 
PlatformWindows VistaOSWindows Vista 
Product Version0.9.27 (SVN) 
Summary0012676: Memo.LoadFromFile(file.txt) does not work if file contains german characters (ä,ö,ü,ß etc.)
DescriptionIf i try to load a german textfile (*.txt or *.csv) with special german characters like ü, ä, ö or ß in it in a TMemo (Form1.Memo1.Lines.LoadFromFile(textfile.txt);) there's nothing displayed and Lines.Count gives back "0" - so the Memo still seems to be empty.

If i copy the same text from Windows-Editor into the Memo by using the Clipboard the text is displayed with all special characters.
I can save the text and reload ist by Lines.LoadFromFile.

Loading a text without german characters works fine.
Steps To ReproduceI take a Form like Form1
I paint a TMemo, a button and an OpenDialog-Symbol on it
The button has the function:

if Form1.OpenDialog1.Execute = TRUE then
begin
  if FileExists(Form1.OpenDialog1.Filename) = TRUE then
  begin
    Form1.Memo1.Lines.LoadFromFile(Form1.OpenDialog1.Filename);
  end else
  begin
    ShowMessage('File does not exist!');
  end;
end;

I select a *.txt File using german characters like ä, ö, ü ... made with any Windows-Program that creates Textfiles (DATEV-GDPdU-Export-Function, Editor etc.)

The Loading-Process finishes without any Exception or Error-Message.

Nothing is displayed in the Memo and a lines.Count-Function gives the result = 0.
Additional InformationIf you have a solution for that i will thank you very much :-).
TagsNo tags attached.
Fixed in Revision
LazTarget-
WidgetsetWin32/Win64
Attached Files

Activities

Vincent Snijders

2008-11-20 08:57

manager   ~0023452

Is the file UTF8 encoded?

If not, load the file into a stringlist, UTF8Encode each line and assign it to the memo.lines.

Graeme Geldenhuys

2008-11-20 09:29

reporter   ~0023453

@Vincent
Yuck! :-( FPC really needs Unicode support built in. Adding such hacks all over our programs are really not ideal.

Vincent Snijders

2008-11-20 09:37

manager   ~0023454

Graeme, fixing TStrings.LoadFromFile is not as easy as it seems. Should TStrings contain ansistrings, UTF8 strings, UTF16 strings? Should it convert files to that encoding, or just assume the file it loads has the encoding of TStrings? etc.

Graeme Geldenhuys

2008-11-20 09:53

reporter   ~0023455

Well using UTF8Encode as your "hack" suggests, you are already assuming the file's content is UTF-8 encoded. But yes I understand the issue - it is a bit more complex as first perceived. I'll discuss this further in the FPC-Devel mailing list, I'm busy writing that message now.

Felipe Monteiro de Carvalho

2008-12-05 03:13

developer   ~0023675

This is not a bug. Lazarus supports only UTF-8. If you load anything into a LCL Component it must be encoded in UTF-8. If it isn't already, convert it.

This isn't a hack, it's just the logical working of this.

WD

2008-12-09 18:02

reporter   ~0023734

It's tragically for german users, that german text with special german characters like ä,ö,ü... can not be loaded by using standard-functions of LCL-Components like Memo.loadfromfile.

I tried a program with a memo-component on it by these functions:

First step: Load a text into the Memo-component:

filename:=Form1.OpenDialog1.Filename;
sl:=TStringList.Create;
sl.LoadFromFile(filename);
for a:=0 to sl.count-1 do
  sl[a]:=utf8encode(sl[a]);
form1.memo1.lines.assign(sl); (same result: form1.memo1.lines.Add(sl[a]);)

I load a file with german ä and ü by using this function. It is displayed in the correct way.

Now i save the file with this procedure:

filename:=Form1.SaveDialog1.FileName;
Form1.Memo1.Lines.SaveToFile(filename);

If I reload this new file again then the special characters are not displayed correctly.

So Lazarus can't reload text that contains german characters when it was saved with a lazarus Memo-Component with the utf8-Conversion Function again?

I don't know how to handle that problem. A simple text-editor seems to be impossible.

In the older lazarus-versions this was never a problem.

Vincent Snijders

2008-12-09 19:21

manager   ~0023736

You changed the encoding of the file by saving it, not it is UTF8 encoded, and you must not UTF8Encode it again when reading it.

So either convert the file to ansi again before saving or don't UTF8Encode it, when loading again.

WD

2009-02-02 17:02

reporter   ~0025024

Thank you for your help in loading and saving german textfiles.
It's very difficult to reconstruct a bigger project with UTF8Encode and UTF8toAnsi - Functions.

I don't know how Lazarus detects, if a file is UTF8Encoded or in ANSI-Format.

Now i recived a unicode-text-file to load in my program - and the function with "ansitoutf8" logically doesn't work propperly.

All load- and save-functions should be able to see, in which format the file is.
And then there must be two different load-functions. Is that right? Or more if there is a UTF16-File and so on?

Perhaps you can help me. In the earlier versions of lazarus all was so simple - and all worked fine. (.loadfromfile..., ShowMessage('Ärgern Sie sich nicht über lazarus'), Ini.ReadString..., fileExists('Vermögensliste.txt') - all is changing now).

:-)

Can you help me to find out, what text-Coding-format is in a text-file?

theo

2009-02-02 19:22

reporter   ~0025027

For FileExists you could use UTF8ToSys.

For Loadfromfile, a combination of my Uniloader and LConvEncoding.GuessEncoding might do what can be done.

ShowMessage: Doesn't this work on Windows?

Please ask here http://www.lazarusforum.de/ if you have more questions.

Vincent Snijders

2009-02-02 19:36

manager   ~0025028

Forums and mailing lists are indeed the place to ask questions and seek clarifications; the bug tracker is not intended for such discussions.

Issue History

Date Modified Username Field Change
2008-11-19 19:29 WD New Issue
2008-11-19 19:29 WD Widgetset => Win32
2008-11-20 08:57 Vincent Snijders LazTarget => -
2008-11-20 08:57 Vincent Snijders Note Added: 0023452
2008-11-20 08:57 Vincent Snijders Status new => feedback
2008-11-20 09:29 Graeme Geldenhuys Note Added: 0023453
2008-11-20 09:37 Vincent Snijders Note Added: 0023454
2008-11-20 09:53 Graeme Geldenhuys Note Added: 0023455
2008-12-05 03:13 Felipe Monteiro de Carvalho Status feedback => resolved
2008-12-05 03:13 Felipe Monteiro de Carvalho Resolution open => not fixable
2008-12-05 03:13 Felipe Monteiro de Carvalho Assigned To => Felipe Monteiro de Carvalho
2008-12-05 03:13 Felipe Monteiro de Carvalho Note Added: 0023675
2008-12-09 18:02 WD Status resolved => assigned
2008-12-09 18:02 WD Resolution not fixable => reopened
2008-12-09 18:02 WD Note Added: 0023734
2008-12-09 19:21 Vincent Snijders Note Added: 0023736
2008-12-09 19:21 Vincent Snijders Status assigned => resolved
2008-12-09 19:21 Vincent Snijders Resolution reopened => no change required
2009-02-02 17:02 WD Status resolved => assigned
2009-02-02 17:02 WD Resolution no change required => reopened
2009-02-02 17:02 WD Note Added: 0025024
2009-02-02 19:22 theo Note Added: 0025027
2009-02-02 19:36 Vincent Snijders Status assigned => resolved
2009-02-02 19:36 Vincent Snijders Resolution reopened => no change required
2009-02-02 19:36 Vincent Snijders Note Added: 0025028
2009-06-13 12:09 Marc Weustink Status resolved => closed