View Issue Details

IDProjectCategoryView StatusLast Update
0017381FPCRTLpublic2011-10-06 13:29
ReporterLacaK Assigned ToFlorian  
Status closedResolutionfixed 
Product Version2.5.1 
Fixed in Version2.6.0 
Summary0017381: UTF8UpperCase on Windows98
DescriptionSample program:

UTF8UpperCase('abc') --> 'abc' on Win98, 'ABC' on WinXP,Vista
UpCase('abc') ---> 'ABC' on Win98,XP,Vista

TagsNo tags attached.
Fixed in Revision17474
Attached Files


child of 0017410 closedBart Broersma Lazarus MaskEdit.Text don't work with cyrilic symbols 


Marco van de Voort

2010-09-10 15:52

manager   ~0040941

I assume this is with MSLU ?


2010-09-13 11:52

developer   ~0041023

I am not sure if I understand
(MSLU = MS Unicode Layer for Windows 9x = unicows.dll)
I have this DLL in Windows/System folder, but this is all.
I do not use it in my app (so if FPC does not load or use it in the background, then ?) ... UTF8UpperCase is declared in LCLProc unit.

Marco van de Voort

2010-09-17 00:12

manager   ~0041149

Ah, then it is a lazarus issue.

Bart Broersma

2010-09-17 15:41

reporter   ~0041164

This probably needs fixing before I can make TMaskEdit UTF-8 compatble.


2010-09-18 12:17

reporter   ~0041178

@Bart: This report is about Win98 only. It should work on other systems, so there is no need to wait.

Bart Broersma

2010-09-19 22:38

reporter   ~0041212

@Theo: I develop on win9x (WinME) system (dual booted with linux) ;-)

Bart Broersma

2010-09-19 23:15

reporter   ~0041213

This is from the fpc RTL.

In win/

procedure InitWin32Widestrings;
    { Widestring }
    widestringmanager.UpperWideStringProc:=@Win32WideUpper; <------

In system.pp:

function Win32WideUpper(const s : WideString) : WideString;
    if length(result)>0 then

function CharUpperBuff(lpsz:LPWSTR; cchLength:DWORD):DWORD;
    stdcall; external 'user32' name 'CharUpperBuffW';

CharUpperBufW on win9x returns inputstring unaltered.

Vincent Snijders

2010-09-20 06:25

manager   ~0041219

Maybe CharUpperBuffWrapW can be used:

Bart Broersma

2010-09-20 18:58

reporter   ~0041237

I defined

function CharUpperBuffWrapW(lpsz:LPWSTR; cchLength:DWORD):DWORD;
    stdcall; external 'shlwapi.dll' name 'CharUpperBuffW';

When I call it like:

var w98: WideString;
 w98 := 'windows98_ëäöï';
 CharUpperBuffWrapW(PWChar(w98), Length(w98));

The string then remains unchanged in WinME/Win98.

If however you define the function as

function CharUpperBuffWrapW(lpsz:LPWSTR; cchLength:DWORD):DWORD;
    stdcall; external 'shlwapi.dll' index 44; //note the index instead of name

then the string gets converted to "WINDOWS98_ëäöï", but that result is the same as a plain UpperCase (if w98 was an AnsiString and not WideString).
(It works both on Win98 and WinME)

So in the system unit, we should use CharUpperBuffWrapW for the widestringmanager.UpperWideStringProc ?

In LCLProc we could temporarily apply a workaround for the UTF8UpperCase function (just calling UpperCase for Win9x systems and not the WideString version) until the issue is fixed in the next stable fpc release (but that "solution" will probably be frowned upon).

Bart Broersma

2010-11-03 12:26

reporter   ~0042712

Shouldn't this be moved to FPC since the problem seems to be in the assigning of the widestringmanager (in win9x) in the system unit?

2010-12-30 10:44


bug17381.pas (236 bytes)

Vincent Snijders

2010-12-30 10:46

manager   ~0044705

Last edited: 2010-12-30 10:48

Lacak2 or Bart, can you run the attached program, that only uses the sysutils unit, on windows 98. If it fails, I will move this issue to the FPC project.

It works on windows XP.

Bart Broersma

2010-12-30 12:00

reporter   ~0044710

It fails with fpc 2.4.1 (don't have trunk, but AFAICS the assignment of win32 WS manager hasn't changed since note 0041237)

F:\LazarusProjecten\bugs\UTF8UpperCase>type utf8up.pp
program bug17381;

{$mode objfpc}

  w1, w2: widestring;

  w1 := 'abc';
  w2 := wideuppercase(w1);
  if w2 <> 'ABC' then
    writeln('Test failed: ', w2);

F:\LazarusProjecten\bugs\UTF8UpperCase>fpc utf8up.pp
Free Pascal Compiler version 2.4.1 [2010/03/12] for i386
Copyright (c) 1993-2009 by Florian Klaempfl
Target OS: Win32 for i386
Compiling utf8up.pp
Linking utf8up.exe
19 lines compiled, 1.9 sec , 57728 bytes code, 11000 bytes data

Test failed: abc

Vincent Snijders

2010-12-30 12:30

manager   ~0044713

Marco, it is an RTL issue after all.

Marco van de Voort

2010-12-30 13:29

manager   ~0044718

Last edited: 2010-12-30 14:48

Well, primarily it is both a win9x issue, and a conceptual problem in the program (that tries to do unicode on a non-unicode OS). I'm tempted to close this as "won't fix" for those reasons, but I'll wait what others have to say.

I don't know much about MSLU/unicows, but afaik you have to specifically link to it, so I would try to {$linklib unicows}, and see what happens. (or maybe -XLAkernel32=kernel32,unicows )

If that doesn't work, try to add with additional parameters -XLOunicows=500 or -XLOunicows=1500

These parameters influence the order of linking (since strictly speaking unicows should be first). That is all in the hope that the importlib for unicows does not do anything special. If so, you need to search for a mingw unicows importlib that still supports win9x.

I also hope that the dynamic loading going on in the RTL won't upset unicows. (iow the existing partial win9x fixes might prevent unicows from working)

Marco van de Voort

2010-12-30 14:42

manager   ~0044720

I happened to run into a Delphi+unicows link today. It contains an "import" unit for Delphi:

Sergei Gorelkin

2010-12-30 19:35

developer   ~0044734

We definitely should not depend on shlwapi.dll, especially on a function that is said to be possibly removed after Windows XP.

It would be good to support MSLU, because it enables us to eventually make whole Windows RTL support Unicode directly, without dropping support of Win9x.

Marco van de Voort

2010-12-31 12:08

manager   ~0044753

See comment 041213 (and source), FPC uses user32.dll functions.
rtl/win and rtl/win32 don't grep on shlwapi.

I think the MSLU route is the only sane one, the unicode dependancy will only rise (and while every compatibility hack is small, they do add up). If it doesn't work, I think it is better to refer people to older versions.

Sergei Gorelkin

2010-12-31 13:38

developer   ~0044754

>See comment 041213 (and source), FPC uses user32.dll functions.

Yes, but comment 041237 suggests importing from shlwapi.dll

Bart Broersma

2010-12-31 18:11

reporter   ~0044766

There should definitely be some sort of fallback mechanism for win9x that can uppercase UTF8 strings (at least it should do an UpperCase all 'a'..'z' in the string, as UppperCase() does), this can either be done in Lazarus (since it is mainly the LCL that is UTF8), or in fpc.

If we (fpc or Lazarus) don't provide at least a minimal fallback, then Lazarus says goodbye to win9x (and I'll have to buy a new computer :-/ ).

> We definitely should not depend on shlwapi.dll, especially on a function that
> is said to be possibly removed after Windows XP.

There is no need to statically link in shlwapi.dll, just dynamically load it _only_ if we are running win9x (on XP+ we use user32.dll as we do now), thus avoiding problems when in later windows versions this functionality will be removed.

> Well, primarily it is both a win9x issue, and a conceptual problem in the
> program (that tries to do unicode on a non-unicode OS)

Conceptual as it may be, the Lazarus LCL is internally fully UTF8 (as I understand it), so (at least vsiual) LCL widgets that need to do UpperCase now need to do UTF8UpperCase instead.
The least a win9x user expects is that when some visual component does an UTF8UpperCase (for example TEdit when setting CharCase to ecUpperCase), any character from 'a'..'z' in a string will be uppercased (as it used to do before, and as Delphi 7 does on Win9x).

The same applies to UTF8LowerCase also, of course.

To recapitulate: IMHO not fixing this (or at least providing a minimal fallback), wether it be in Lazarus (easily done), or in fpc RTL, is a bad thing (at least for Lazarus).

Marco van de Voort

2010-12-31 20:55

manager   ~0044772

Last edited: 2010-12-31 20:56

"should" is a strong word. Currently win9x support is advocated by "only" needing an occasional fix. However this is slowly becoming a slippery slope, since we had several in the last few months, all adding hacks to the RTL. Now this is even being advocated for strict NT (Unicode!) functionality.

Each time it is "just" a minor hack in the RTL, for an ever decreasing crowd of users (and even less when that hack actually goes live to the public in the next release may 2011 or so).

IMHO it is not worth it anymore. We are already the last ones to give win9x up for active development, while other projects (and commercial entities) stopped years ago. The fact that issues like this go undetected so long says enough.

win9x is a goner as far as I'm concerned, and I'm not even shedding a tear for it

Bart Broersma

2011-01-02 13:18

reporter   ~0044792

> "should" is a strong word
Yes, it is. It reflects my personal emotions on the subject.

> win9x is a goner as far as I'm concerned, and I'm not even shedding a
> tear for it
Thank you very much ;-)
We (fpc) still support DOS though? <G>

It seems reasonable not to "hack" the fpc RTL for this issue, for all teh reasons you gave in your previous notes.

I would still strongly advocate to at least offer a fallback mechanisme (simple workaround) in Lazarus LCL. I can provide one.
We could adopt the policy as it is for GTK(1); if someone provides a solution (fix or workaround) that doesn't screw up WinXP+, it'll be accepted (possibly surrounded by compiler directives for win9x, to make it easier to remove when we eventually (inevitably) drop win9x support)

If fpc/Lazarus drops support for win9x, I think we should also discuss (on Laz mailinglist?) the minimum requirements for other OS/WidgetSets (minimal GTK2 requirement comes to mind (Zeljan's life is gonna be so much easier then)).

Sergei Gorelkin

2011-01-02 14:27

developer   ~0044795

Bart, please note that even if we support MSLU, it will at most work as it does for AnsiStrings - so the issue won't be formally fixed.
RTL already allows to implement any kind of workaround or fallback by replacing WideStringManager/UnicodeStringManager. You don't need to replace it as a whole, only individual routines.

As for the DOS, today it is a free and maintained platform (FreeDos, not MS-DOS of course), and it is very possible to buy a computer with it. Therefore, no point for FPC to drop support. Similar story with Wine/ReactOS, but even they are developing an analog to Win2k, not Win9x.

Bart Broersma

2011-01-02 17:03

reporter   ~0044798

> Bart, please note that even if we support MSLU, it will at most work as it does
> for AnsiStrings
I know, but Win9x users will not expect more probably.

I also know that I can easily make a workaround for my own units/components, but providing at least a workaround for UTF8UpperCase/UTF8LowerCase _in_ LCLProc makes all visual LCL components behave (sort of) allright in win9x.

Q: if I replace WideStringManger.UpperWideStringProc (at runtime) with one that uses CharUpperBuffWrapW, will all LCL components then use that one instead?

Sergei Gorelkin

2011-01-02 17:30

developer   ~0044799

>Q: if I replace WideStringManger.UpperWideStringProc (at runtime) with one that uses CharUpperBuffWrapW, will all LCL components then use that one instead?

Hard to answer without knowing how "all LCL components" are written in details. But everything that calls SysUtils.WideUpperCase or System.UpCase(WideString) will be hooked.

Marco van de Voort

2011-01-02 20:03

manager   ~0044804

Last edited: 2011-01-02 20:43

In theory we could split win9x off as separate target. Then it would be like dos, and can rot in peace.

But in reality, it will IMHO only increase the snapshot with 10MB, and not do much else after a lot of upsetting of the source and an initial amount of work..

This because I don't expect that much interest, and it will be yet another sleeping target. (like dos that has had less than minimal management since the 2000-2003 timeframe)

(the few legacy needs remaining are mostly covered by existing versions)

A compromise would be to keep 2.4.4 still win9x compat, and then kill it off after. But IMHO the time for open ended fixing of any win9x compat issue is over.

And Sergei is right that this can be fixed by providing a 3rd party widestring manager.

Bart Broersma

2011-01-02 23:51

reporter   ~0044815

OK then, so no statically linking shlwapi.dll.

To provide a reasonable win9x WS manager substitute (not as part of fpc, but as a seperate unit for whomever is interested) a solution would be to dynamically load shlwapi.dll and then link in CharUpperBufWrapW and CharLowerBufWrapW functions.

Problem herein is that said functions can only be loaded by index (not by name), so after doing a LoadLibrary() a GetProcAddres(Handle,Name) fails.
Q: Is it at all possible to dynamically load indexed functions from a dll?

Sergei Gorelkin

2011-01-03 00:57

developer   ~0044817

It is, but not with dynlibs unit which accepts only string names. Call WinAPI GetProcAddress() and supply the index casted to PChar.
Also, in a separate unit that is used on demand, static linking is fine. I'm only against linking shlwapi statically to main RTL units like system/sysutils.
A simpler solution without additional dlls could be converting to AnsiString, calling CharUpperBuffA, then converting back. MSLU does just that, and probably function from shlwapi works the same way too.

2011-01-03 13:22


win9xwsmanager.pp (5,475 bytes)   
{ *********************************************************************

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of

    This file is distributed under the LGPL2 license


unit win9xwsmanager;

 Objective: to provide minimal WideString Upper- and LowerCase
 functionality under Win9x systems.
 This is achievd by dynamically linking shlwapi.dll functions.
 If this fails a fallback mechanism is provided so that at least all
 lower ASCII characters in a WideString are Upper/LowerCased

 Without this library UTF8UpperCase/UTF8LowerCase will fail
 on win9x systems (which makes many Lazarus LCL controls that
 handle strings behave wrong).

 You can use this unit in your uses clause independent of yout target OS.
 No code will be linked in if yout target is not Windows.
 On true Unicode Windows (WinCE, WinNT-based) no additional libraries
 will be linked in (see: InitWin9xWSManager).

 Currently only LowerWideStringProc and UpperWideStringProc are
 replaced on win9x.
 Possibly other functions might need replacin too.


{$mode objfpc}{$H+}


  Windows, SysUtils {$ifdef DEBUG_WIN9X_WSMANAGER}, LCLProc{$endif};

{$endif WINDOWS}


{$ifdef WINDOWS}

  TShlwapiFunc = function(lpsz: LPWSTR; ccLength: DWORD): DWORD; stdcall;

  CharUpperBuffWrapW: TShlwapifunc = nil;
  CharLowerBuffWrapW: TShlwapifunc = nil;
  ShlwapiHandle: THandle = 0;
  SavedUnicodeStringManager: TUnicodeStringManager;

// Win9x**Simple functions do essentially Upper/LowerCase on
// lower ASCII characters in the string

function Win9xWideUpperSimple(const S: WideString): WideString;
  diff = Ord('a') - Ord('A');
  W: WideChar;
  i: Integer;
  Result := S;
  for i := 1 to length(Result) do
    W := Result[i];
    if (Ord(W) in [Ord(Char('a'))..Ord(Char('z'))]) then
      Word(W) := Word(W) - diff;
      Result[i] := W;

function Win9xWideLowerSimple(const S: WideString): WideString;
  diff = Ord('a') - Ord('A');
  W: WideChar;
  i: Integer;
  Result := S;
  for i := 1 to length(Result) do
    W := Result[i];
    if (Ord(W) in [Ord(Char('A'))..Ord(Char('Z'))]) then
      Word(W) := Word(W) + diff;
      Result[i] := W;

function Win9xWideUpper(const S: WideString): WideString;
  Result := S;
  CharUpperBuffWrapW(PWChar(Result), Length(Result));

function Win9xWideLower(const S: WideString): WideString;
  Result := S;
  CharLowerBuffWrapW(PWChar(Result), Length(Result));

procedure FreeDll;
  if ShlwapiHandle <> 0 then
    ShlwapiHandle := 0;

procedure InitDll;
  PU,PL: Pointer;
  ShlwapiHandle := LoadLibrary('shlwapi.dll');
  if (ShlwapiHandle <> 0) then
    //shlwapi functions cannot be loaded by name, only by index!
    PU := GetProcAddress(ShlwapiHandle,PChar(44));
    if (PU <> nil) then CharUpperBuffWrapW := TShlwapiFunc(PU);
    PL := GetProcAddress(ShlwapiHandle,PChar(39));
    if (PL <> nil) then CharLowerBuffWrapW := TShlwapiFunc(PL);
    DebugLn('Successfully loaded shlwapi.dll');
    if (PU <> nil) then
      DebugLn('Assigning CharUpperBuffWrapW @: ',dbgs(PU))
      DebugLn('Unable to load CharUpperBuffWrapW');
    if (PL <> nil) then
      DebugLn('Assigning CharLowerBuffWrapW @: ',dbgs(PL))
      DebugLn('Unable to load CharLowerBuffWrapW');
    if (PU = nil) and (PL = nil) then
    DebugLn('Unable to load shlwapi.dll');

procedure InitWin9xWSManager;
  WS: WideString;
  SavedUnicodeStringManager := WideStringManager;
  WS := 'abc';
  if WideUpperCase(WS) <> 'ABC' then
    if Assigned(CharUpperBuffWrapW) then
      WideStringManager.UpperWideStringProc := @Win9xWideUpper;
      WS := 'abc';
      if WideUpperCase(WS) <> 'ABC' then WideStringManager.UpperWideStringProc := @Win9xWideUpperSimple;
      WideStringManager.UpperWideStringProc := @Win9xWideUpperSimple;
    if Assigned(CharLowerBuffWrapW) then
      WideStringmanager.LowerWideStringProc := @Win9xWideLower;
      WS := 'ABC';
      if WideLowerCase(WS) <> 'abc' then WideStringManager.LowerWideStringProc := @Win9xWideLowerSimple;
      WideStringManager.LowerWideStringProc := @Win9xWideLowerSimple;


  WideStringManager := SavedUnicodeStringManager;
  if (ShlwapiHandle <> 0) then FreeDll;


win9xwsmanager.pp (5,475 bytes)   

Bart Broersma

2011-01-03 13:33

reporter   ~0044831

Last edited: 2011-01-03 13:34

Attached file win9xwsmanager.pp dynamically links in shlwapi.dll (only when WideUpperCase fails) and replaces WideStringManager.Upper/LowerWideStringProc with CharUpper/LowerBuffWrapW functions, if these can be found, or with a fallback routine that Upper/LowerCases lower ASCII chars in a WideString.

The unit can be linked in by anyone who still supports Win9x, without a penalty if the target OS is actually true Unicode (WinXP+, Linux etc).

I've tested the mechanism under Win98 and WinMe (I have no Win95).
It correctly UpperCases all lower ASCII, but also ä to Ä etc (and vice versa).

If no-one sees an objection to this solution, it would be nice if this unit could be distributed with either fpc or Lazarus sources.

2011-01-05 20:05


win9xwsmanager.pp.2 (5,672 bytes)

Bart Broersma

2011-01-05 20:08

reporter   ~0044916

Uploade a slightly modified version (I cannot delete the old file):
Removed dependencies on LCLProc for debug output (when defined DEBUG_WIN9X_WSMANAGER).


2011-02-05 22:08

administrator   ~0045828

I propose to include the unit into packages\win-base, but there must be two things fixed:
- I think the unit should work silently, no writelns etc.
- The license must be changed into the modified LGPL.

Bart Broersma

2011-02-06 00:05

reporter   ~0045831


See the file: win9xwsmanager.pp.3

The writelns are only included via a condtional define for debugging purposes.
This define is now disabled by default (forgot to disable it in previous version), so the unit it silent now.

The header now points to the COPYING.FPC file, so that should be OK now?

2011-02-06 00:06


win9xwsmanager.pp.3 (5,760 bytes)


2011-05-15 20:27

administrator   ~0048315

Thanks for the contribution, finally applied :)

Issue History

Date Modified Username Field Change
2010-09-10 14:56 LacaK New Issue
2010-09-10 15:52 Marco van de Voort Note Added: 0040941
2010-09-13 06:01 Marco van de Voort Status new => feedback
2010-09-13 11:52 LacaK Note Added: 0041023
2010-09-17 00:12 Marco van de Voort Note Added: 0041149
2010-09-17 00:12 Marco van de Voort Project FPC => Lazarus
2010-09-17 15:41 Bart Broersma Note Added: 0041164
2010-09-18 12:17 theo Note Added: 0041178
2010-09-19 22:38 Bart Broersma Note Added: 0041212
2010-09-19 23:15 Bart Broersma Note Added: 0041213
2010-09-20 06:25 Vincent Snijders Note Added: 0041219
2010-09-20 18:58 Bart Broersma Note Added: 0041237
2010-10-29 16:21 Vincent Snijders LazTarget => 1.0
2010-10-29 16:21 Vincent Snijders Status feedback => acknowledged
2010-10-29 16:21 Vincent Snijders Target Version => 1.0.0
2010-11-03 12:26 Bart Broersma Note Added: 0042712
2010-12-03 16:34 Vincent Snijders Relationship added child of 0017410
2010-12-30 10:44 Vincent Snijders File Added: bug17381.pas
2010-12-30 10:46 Vincent Snijders Note Added: 0044705
2010-12-30 10:48 Vincent Snijders Note Edited: 0044705
2010-12-30 10:48 Vincent Snijders Status acknowledged => feedback
2010-12-30 12:00 Bart Broersma Note Added: 0044710
2010-12-30 12:30 Vincent Snijders LazTarget 1.0 => -
2010-12-30 12:30 Vincent Snijders Note Added: 0044713
2010-12-30 12:30 Vincent Snijders Status feedback => new
2010-12-30 12:30 Vincent Snijders Target Version 1.0.0 =>
2010-12-30 12:30 Vincent Snijders Project Lazarus => FPC
2010-12-30 13:29 Marco van de Voort Note Added: 0044718
2010-12-30 13:30 Marco van de Voort Note Edited: 0044718
2010-12-30 14:42 Marco van de Voort Note Added: 0044720
2010-12-30 14:48 Marco van de Voort Note Edited: 0044718
2010-12-30 19:35 Sergei Gorelkin Note Added: 0044734
2010-12-31 12:08 Marco van de Voort Note Added: 0044753
2010-12-31 13:38 Sergei Gorelkin Note Added: 0044754
2010-12-31 18:11 Bart Broersma Note Added: 0044766
2010-12-31 20:55 Marco van de Voort Note Added: 0044772
2010-12-31 20:56 Marco van de Voort Note Edited: 0044772
2011-01-02 13:18 Bart Broersma Note Added: 0044792
2011-01-02 14:27 Sergei Gorelkin Note Added: 0044795
2011-01-02 17:03 Bart Broersma Note Added: 0044798
2011-01-02 17:30 Sergei Gorelkin Note Added: 0044799
2011-01-02 20:03 Marco van de Voort Note Added: 0044804
2011-01-02 20:43 Marco van de Voort Note Edited: 0044804
2011-01-02 20:43 Marco van de Voort Note Edited: 0044804
2011-01-02 23:51 Bart Broersma Note Added: 0044815
2011-01-03 00:57 Sergei Gorelkin Note Added: 0044817
2011-01-03 13:22 Bart Broersma File Added: win9xwsmanager.pp
2011-01-03 13:33 Bart Broersma Note Added: 0044831
2011-01-03 13:34 Bart Broersma Note Edited: 0044831
2011-01-05 20:05 Bart Broersma File Added: win9xwsmanager.pp.2
2011-01-05 20:08 Bart Broersma Note Added: 0044916
2011-02-05 22:08 Florian Note Added: 0045828
2011-02-06 00:05 Bart Broersma Note Added: 0045831
2011-02-06 00:06 Bart Broersma File Added: win9xwsmanager.pp.3
2011-05-15 20:27 Florian Fixed in Revision => 17474
2011-05-15 20:27 Florian Status new => resolved
2011-05-15 20:27 Florian Fixed in Version => 2.5.1
2011-05-15 20:27 Florian Resolution open => fixed
2011-05-15 20:27 Florian Assigned To => Florian
2011-05-15 20:27 Florian Note Added: 0048315
2011-10-06 13:29 LacaK Status resolved => closed