View Issue Details

IDProjectCategoryView StatusLast Update
0031998FPCCompilerpublic2018-02-04 19:41
Reporterjamie philbrook Assigned ToJonas Maebe  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionno change required 
Platform32/64 bit etc.OSWindows 
Product Version3.0.0 
Summary0031998: LO, HI functions using WORD as parameter and MATH inside switches it to inetger;
DescriptionI just tracked down a bug that really took me a while to find.

The LO/HI functions when used with a WORD type and +/- enclosed operations
will convert the results to a integer which is a 32 bit.

 The Lo will return the lower WORD and HI the UPPER word instead.
Steps To ReproduceVar
 V:WORD;
Begin
 V := 320;
 ByteResult := LO(V-1); //this returns 63, which is correct, but there is an error regardless.
 ByteResult := HI(V-1); // This returns 0, it should be returning 1;
 
 What has happen is the V-1 operation converted it to an integer and thus ended up using the Integer version of the Functions.

 Since my return types are BYTE, only the lower order gets loaded and the compiler never complained about the return type not being compatible.

 If I Cast...
 LO(Word(V-1)), all is well.

 With my old D3 this works as expected, there is no conversion of types and WORD type is maintain.
 
 So either a Document addition is needed or this resolved?

 I don't see any comments about it in the DOCS.

TagsNo tags attached.
Fixed in Revision
FPCOldBugId
FPCTarget
Attached Files

Relationships

duplicate of 0033128 resolvedMichael Van Canneyt Compiler does not respect evaluation order with HI/LO 

Activities

Marco van de Voort

2017-06-10 13:36

manager   ~0100992

D3 had word and longint versions too? I can't remember it supported overloading.

Sven Barth

2017-06-10 15:39

manager   ~0100994

In FPC they're compiler intrinsics, maybe they are/were in Delphi as well?

Serge Anvarov

2017-06-10 18:18

reporter   ~0101000

Lo and Hi in FPC have many overloaded versions. The expressions V-1 are converted to Integer. As a result, the "function Lo(l: LongInt): Word" is used (for x32, for x64 is used "function Lo(i: Int64): DWord"). In the case of a range check, there will be a range error, because the word result (319) does not fit into the byte. It is obvious that the hi word of long 319 is zero.
Correctly to write so:
ByteResult := Lo(Word(V-1));
ByteResult := Hi(Word(V-1));

Bart Broersma

2017-06-11 00:07

reporter   ~0101012

D7 returns 63 and 1 repectively on the original code.

jamie philbrook

2017-06-11 00:23

reporter   ~0101013

Thanks for verifying, this confirms my suspicion of the problem.

Serge Anvarov

2017-06-11 01:39

reporter   ~0101015

Delphi has only one Lo function, so the test passes. For FPC, you must specify which function to use.
No problem.

jamie philbrook

2017-06-11 02:01

reporter   ~0101016

Delphi's LO function works on DWORDS and WORDS and it works fine on both,.
the same goes for HI, There is no changing of the types to suite its needs.

 If there is no interest to fix it, At least we could put a note in the
help file/Online docs about it.

jamie philbrook

2017-06-11 16:59

reporter   ~0101022

Last edited: 2017-06-11 17:02

View 2 revisions

This problem goes deeper!
// Check out below code..

Function MyHi(Avalue :Word):Byte;
Begin
   Result := Avalue Shr 8;
end;
Function MyHi(aValue:DWord):Word;
Begin
   Result := AValue shr 16;
end;

procedure TForm1.Button1Click(Sender: TObject);
Var
   W:WORD;
   B:Byte;
begin
   W:= 319;
   B := MyHi(W-1);
   Caption := IntToStr(B);
end;

Errors: 1, Hints: 2
unit1.pas(46,9) Error: Can't determine which overloaded function to call
unit1.pas(35,10) Hint: Found declaration: MyHi(LongWord):Word;
unit1.pas(31,10) Hint: Found declaration: MyHi(Word):Byte;

 It should be obvious, it should be calling the WORD one..

The compiler complains about this, even though it should be able to resolve it

However, the LO/HI simply picks the last resolved type, which is in error.

EDIT:
 Wanted to add, this goes all the way back to the oldest FPC I have, 2.6.4
and wouldn't surprise me to be present on most common targets.

Károly Balogh

2017-06-11 20:26

manager   ~0101023

Can someone test what this code gives with various Delphi versions and with FPC on 64bit platforms?

var
  u: byte;
  v: word;
  w: dword;

begin
  writeln(sizeof(u));
  writeln(sizeof(u+1));
  writeln(sizeof(u-1));
  writeln(sizeof(v));
  writeln(sizeof(v+1));
  writeln(sizeof(v-1));
  writeln(sizeof(w));
  writeln(sizeof(w+1));
  writeln(sizeof(w-1));
end.

On Darwin-i386 the result is:

1
4
4
2
4
4
4
4
8

The whole issue is not with Hi/Lo or overloading. It is just a side effect of FPC extending the result of all operations to native signed int size and even beyond in case of subtractions (to avoid overflows and provide correct signedness). This is a known behavior. However, if Delphi behaves otherwise, it might be worth considering a fix, if possible at all.

I'm pretty sure that this is documented somewhere though. I agree that it's hard to recognize this behavior on some cases, when one gets unexpected results.

Serge Anvarov

2017-06-11 20:37

reporter   ~0101024

Last edited: 2017-06-11 20:39

View 2 revisions

@jamie philbrook
In the documentation https://www.freepascal.org/docs-html/current/ref/refsu45.html#x148-17000012.8.1 "For binary operators, the result type will be integer if both operands are integer type expressions" it's not so obvious that the result of W-1 will be an Integer (not a Word).
In the Delphi documentation http://docwiki.embarcadero.com/RADStudio/Tokyo/en/Expressions_(Delphi) this is more clearly pronounced "the result is of type Int64 when at least one operand is of type Int64; otherwise, the result is of type Integer. If an operand's type is a subrange of an integer type, it is treated as if it were of the integer type".
I think in FPC it is implemented that way. As a result, if you look at your example, it looks for the function MyHi(Avalue: Integer), but it does not.

jamie philbrook

2017-06-11 20:57

reporter   ~0101028

Last edited: 2017-06-11 21:12

View 2 revisions

working with integers I can understand how it will convert but in this case
we are working with NON-integrators and the results should return as such or
force an insert of a Type case internally to resolve it as the original.
If using a Constant is causing this, then this is incorrect. I always thought
the constant will switch to what ever the type prior to it is? That is the
way it has always been using Delphi.

   Delphi does not switch to integrator versions of functions when using
Byte, WORD, DWORD QWORD with enclosed math, that much I know..

EDIT:
 Just tested with a Type Cast around the constant Word(1); it makes no change.
so that concludes it is the final results that is causing type issues.

my opinion: FPC needs to resolve this, putting loads of Type cast around this
is tedious.

Károly Balogh

2017-06-11 22:02

manager   ~0101035

@Serge Anvarov:
To be fair, I think the FPC documentation there doesn't mean Integer as in type Integer, but integer number as in not floating point/real type, as discussed in the following sentence. The Delphi documentation seems more clear in this sense.

@jaime philbrook:
"FPC needs to resolve this, putting loads of Type cast around this is tedious" - well, if FPC needs to resolve this for any reason, it's not because adding casts are tedious, but because it causes a hard to debug different behavior in Delphi compatible code.

Károly Balogh

2017-06-11 22:20

manager   ~0101036

Last edited: 2017-06-11 22:23

View 3 revisions

Edit: Nevermind this comment, I was confused, so comment deleted.

Marco van de Voort

2017-06-11 22:24

manager   ~0101038

It might be Delphi version specific. Starting with D2009 they started to overload more too.

Károly Balogh

2017-06-11 22:31

manager   ~0101039

Well, FPC's behavior is described here:

https://www.freepascal.org/docs-html/ref/refsu4.html

"As a pascal compiler, Free Pascal does automatic type conversion and upgrading in expressions where different kinds of integer types are used:

    Every platform has a ”native” integer size, depending on whether the platform is 8-bit, 16-bit, 32-bit or 64-bit. e.g. On AVR this is 8-bit.
    Every integer smaller than the ”native” size is promoted to a signed version of the ”native” size. Integers equal to the ”native” size keep their signedness.
    The result of binary arithmetic operators (+, -, *, etc.) is determined in the following way:
        If at least one of the operands is larger than the native integer size, the result is chosen to be the smallest type that encompasses the ranges of the types of both operands. This means that mixing an unsigned with a smaller or equal in size signed will produce a signed type that is larger than both of them.
        If both operands have the same signedness, the result is the same type as them. The only exception is subtracting (-): in the case of unsigned-unsigned subtracting produces a signed result in FPC (as in Delphi, but not in TP7).
        Mixing signed and unsigned operands of the ”native” int size produces a larger signed result. This means that mixing longint and longword on 32-bit platforms will produce an int64. Similarly, mixing byte and shortint on 8-bit platforms (AVR) will produce a smallint."

Which means this behavior is perfectly documented, i.e.: not a bug? :)

Károly Balogh

2017-06-11 22:32

manager   ~0101040

Maybe Delphi behaves differently with constants, and doesn't follow point 3, as described above?

jamie philbrook

2017-06-11 22:49

reporter   ~0101042

I checked that, Constants isn't making it do this, it is the whole process of
doing the equation off to the side and then ignoring the fact that
the starting type was not an INTEGER.

 Switching to integers to perform the math is fine but it has to remember to
put it back to the NON-INTEGER, even if it does over flow it, that is the
point of doing math on non integrator types.

Thaddy de Koning

2017-06-12 11:09

reporter   ~0101056

Last edited: 2017-06-12 11:11

View 3 revisions

The only way I can get it consistent across Delphi versions (from D2 to D10.1) is like this:

program Project1;
{$ifdef fpc}{$mode objfpc}{$endif}{$apptype console}
Var
 V:WORD;
 Byteresult:Byte;
Begin
 V := 320;
 ByteResult := LO(word(V-1)); //this returns 63, which is correct.
 writeln(byteresult);
 ByteResult := HI(word(V-1)); // This returns 1, it should be returning 1;
 writeln(byteresult);
 readln;
end.

Ergo, it is only compatible between different delphi versions AND with FPC if a cast to the proper size of the calculation is used.
That seems acceptible to me.

Thaddy de Koning

2017-06-13 07:45

reporter   ~0101074

Last edited: 2017-06-13 18:18

View 4 revisions

https://bugs.freepascal.org/view.php?id=31998#c101023 would take too many casts on all delphi versions. You can't use dword w/o types unit (or unless type cardinal) and you can't take the sizeof(calculation) w/o casts.
Which means I am unable to test that sequence.

Sven Barth

2017-06-23 15:00

manager   ~0101308

DWord is defined in System unit, not Types unit and just like Cardinal it's an alias to LongWord. You could also use UInt32 if you want...

Thaddy de Koning

2017-06-25 20:31

reporter   ~0101336

Last edited: 2017-06-25 20:37

View 2 revisions

Sven, I was referring to Delphi versions. There dword is not defined in system, but in types. I just rechecked D7 and D2007. Means the example by Károly Balogh could not be tested as-is against Delphi. I had to add type dword = type Cardinal;

Thaddy de Koning

2017-06-25 20:40

reporter   ~0101337

Oh, Iforgot: in Delphi LO/HI on the types that are defined in their system unit really work based on inferred size.

jamie philbrook

2017-06-25 21:23

reporter   ~0101340

This problem goes deeper.
as indicate in one of my other post up there..

 It also effects the calling of overloaded functions.
If I can do this

MY_WORD_VAR := MYWORD_VAR+1;

The compile does not complain on this. It knows to keep the final results to a WORD..
 The same should hold true with enclosed math operations when passing the
final results to the function call.

Thaddy de Koning

2017-06-25 22:31

reporter   ~0101341

Last edited: 2017-06-25 22:38

View 3 revisions

program wordvar;
var
  MYWORD_VAR:word;
begin
  MY_WORD_VAR := $FFFF;
  MY_WORD_VAR := MYWORD_VAR+1;
end.
  
wordvar.pas(6,27) Warning: (4110) range check error while evaluating constants (65536 must be between 0 and 65535)

In other words, there is compile time evaluation of the expression.
as expected it overflows when rangechecks are off (that's a feature)

jamie philbrook

2017-06-26 00:46

reporter   ~0101344

Right, Range check works fine...

This isn't the case, there is no range error with the original example.

The mere adding a 1 to a value that is not even close to be out of range
converted the call parameter to an integer. This causes it to call the wrong
function. There is no compiler warning/error when a function of multiple parameter types is called, it just decides on which one it wants to use, in this
case the wrong one.. Because it converted all to an integer for its math..

 If you create two functions in your own code of the same name, one with integer and one with WORD, past the same WORDVAR+1; the compiler actually knows
there are two different types now at play and can't decide which one to use.
 because it created a new type and it also knows what the original type is.
 it should be just calling the original type function. But at least the compiler knows something at this point. the System.functions seems to skip this obvious check.
 
 Delphi handles this correctly...

Thaddy de Koning

2017-06-26 08:09

reporter   ~0101347

Last edited: 2017-06-26 08:09

View 2 revisions

Yes Delphi handles it fine, but it seems only for the types that are defined in its system unit. It uses type inference to decide what to return for LO/HI. Mask half, shr half.
FPC has many more types in its system unit.

Jonas Maebe

2018-01-01 19:15

manager   ~0105233

In general, a Pascal compiler always evaluates an expression in a generic way (without looking at the context). Then it will try to convert the result of this expression to whatever the type is that is required the consumer/user of this result. That's also the reason why you cannot overload regular functions by changing only the result type.

lo/hi in Turbo Pascal only worked on 16 bit. Initially, it was the same in Delphi. In FPC, from the start (as mentioned still at https://www.freepascal.org/port.var ), they always worked on whatever type you passed in and returned the lower/upper half.

As explained above, and as documented (see earlier comments), integer expressions are always evaluated in a generic way. When you use the result of such an expression, the result type of this expression determines which overload of lo/hi (or other function) gets called. This is a fundamental property of FPC.

Issue History

Date Modified Username Field Change
2017-06-10 05:24 jamie philbrook New Issue
2017-06-10 13:36 Marco van de Voort Note Added: 0100992
2017-06-10 15:39 Sven Barth Note Added: 0100994
2017-06-10 18:18 Serge Anvarov Note Added: 0101000
2017-06-11 00:07 Bart Broersma Note Added: 0101012
2017-06-11 00:23 jamie philbrook Note Added: 0101013
2017-06-11 01:39 Serge Anvarov Note Added: 0101015
2017-06-11 02:01 jamie philbrook Note Added: 0101016
2017-06-11 16:59 jamie philbrook Note Added: 0101022
2017-06-11 17:02 jamie philbrook Note Edited: 0101022 View Revisions
2017-06-11 20:26 Károly Balogh Note Added: 0101023
2017-06-11 20:37 Serge Anvarov Note Added: 0101024
2017-06-11 20:39 Serge Anvarov Note Edited: 0101024 View Revisions
2017-06-11 20:57 jamie philbrook Note Added: 0101028
2017-06-11 21:12 jamie philbrook Note Edited: 0101028 View Revisions
2017-06-11 22:02 Károly Balogh Note Added: 0101035
2017-06-11 22:20 Károly Balogh Note Added: 0101036
2017-06-11 22:21 Károly Balogh Note Edited: 0101036 View Revisions
2017-06-11 22:23 Károly Balogh Note Edited: 0101036 View Revisions
2017-06-11 22:24 Marco van de Voort Note Added: 0101038
2017-06-11 22:31 Károly Balogh Note Added: 0101039
2017-06-11 22:32 Károly Balogh Note Added: 0101040
2017-06-11 22:49 jamie philbrook Note Added: 0101042
2017-06-12 11:09 Thaddy de Koning Note Added: 0101056
2017-06-12 11:10 Thaddy de Koning Note Edited: 0101056 View Revisions
2017-06-12 11:11 Thaddy de Koning Note Edited: 0101056 View Revisions
2017-06-13 07:45 Thaddy de Koning Note Added: 0101074
2017-06-13 07:45 Thaddy de Koning Note Edited: 0101074 View Revisions
2017-06-13 07:50 Thaddy de Koning Note Edited: 0101074 View Revisions
2017-06-13 18:18 Thaddy de Koning Note Edited: 0101074 View Revisions
2017-06-23 15:00 Sven Barth Note Added: 0101308
2017-06-25 20:31 Thaddy de Koning Note Added: 0101336
2017-06-25 20:37 Thaddy de Koning Note Edited: 0101336 View Revisions
2017-06-25 20:40 Thaddy de Koning Note Added: 0101337
2017-06-25 21:23 jamie philbrook Note Added: 0101340
2017-06-25 22:31 Thaddy de Koning Note Added: 0101341
2017-06-25 22:33 Thaddy de Koning Note Edited: 0101341 View Revisions
2017-06-25 22:38 Thaddy de Koning Note Edited: 0101341 View Revisions
2017-06-26 00:46 jamie philbrook Note Added: 0101344
2017-06-26 08:09 Thaddy de Koning Note Added: 0101347
2017-06-26 08:09 Thaddy de Koning Note Edited: 0101347 View Revisions
2018-01-01 19:15 Jonas Maebe Note Added: 0105233
2018-01-01 19:15 Jonas Maebe Status new => resolved
2018-01-01 19:15 Jonas Maebe Resolution open => no change required
2018-01-01 19:15 Jonas Maebe Assigned To => Jonas Maebe
2018-02-04 19:41 Florian Relationship added duplicate of 0033128