View Issue Details

IDProjectCategoryView StatusLast Update
0035306FPCFCLpublic2019-08-30 09:02
ReporterOndrej PokornyAssigned ToMichael Van Canneyt 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
Product VersionProduct Build 
Target VersionFixed in Version3.3.1 
Summary0035306: fphttpclient: encoding problems in Get(), Post() etc. that use TStringStream
DescriptionDue to changes in TStringStream http://wiki.freepascal.org/User_Changes_Trunk#TStringStream_now_observe_system_encoding

all methods in fphttpclient that internally use TStringStream assume Delphi=ANSI codepage. E.g.:

function TFPCustomHTTPClient.Post(const URL: string): String;
Var
  SS : TStringStream;
begin
  SS:=TStringStream.Create('');
  try
    Post(URL,SS);
    Result:=SS.Datastring;
  finally
    SS.Free;
  end;
end;

This effectively prohibits the use of these methods on Windows because TEncoding.Default is always ANSI on Windows but almost no responses are encoded in ANSI.
Additional InformationI suggest the 2 possible fixes:

1.) Add TFPCustomHTTPClient.DefaultEncoding property.
2.) Read the response encoding from the "Content-Type" header automatically.
3.) If encoding information is found in the Content-Type header, use it. Otherwise use the DefaultEncoding for the string conversion.

-- OR --

Return RawByteString from these methods with no encoding changes from TStringStream.
TagsNo tags attached.
Fixed in Revision42875
FPCOldBugId
FPCTarget3.2.0
Attached Files

Activities

Ondrej Pokorny

2019-04-01 17:31

reporter   ~0115165

Could somebody please move this to FPC? I accidentally filled up "Lazarus".

Michael Van Canneyt

2019-04-01 21:51

administrator   ~0115174

Ondrej,

Is changing the declaration to use RawByteString enough ?
Will it not use the default encoding of TStringStream in that case ?
Probably a 'SetCodePage(CP_NONE)' is needed as well.

Ondrej Pokorny

2019-04-01 22:56

reporter   ~0115175

> Is changing the declaration to use RawByteString enough ?

No, definitely not. You must not use TStringStream if you want to ignore encoding. I use this helper in my code:

  TMemoryStreamHelper = class helper for TCustomMemoryStream
  public
    function ByteString: RawByteString;
  end;

function TMemoryStreamHelper.ByteString: RawByteString;
begin
  SetLength(Result{%H-}, Size);
  if Size>0 then
    Move(Memory^, Result[1], Size);
end;

Michael Van Canneyt

2019-08-30 09:02

administrator   ~0117870

Changed formdata and result to rawbytestring.
Later on we can probably add a SetCodePage() call in combination with 'Content-Type' charset info, if any such info exists.

Initial tests (on linux) look OK.

Please test and close if OK.

Issue History

Date Modified Username Field Change
2019-04-01 17:22 Ondrej Pokorny New Issue
2019-04-01 17:31 Ondrej Pokorny Note Added: 0115165
2019-04-01 19:38 Bart Broersma Project Lazarus => FPC
2019-04-01 21:48 Michael Van Canneyt Assigned To => Michael Van Canneyt
2019-04-01 21:48 Michael Van Canneyt Status new => assigned
2019-04-01 21:51 Michael Van Canneyt Note Added: 0115174
2019-04-01 22:56 Ondrej Pokorny Note Added: 0115175
2019-08-30 09:02 Michael Van Canneyt Status assigned => resolved
2019-08-30 09:02 Michael Van Canneyt Resolution open => fixed
2019-08-30 09:02 Michael Van Canneyt Fixed in Version => 3.3.1
2019-08-30 09:02 Michael Van Canneyt Fixed in Revision => 42875
2019-08-30 09:02 Michael Van Canneyt FPCTarget => 3.2.0
2019-08-30 09:02 Michael Van Canneyt Note Added: 0117870