ReadLn with custom line ending
Original Reporter info from Mantis: kalagan75
-
Reporter name:
Original Reporter info from Mantis: kalagan75
- Reporter name:
Description:
I'm developing a little Microsoft SCCM multi log parser and viewer with tail function.
Note: Microsoft SCCM service has a lot of indipendent threads and all of them write thier own standard ASCII text log file.
My aim is to have a big grid with inside all lines of every file on it, sorted by a column representing a DateTime.
Because there are two record layouts (agent log files and server log files), I need to read a single file, line by line, and split the line in 3 tokens (datetime, component and description) accordingly with their own layout record.
To achieve this, I use the unit StreamIO to have the ability both to open a file in "fmReadOnly or fmShareDenyNone" mode (so SCCM can continue to write on log files, without locks) and to use standard ReadLn procedure.
And here I have some problems, because on some of these log files, Microsoft inserts some LF (#10) character in the middle of a single line.
That behaviour causes to me problems to read a line because FPC ReadLn procedure considers EndOfString every time a character in buffer is "in [CR,LF]".
This is correct because, for portability reason, we can read either a windows text delimited (CrLf) or a linux text delimited (LF) or a Mac text delimited (CR) file. (But it seems that in Delphi this problem doesn't happens, because each line "must" teminate with crlf).
Note also that if I open the same log file with "notepad", I can see the whole line correctly (yes, of course, with some "strange" characters representing LF)
As example I attach a file with a line that causes this problem.
Here, all characters starting from "&LtPos;![LOG[" to "]LOG]!>" are my "single line" ansistring description; time="..." and date="..." are my datetime; component="..." is my component.
So here is my question. Is it possible to have a new (overloaded?) readln procedure to "enforce" the default LineEnding (based on the target platform)? This means if I'm on linux, a line doesn't finish until a LF is reached, a CR for Mac and a CRLF for Windows.
Or, perhaps better, as you can specify a custom LineEnding short string (actually used for writeln procedure only!), is it possible to have a new readln procedure to read text file with a custom LineEnding? In this way it is possible to solve both my problem (enforce CRLF detection) and read file created by writeln with a custom lineending, in a shot.
Something like readln(var f: text; out s: ansistring; const lineend: string).
Of course, if you think my request can be implemented, this should be done everywhere a "readln" is involved (TStreams, ecc).
Feel free to close this request if it cannot be implemented.
Thanks for your attention.
Additional information:
At the moment, to workaround my problem, I created a new unit copying and adjusting some code from text.inc. In particular, I modified the ReadPCharLen function (to force CRLF detection). Here is my code:
Function ReadPCharLen(var f:Text;s:pchar;maxlen:longint):longint;
var
sPos,len : Longint;
p,startp : pchar;
end_of_string:boolean;
count:longint;
needlf:boolean; //this is if we found a #10 without a #13
lastpos:longint;
startbuffer:integer;
maxp:integer;
Begin
ReadPCharLen:=0;
If not CheckRead(f) then
exit;
sPos:=0;
len:=0;
needlf:=false;
end_of_string:=false;
repeat
If TextRec(f).BufPos>=TextRec(f).BufEnd Then
begin
FileFunc(TextRec(f).InOutFunc)(TextRec(f));
If TextRec(f).BufPos>=TextRec(f).BufEnd Then
break;
end;
startbuffer:=TextRec(f).BufPos;
count:=textrec(f).bufpos;
if SPos+TextRec(f).BufEnd-TextRec(f).BufPos>MaxLen then
maxp:=TextRec(f).BufPos+MaxLen-SPos
else
maxp:=TextRec(f).BufEnd;
while (count < maxp) do
begin
if textrec(f).bufptr^[count]=#13 then
begin
needlf:=true;
break;
end;
if textrec(f).bufptr^[count]=#10 then
if needlf then
begin
dec(count);
break;
end;
inc(count);
end;
if needlf then
if count+1<=textrec(f).bufend-1 then //is #10 within buffer?
begin
if textrec(f).bufptr^[count+1]=#10 then
begin
end_of_string:=true;
textrec(f).bufpos:=count+2;
if count>1 then
lastpos:=count
else
lastpos:=0;
end;
end
else //this is in case last char in buffer=#13, so we cannot check for #10
begin
textrec(f).bufpos:=count+1;
lastpos:=count;
end;
if not(needlf) then
begin
textrec(f).bufpos:=count;
lastpos:=count;
end;
len:=lastpos-startbuffer;
move(textrec(f).bufptr^[startbuffer],s[spos],len);
inc(sPos,len);
until (spos=maxlen) or end_of_string;
ReadPCharLen:=spos;
End;
Mantis conversion info:
- Mantis ID: 20253
- OS: XP
- OS Build: SP3
- Build: no paches
- Platform: Win32
- Version: 2.4.2