View Issue Details

IDProjectCategoryView StatusLast Update
0031469FPCPackagespublic2017-03-06 14:23
ReporterJuha ManninenAssigned ToMichael Van Canneyt 
PrioritynormalSeverityminorReproducibilityhave not tried
Status closedResolutionfixed 
PlatformIntel x86_64OSLinuxOS VersionManjaro
Product Version3.1.1Product Buildr35505 
Target Version3.2.0Fixed in Version3.1.1 
Summary0031469: paszlib: Backport changes in Zipper used in Lazarus OPM to FPC project.
DescriptionLazarus Online Package Manager has a temporary copy of Zipper unit.
The attached patch backports the changes to the master version in FPC packages. The changes are for general use, not specific to the Lazarus OPM.
The main changes are :

1. Use String instead of RawByteString for file names. I have no idea why RawByteString was used in the first place.

2. Provide an extended progress event.

Most changes were made by Balazs Szekely. He can answer specific questions if needed.
The code has been used in OPM for a while and is well tested.
TagsNo tags attached.
Fixed in Revision35517
FPCOldBugId
FPCTarget
Attached Files
  • 0001-paszlib-Backport-changes-in-Zipper-used-in-OPM.patch (36,223 bytes)
    From b770e96aa8763ebbfc35eb9ed92a5a0419e9c679 Mon Sep 17 00:00:00 2001
    From: juha <juha.manninen62@gmail.com>
    Date: Thu, 2 Mar 2017 15:38:27 +0200
    Subject: [PATCH] paszlib: Backport changes in Zipper used in OPM to FPC
     project.
    
    ---
     packages/paszlib/src/zipper.pp | 555 ++++++++++++++++-------------------------
     1 file changed, 211 insertions(+), 344 deletions(-)
    
    diff --git a/packages/paszlib/src/zipper.pp b/packages/paszlib/src/zipper.pp
    index 753a64caa0..7c9ed40c74 100644
    --- a/packages/paszlib/src/zipper.pp
    +++ b/packages/paszlib/src/zipper.pp
    @@ -32,8 +32,6 @@ Const
       LOCAL_FILE_HEADER_SIGNATURE                = $04034B50;
       CENTRAL_FILE_HEADER_SIGNATURE              = $02014B50;
       ZIP64_HEADER_ID                            = $0001;
    -  // infozip unicode path
    -  INFOZIP_UNICODE_PATH_ID                    = $7075;
     
     const
       OS_FAT  = 0; //MS-DOS and OS/2 (FAT/VFAT/FAT32)
    @@ -71,7 +69,6 @@ Type
        Local_File_Header_Type = Packed Record //1 per zipped file
          Signature              :  LongInt; //4 bytes
          Extract_Version_Reqd   :  Word; //if zip64: >= 45
    -     {$warning TODO implement EFS/language enooding using UTF-8}
          Bit_Flag               :  Word; //"General purpose bit flag in PKZip appnote
          Compress_Method        :  Word;
          Last_Mod_Time          :  Word;
    @@ -198,6 +195,7 @@ Const
     Type
     
       TProgressEvent = Procedure(Sender : TObject; Const Pct : Double) of object;
    +  TProgressEventEx = Procedure(Sender : TObject; Const ATotPos, ATotSize: Int64) of object;
       TOnEndOfFileEvent = Procedure(Sender : TObject; Const Ratio : Double) of object;
       TOnStartFileEvent = Procedure(Sender : TObject; Const AFileName : String) of object;
     
    @@ -228,12 +226,16 @@ Type
       { TDeCompressor }
       TDeCompressor = Class(TObject)
       Protected
    -    FInFile     : TStream;        { I/O file variables                         }
    -    FOutFile    : TStream;
    -    FCrc32Val   : LongWord;       { CRC calculation variable                   }
    -    FBufferSize : LongWord;
    -    FOnPercent  : Integer;
    -    FOnProgress : TProgressEvent;
    +    FInFile      : TStream;        { I/O file variables                         }
    +    FOutFile     : TStream;
    +    FCrc32Val    : LongWord;       { CRC calculation variable                   }
    +    FBufferSize  : LongWord;
    +    FOnPercent   : Integer;
    +    FOnProgress  : TProgressEvent;
    +    FOnProgressEx: TProgressEventEx;
    +    FTotPos      : Int64;
    +    FTotSize     : Int64;
    +    FNeedToBreak : Boolean;
         Procedure UpdC32(Octet: Byte);
       Public
         Constructor Create(AInFile, AOutFile : TStream; ABufSize : LongWord); virtual;
    @@ -242,6 +244,7 @@ Type
         Property BufferSize : LongWord read FBufferSize;
         Property OnPercent : Integer Read FOnPercent Write FOnPercent;
         Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
    +    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
         Property Crc32Val : LongWord Read FCrc32Val Write FCrc32Val;
       end;
     
    @@ -341,8 +344,6 @@ Type
       TZipFileEntry = Class(TCollectionItem)
       private
         FArchiveFileName: String; //Name of the file as it appears in the zip file list
    -    FUTF8FileName : UTF8String;
    -    FUTF8DiskFileName : UTF8String;
         FAttributes: LongWord;
         FDateTime: TDateTime;
         FDiskFileName: String; {Name of the file on disk (i.e. uncompressed. Can be empty if based on a stream.);
    @@ -354,12 +355,8 @@ Type
         FStream: TStream;
         FCompressionLevel: TCompressionlevel;
         function GetArchiveFileName: String;
    -    function GetUTF8ArchiveFileName: UTF8String;
    -    function GetUTF8DiskFileName: UTF8String;
         procedure SetArchiveFileName(Const AValue: String);
         procedure SetDiskFileName(Const AValue: String);
    -    procedure SetUTF8ArchiveFileName(AValue: UTF8String);
    -    procedure SetUTF8DiskFileName(AValue: UTF8String);
       Protected
         // For multi-disk support, a disk number property could be added here.
         Property HdrPos : int64 Read FHeaderPos Write FheaderPos;
    @@ -372,9 +369,7 @@ Type
         Property Stream : TStream Read FStream Write FStream;
       Published
         Property ArchiveFileName : String Read GetArchiveFileName Write SetArchiveFileName;
    -    Property UTF8ArchiveFileName : UTF8String Read GetUTF8ArchiveFileName Write SetUTF8ArchiveFileName;
         Property DiskFileName : String Read FDiskFileName Write SetDiskFileName;
    -    Property UTF8DiskFileName : UTF8String Read GetUTF8DiskFileName Write SetUTF8DiskFileName;
         Property Size : Int64 Read FSize Write FSize;
         Property DateTime : TDateTime Read FDateTime Write FDateTime;
         property OS: Byte read FOS write FOS;
    @@ -403,7 +398,7 @@ Type
         FEntries        : TZipFileEntries;
         FZipping        : Boolean;
         FBufSize        : LongWord;
    -    FFileName       : RawByteString;         { Name of resulting Zip file                 }
    +    FFileName       : String;         { Name of resulting Zip file                 }
         FFileComment    : String;
         FFiles          : TStrings;
         FInMemSize      : Int64;
    @@ -415,8 +410,10 @@ Type
         LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
         CentralHdr      : Central_File_Header_Type;
         EndHdr          : End_of_Central_Dir_Type;
    +    FNeedToBreak    : Boolean;
         FOnPercent      : LongInt;
         FOnProgress     : TProgressEvent;
    +    FOnProgressEx   : TProgressEventEx;
         FOnEndOfFile    : TOnEndOfFileEvent;
         FOnStartFile    : TOnStartFileEvent;
         function CheckEntries: Integer;
    @@ -431,7 +428,7 @@ Type
         Function  OpenInput(Item : TZipFileEntry) : Boolean;
         Procedure GetFileInfo;
         Procedure SetBufSize(Value : LongWord);
    -    Procedure SetFileName(Value : RawByteString);
    +    Procedure SetFileName(Value : String);
         Function CreateCompressor(Item : TZipFileEntry; AinFile,AZipStream : TStream) : TCompressor; virtual;
         Property NeedsZip64 : boolean Read FZipFileNeedsZip64 Write FZipFileNeedsZip64;
       Public
    @@ -439,40 +436,40 @@ Type
         Destructor Destroy;override;
         Procedure ZipAllFiles; virtual;
         // Saves zip to file and changes FileName
    -    Procedure SaveToFile(AFileName: RawByteString);
    +    Procedure SaveToFile(AFileName: string);
         // Saves zip to stream
         Procedure SaveToStream(AStream: TStream);
         // Zips specified files into a zip with name AFileName
    -    Procedure ZipFiles(AFileName : RawByteString; FileList : TStrings);
    +    Procedure ZipFiles(AFileName : String; FileList : TStrings);
         Procedure ZipFiles(FileList : TStrings);
         // Zips specified entries into a zip with name AFileName
    -    Procedure ZipFiles(AFileName : RawByteString; Entries : TZipFileEntries);
    +    Procedure ZipFiles(AFileName : String; Entries : TZipFileEntries);
         Procedure ZipFiles(Entries : TZipFileEntries);
         Procedure Clear;
       Public
         Property BufferSize : LongWord Read FBufSize Write SetBufSize;
         Property OnPercent : Integer Read FOnPercent Write FOnPercent;
         Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
    +    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
         Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
         Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
    -    Property FileName : RawByteString Read FFileName Write SetFileName;
    +    Property FileName : String Read FFileName Write SetFileName;
         Property FileComment: String Read FFileComment Write FFileComment;
         // Deprecated. Use Entries.AddFileEntry(FileName) or Entries.AddFileEntries(List) instead.
         Property Files : TStrings Read FFiles; deprecated;
         Property InMemSize : Int64 Read FInMemSize Write FInMemSize;
         Property Entries : TZipFileEntries Read FEntries Write SetEntries;
    +    Property NeedToBreak : Boolean Read FNeedToBreak Write FNeedToBreak;
       end;
     
       { TFullZipFileEntry }
     
       TFullZipFileEntry = Class(TZipFileEntry)
       private
    -    FBitFlags: Word;
         FCompressedSize: QWord;
         FCompressMethod: Word;
         FCRC32: LongWord;
       Public
    -    Property BitFlags : Word Read FBitFlags;
         Property CompressMethod : Word Read FCompressMethod;
         Property CompressedSize : QWord Read FCompressedSize;
         property CRC32: LongWord read FCRC32 write FCRC32;
    @@ -501,19 +498,21 @@ Type
         FOnOpenInputStream: TCustomInputStreamEvent;
         FUnZipping  : Boolean;
         FBufSize    : LongWord;
    -    FFileName   : RawByteString;         { Name of resulting Zip file                 }
    -    FOutputPath : RawByteString;
    +    FFileName   : String;         { Name of resulting Zip file                 }
    +    FOutputPath : String;
         FFileComment: String;
         FEntries    : TFullZipFileEntries;
         FFiles      : TStrings;
    -    FUseUTF8: Boolean;
         FZipStream  : TStream;     { I/O file variables                         }
         LocalHdr    : Local_File_Header_Type; //Local header, before compressed file data
         LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
         CentralHdr  : Central_File_Header_Type;
    -
    +    FTotPos     : Int64;
    +    FTotSize    : Int64;
    +    FNeedToBreak: Boolean;
         FOnPercent  : LongInt;
         FOnProgress : TProgressEvent;
    +    FOnProgressEx : TProgressEventEx;
         FOnEndOfFile : TOnEndOfFileEvent;
         FOnStartFile : TOnStartFileEvent;
       Protected
    @@ -529,20 +528,21 @@ Type
         Procedure ReadZipHeader(Item : TFullZipFileEntry; out AMethod : Word);
         Procedure DoEndOfFile;
         Procedure UnZipOneFile(Item : TFullZipFileEntry); virtual;
    -    Function  OpenOutput(OutFileName : RawByteString; Out OutStream: TStream; Item : TFullZipFileEntry) : Boolean;
    +    Function  OpenOutput(OutFileName : String; var OutStream: TStream; Item : TFullZipFileEntry) : Boolean;
         Procedure SetBufSize(Value : LongWord);
    -    Procedure SetFileName(Value : RawByteString);
    -    Procedure SetOutputPath(Value: RawByteString);
    +    Procedure SetFileName(Value : String);
    +    Procedure SetOutputPath(Value:String);
         Function CreateDeCompressor(Item : TZipFileEntry; AMethod : Word;AZipFile,AOutFile : TStream) : TDeCompressor; virtual;
       Public
         Constructor Create;
         Destructor Destroy;override;
         Procedure UnZipAllFiles; virtual;
    -    Procedure UnZipFiles(AFileName : RawByteString; FileList : TStrings);
    +    Procedure UnZipFiles(AFileName : String; FileList : TStrings);
         Procedure UnZipFiles(FileList : TStrings);
    -    Procedure UnZipAllFiles(AFileName : RawByteString);
    +    Procedure UnZipAllFiles(AFileName : String);
         Procedure Clear;
         Procedure Examine;
    +    Function GetZipSize(var IsDirZipped: Boolean; var ABaseDir: String): Int64;
       Public
         Property BufferSize : LongWord Read FBufSize Write SetBufSize;
         Property OnOpenInputStream: TCustomInputStreamEvent read FOnOpenInputStream write FOnOpenInputStream;
    @@ -551,28 +551,27 @@ Type
         Property OnDoneStream : TOnCustomStreamEvent Read FOnDoneStream Write FOnDoneStream;
         Property OnPercent : Integer Read FOnPercent Write FOnPercent;
         Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
    +    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
         Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
         Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
    -    Property FileName : RawByteString Read FFileName Write SetFileName;
    -    Property OutputPath : RawByteString Read FOutputPath Write SetOutputPath;
    +    Property FileName : String Read FFileName Write SetFileName;
    +    Property OutputPath : String Read FOutputPath Write SetOutputPath;
         Property FileComment: String Read FFileComment;
         Property Files : TStrings Read FFiles;
         Property Entries : TFullZipFileEntries Read FEntries;
    -    Property UseUTF8 : Boolean Read FUseUTF8 Write FUseUTF8;
    +    Property NeedToBreak: Boolean Read FNeedToBreak Write FNeedToBreak;
       end;
     
       EZipError = Class(Exception);
     
     Implementation
     
    -uses rtlconsts;
    -
     ResourceString
       SErrBufsizeChange = 'Changing buffer size is not allowed while (un)zipping.';
       SErrFileChange = 'Changing output file name is not allowed while (un)zipping.';
       SErrInvalidCRC = 'Invalid CRC checksum while unzipping %s.';
       SErrCorruptZIP = 'Corrupt ZIP file %s.';
    -  SErrUnsupportedCompressionFormat = 'Unsupported compression format %d';
    +  SErrUnsupportedCompressionFormat = 'Unsupported compression format %d.';
       SErrUnsupportedMultipleDisksCD = 'A central directory split over multiple disks is unsupported.';
       SErrMaxEntries = 'Encountered %d file entries; maximum supported is %d.';
       SErrMissingFileName = 'Missing filename in entry %d.';
    @@ -581,56 +580,10 @@ ResourceString
       SErrPosTooLarge = 'Position/offset %d is larger than maximum supported %d.';
       SErrNoFileName = 'No archive filename for examine operation.';
       SErrNoStream = 'No stream is opened.';
    -  SErrEncryptionNotSupported = 'Cannot unzip item "%s" : encryption is not supported.';
    -  SErrPatchSetNotSupported = 'Cannot unzip item "%s" : Patch sets are not supported.';
     
     { ---------------------------------------------------------------------
         Auxiliary
       ---------------------------------------------------------------------}
    -Type
    -  // A local version of TFileStream which uses rawbytestring. It
    -  TFileStream = class(THandleStream)
    -  Private
    -    FFileName : RawBytestring;
    -  public
    -    constructor Create(const AFileName: RawBytestring; Mode: Word);
    -    constructor Create(const AFileName: RawBytestring; Mode: Word; Rights: Cardinal);
    -    destructor Destroy; override;
    -    property FileName : RawBytestring Read FFilename;
    -  end;
    -  constructor TFileStream.Create(const AFileName: rawbytestring; Mode: Word);
    -
    -  begin
    -    Create(AFileName,Mode,438);
    -  end;
    -
    -
    -  constructor TFileStream.Create(const AFileName: rawbytestring; Mode: Word; Rights: Cardinal);
    -
    -  Var
    -    H : Thandle;
    -
    -  begin
    -    FFileName:=AFileName;
    -    If (Mode and fmCreate) > 0 then
    -      H:=FileCreate(AFileName,Mode,Rights)
    -    else
    -      H:=FileOpen(AFileName,Mode);
    -
    -    If (THandle(H)=feInvalidHandle) then
    -      If Mode=fmcreate then
    -        raise EFCreateError.createfmt(SFCreateError,[AFileName])
    -      else
    -        raise EFOpenError.Createfmt(SFOpenError,[AFilename]);
    -    Inherited Create(H);
    -  end;
    -
    -
    -  destructor TFileStream.Destroy;
    -
    -  begin
    -    FileClose(Handle);
    -  end;
     
     {$IFDEF FPC_BIG_ENDIAN}
     function SwapLFH(const Values: Local_File_Header_Type): Local_File_Header_Type;
    @@ -819,17 +772,6 @@ begin
           Result := Result or UNIX_FILE;
     end;
     
    -function CRC32Str(const s:string):DWord;
    -var
    -  i:Integer;
    -begin
    -  Result:=$FFFFFFFF;
    -  if Length(S)>0 then
    -    for i:=1 to Length(s) do
    -      Result:=Crc_32_Tab[Byte(Result XOR LongInt(s[i]))] XOR ((Result SHR 8) AND $00FFFFFF);
    -  Result:=not Result;
    -end;
    -
     { ---------------------------------------------------------------------
         TDeCompressor
       ---------------------------------------------------------------------}
    @@ -988,6 +930,8 @@ begin
       Try
         C:=TDeCompressionStream.Create(FInFile,True);
         Try
    +      if assigned(FOnProgress) then
    +        fOnProgress(self,0);
           Repeat
             Count:=C.Read(Buf^,FBufferSize);
             For I:=0 to Count-1 do
    @@ -998,9 +942,14 @@ begin
                begin
                  if (FSize>0) and assigned(FOnProgress) Then
                    FOnProgress(self,100 * ( BytesNow / FSize));
    +             if assigned(FOnProgressEx) Then
    +               FOnProgressEx(Self, FTotPos + BytesNow, FTotSize);
                  inc(NextMark,OnBytes);
                end;
    +        if FNeedToBreak then
    +          Break;
           Until (Count=0);
    +      FTotPos := FTotPos + FOutFile.Size;
         Finally
           C.Free;
         end;
    @@ -1009,6 +958,8 @@ begin
       end;
      if assigned(FOnProgress) then
        fOnProgress(self,100.0);
    + if assigned(FOnProgressEx) then
    +   FOnProgressEx(Self, FTotPos, FTotSize);
       Crc32Val:=NOT Crc32Val;
     end;
     
    @@ -1488,7 +1439,7 @@ Begin
       With LocalHdr do
         begin
         Signature := LOCAL_FILE_HEADER_SIGNATURE;
    -    Extract_Version_Reqd := 20; //default value, v2.0
    +    Extract_Version_Reqd := 10; //default value, v1.0
         Bit_Flag := 0;
         Compress_Method := 1;
         DateTimeToZipDateTime(Item.DateTime,Last_Mod_Date,Last_Mod_Time);
    @@ -1578,13 +1529,12 @@ Begin
         LocalHdr.Extra_Field_Length:=SizeOf(LocalZip64ExtHdr)+SizeOf(LocalZip64Fld);
       FOutStream.WriteBuffer({$IFDEF ENDIAN_BIG}SwapLFH{$ENDIF}(LocalHdr),SizeOf(LocalHdr));
       // Append extensible field header+zip64 extensible field if needed:
    -  FOutStream.WriteBuffer(ZFileName[1],Length(ZFileName));
       if IsZip64 then
       begin
    -    LocalZip64ExtHdr.Header_ID:=ZIP64_HEADER_ID;
         FOutStream.WriteBuffer({$IFDEF ENDIAN_BIG}SwapEDFH{$ENDIF}(LocalZip64ExtHdr),SizeOf(LocalZip64ExtHdr));
         FOutStream.WriteBuffer({$IFDEF ENDIAN_BIG}SwapZ64EIF{$ENDIF}(LocalZip64Fld),SizeOf(LocalZip64Fld));
       end;
    +  FOutStream.WriteBuffer(ZFileName[1],Length(ZFileName));
     End;
     
     
    @@ -1660,7 +1610,6 @@ Begin
         {$IFDEF OS2}
           MadeBy_Version := MadeBy_Version or (OS_OS2 shl 8);
         {$ENDIF}
    -      {$warning TODO: find a way to recognize VFAT and NTFS}
           // Copy over extract_version_reqd..extra_field_length
           Move(LocalHdr.Extract_Version_Reqd, Extract_Version_Reqd, 26);
           if (IsZip64) and (Extract_Version_Reqd<45) then
    @@ -1702,7 +1651,7 @@ Begin
     
         Inc(ACount);
         // Move past compressed file data to next header:
    -    if Iszip64 then
    +    if LocalHdr.Compressed_Size=$FFFFFFFF then
           FOutStream.Seek(SavePos + LocalZip64Fld.Compressed_Size,soBeginning)
         else
           FOutStream.Seek(SavePos + LocalHdr.Compressed_Size,soBeginning);
    @@ -1826,6 +1775,7 @@ Begin
           With CreateCompressor(Item, FinFile,ZipStream) do
             Try
               OnProgress:=Self.OnProgress;
    +          OnProgressEx := Self.OnProgressEx;
               OnPercent:=Self.OnPercent;
               Compress;
               CRC:=Crc32Val;
    @@ -1860,7 +1810,7 @@ begin
       SaveToFile(FileName);
     end;
     
    -procedure TZipper.SaveToFile(AFileName: RawByteString);
    +procedure TZipper.SaveToFile(AFileName: string);
     var
       lStream: TFileStream;
     begin
    @@ -1905,7 +1855,7 @@ begin
         FBufSize:=Value;
     end;
     
    -Procedure TZipper.SetFileName(Value : RawByteString);
    +Procedure TZipper.SetFileName(Value : String);
     
     begin
       If FZipping then
    @@ -1913,7 +1863,7 @@ begin
       FFileName:=Value;
     end;
     
    -Procedure TZipper.ZipFiles(AFileName : RawByteString; FileList : TStrings);
    +Procedure TZipper.ZipFiles(AFileName : String; FileList : TStrings);
     
     begin
       FFileName:=AFileName;
    @@ -1926,7 +1876,7 @@ begin
       ZipAllFiles;
     end;
     
    -procedure TZipper.ZipFiles(AFileName: RawByteString; Entries: TZipFileEntries);
    +procedure TZipper.ZipFiles(AFileName: String; Entries: TZipFileEntries);
     begin
       FFileName:=AFileName;
       ZipFiles(Entries);
    @@ -2028,12 +1978,12 @@ Begin
     End;
     
     
    -function TUnZipper.OpenOutput(OutFileName: RawByteString;
    -  out OutStream: TStream; Item: TFullZipFileEntry): Boolean;
    +function TUnZipper.OpenOutput(OutFileName: String; var OutStream: TStream;
    +  Item: TFullZipFileEntry): Boolean;
     Var
    -  Path: RawByteString;
    +  Path: String;
       OldDirectorySeparators: set of char;
    -  
    +
     Begin
       { the default RTL behavior is broken on Unix platforms
         for Windows compatibility: it allows both '/' and '\'
    @@ -2068,9 +2018,8 @@ Begin
           ForceDirectories(Path);
         AllowDirectorySeparators:=OldDirectorySeparators;
         OutStream:=TFileStream.Create(OutFileName,fmCreate);
    -	
         end;
    -	
    +
       AllowDirectorySeparators:=OldDirectorySeparators;
       Result:=True;
       If Assigned(FOnStartFile) then
    @@ -2078,8 +2027,7 @@ Begin
     End;
     
     
    -procedure TUnZipper.CloseOutput(Item: TFullZipFileEntry; var OutStream: TStream
    -  );
    +procedure TUnZipper.CloseOutput(Item: TFullZipFileEntry; var OutStream: TStream);
     
     Begin
       if Assigned(FOnDoneStream) then
    @@ -2105,13 +2053,9 @@ end;
     procedure TUnZipper.ReadZipHeader(Item: TFullZipFileEntry; out AMethod: Word);
     Var
       S : String;
    -  U : UTF8String;
       D : TDateTime;
       ExtraFieldHdr: Extensible_Data_Field_Header_Type;
       SavePos: int64; //could be qword but limited by stream
    -  // Infozip unicode path
    -  Infozip_Unicode_Path_Ver:Byte;
    -  Infozip_Unicode_Path_CRC32:DWord;
     Begin
       FZipStream.Seek(Item.HdrPos,soBeginning);
       FZipStream.ReadBuffer(LocalHdr,SizeOf(LocalHdr));
    @@ -2121,7 +2065,6 @@ Begin
       FillChar(LocalZip64Fld,SizeOf(LocalZip64Fld),0); //ensure no erroneous info
       With LocalHdr do
         begin
    -      Item.FBitFlags:=Bit_Flag;
           SetLength(S,Filename_Length);
           FZipStream.ReadBuffer(S[1],Filename_Length);
           Item.ArchiveFileName:=S;
    @@ -2130,7 +2073,7 @@ Begin
           if Extra_Field_Length>0 then
             begin
             SavePos := FZipStream.Position;
    -        if (LocalHdr.Extra_Field_Length>=SizeOf(ExtraFieldHdr)) then
    +        if (LocalHdr.Extra_Field_Length>=SizeOf(ExtraFieldHdr)+SizeOf(LocalZip64Fld)) then
               while FZipStream.Position<SavePos+LocalHdr.Extra_Field_Length do
                 begin
                 FZipStream.ReadBuffer(ExtraFieldHdr, SizeOf(ExtraFieldHdr));
    @@ -2143,32 +2086,7 @@ Begin
                 {$IFDEF FPC_BIG_ENDIAN}
                   LocalZip64Fld := SwapZ64EIF(LocalZip64Fld);
                 {$ENDIF}
    -              end
    -            // Infozip unicode path
    -            else if ExtraFieldHdr.Header_ID=INFOZIP_UNICODE_PATH_ID then
    -              begin
    -              FZipStream.ReadBuffer(Infozip_Unicode_Path_Ver,1);
    -              if Infozip_Unicode_Path_Ver=1 then
    -                begin
    -                FZipStream.ReadBuffer(Infozip_Unicode_Path_CRC32,sizeof(Infozip_Unicode_Path_CRC32));
    -                {$IFDEF FPC_BIG_ENDIAN}
    -                Infozip_Unicode_Path_CRC32:=SwapEndian(Infozip_Unicode_Path_CRC32);
    -                {$ENDIF}
    -                if CRC32Str(S)=Infozip_Unicode_Path_CRC32 then
    -                  begin
    -                  SetLength(U,ExtraFieldHdr.Data_Size-5);
    -                  FZipStream.ReadBuffer(U[1],Length(U));
    -                  Item.UTF8ArchiveFileName:=U;
    -                  Item.UTF8DiskFileName:=U;
    -                  end
    -                else
    -                  FZipStream.Seek(ExtraFieldHdr.Data_Size-5,soFromCurrent);
    -                end
    -              else
    -                FZipStream.Seek(ExtraFieldHdr.Data_Size-1,soFromCurrent);
    -              end
    -            else
    -              FZipStream.Seek(ExtraFieldHdr.Data_Size,soFromCurrent);
    +              end;
                 end;
             // Move past extra fields
             FZipStream.Seek(SavePos+Extra_Field_Length,soFromBeginning);
    @@ -2336,10 +2254,6 @@ Var
       NewNode   : TFullZipFileEntry;
       D : TDateTime;
       S : String;
    -  U : UTF8String;
    -  // infozip unicode path
    -  Infozip_unicode_path_ver : byte; // always 1
    -  Infozip_unicode_path_crc32 : DWord;
     Begin
       FindEndHeaders(EndHdr, EndHdrPos,
         EndZip64Hdr, EndZip64HdrPos);
    @@ -2387,7 +2301,6 @@ Begin
           NewNode:=FEntries.Add as TFullZipFileEntry;
           // Header position will be corrected later with zip64 version, if needed..
           NewNode.HdrPos := Local_Header_Offset;
    -      NewNode.FBitFlags:=Bit_Flag;
           SetLength(S,Filename_Length);
           FZipStream.ReadBuffer(S[1],Filename_Length);
           SavePos:=FZipStream.Position; //After fixed part of central directory...
    @@ -2431,28 +2344,6 @@ Begin
                   NewNode.HdrPos := Zip64Field.Relative_Hdr_Offset;
                   end;
                 end
    -            // infozip unicode path extra field
    -          else if ExtraFieldHeader.Header_ID = INFOZIP_UNICODE_PATH_ID then
    -            begin
    -            FZipStream.ReadBuffer(Infozip_unicode_path_ver,1);
    -            if Infozip_unicode_path_ver=1 then
    -              begin
    -              FZipStream.ReadBuffer(Infozip_unicode_path_crc32,sizeof(Infozip_unicode_path_crc32));
    -              {$IFDEF FPC_BIG_ENDIAN}
    -              Infozip_unicode_path_crc32:=SwapEndian(Infozip_unicode_path_crc32);
    -              {$ENDIF}
    -              if CRC32Str(S)=Infozip_unicode_path_crc32 then
    -                begin
    -                SetLength(U,ExtraFieldHeader.Data_Size-5);
    -				FZipStream.ReadBuffer(U[1],Length(U));
    -                NewNode.UTF8ArchiveFileName:=U;
    -                end
    -              else
    -                FZipStream.Seek(ExtraFieldHeader.Data_Size-5,soFromCurrent);
    -              end
    -            else
    -              FZipStream.Seek(ExtraFieldHeader.Data_Size-1,soFromCurrent);
    -            end
               else
                 begin
                   // Read past non-Zip64 extra field
    @@ -2480,120 +2371,68 @@ end;
     procedure TUnZipper.UnZipOneFile(Item: TFullZipFileEntry);
     
     Var
    +  Attrs: Longint;
       ZMethod : Word;
    -{$ifdef unix}
    -  LinkTargetStream: TStringStream;
    -{$endif}
    -  OutputFileName: RawByteString;
    +  OutputFileName: string;
       FOutStream: TStream;
       IsLink: Boolean;
       IsCustomStream: Boolean;
    -  U : UnicodeString;
    -
    -  Procedure SetAttributes;
    -  Var
    -    Attrs : Longint;
    -  begin
    -    // set attributes
    -    FileSetDate(OutputFileName, DateTimeToFileDate(Item.DateTime));
    -    if (Item.Attributes <> 0) then
    -      begin
    -      Attrs := 0;
    -      {$IFDEF UNIX}
    -      if (Item.OS in [OS_UNIX,OS_OSX]) then Attrs := Item.Attributes;
    -      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then
    -        Attrs := ZipFatAttrsToUnixAttrs(Item.Attributes);
    -      {$ELSE}
    -      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then Attrs := Item.Attributes;
    -      if (Item.OS in [OS_UNIX,OS_OSX]) then
    -        Attrs := ZipUnixAttrsToFatAttrs(ExtractFileName(Item.ArchiveFileName), Item.Attributes);
    -      {$ENDIF}
    -      if Attrs <> 0 then
    -        begin
    -        {$IFDEF UNIX}
    -        FpChmod(OutputFileName, Attrs);
    -        {$ELSE}
    -        FileSetAttr(OutputFileName, Attrs);
    -        {$ENDIF}
    -        end;
    -      end;
    -  end;
    -
    +  {$IFDEF UNIX}LinkTargetStream: TStringStream;{$ENDIF}
       procedure DoUnzip(const Dest: TStream);
    -
    +  var
    +    Count: Int64;
       begin
         if ZMethod=0 then
           begin
           if (LocalHdr.Compressed_Size<>0) then
             begin
    -        if LocalZip64Fld.Compressed_Size>0 then
    -          Dest.CopyFrom(FZipStream,LocalZip64Fld.Compressed_Size)
    -        else
    -          Dest.CopyFrom(FZipStream,LocalHdr.Compressed_Size);
    -        {$warning TODO: Implement CRC Check}
    -        end;
    -      end
    -    else
    -      With CreateDecompressor(Item, ZMethod, FZipStream, Dest) do
    -        Try
    -          OnProgress:=Self.OnProgress;
    -          OnPercent:=Self.OnPercent;
    -          DeCompress;
    -          if Item.CRC32 <> Crc32Val then
    -            raise EZipError.CreateFmt(SErrInvalidCRC,[Item.ArchiveFileName]);
    -        Finally
    -          Free;
    -        end;
    -  end;
    -
    -  Procedure GetOutputFileName;
    -
    -  Var
    -    I : Integer;
    -
    -  begin
    -    if Not UseUTF8 then
    -      OutputFileName:=StringReplace(Item.DiskFileName,'/',DirectorySeparator,[rfReplaceAll])
    +          if LocalZip64Fld.Compressed_Size>0 then
    +            Count:=Dest.CopyFrom(FZipStream,LocalZip64Fld.Compressed_Size)
    +          else
    +            Count:=Dest.CopyFrom(FZipStream,LocalHdr.Compressed_Size);
    +        end
    +      else
    +        Count:=0;
    +    end
         else
    -      begin
    -      // Sets codepage.
    -      OutputFileName:=Item.UTF8DiskFileName;
    -      U:=UTF8Decode(OutputFileName);
    -      // Do not use stringreplace, it will mess up the codepage.
    -      if '/'<>DirectorySeparator then
    -        For I:=1 to Length(U) do
    -          if U[i]='/' then
    -            U[i]:=DirectorySeparator;
    -      OutputFileName:=UTF8Encode(U);
    -      end;
    -    if (Not IsCustomStream) and (FOutputPath<>'') then
    -      begin
    -      // Do not use IncludeTrailingPathdelimiter
    -      OutputFileName:=FOutputPath+OutputFileName;
    +    With CreateDecompressor(Item, ZMethod, FZipStream, Dest) do
    +      Try
    +        FTotPos := Self.FTotPos;
    +        FTotSize := Self.FTotSize;
    +        OnProgress:=Self.OnProgress;
    +        OnProgressEx := Self.OnProgressEx;
    +        OnPercent:=Self.OnPercent;
    +        DeCompress;
    +        Self.FTotPos := FTotPos;
    +        if Item.CRC32 <> Crc32Val then
    +          raise EZipError.CreateFmt(SErrInvalidCRC,[Item.ArchiveFileName]);
    +      Finally
    +        Free;
           end;
       end;
     
     Begin
       ReadZipHeader(Item, ZMethod);
    -  if (Item.BitFlags and 1)<>0 then
    -    Raise EZipError.CreateFmt(SErrEncryptionNotSupported,[Item.ArchiveFileName]);
    -  if (Item.BitFlags and (1 shl 5))<>0 then
    -    Raise EZipError.CreateFmt(SErrPatchSetNotSupported,[Item.ArchiveFileName]);
       // Normalize output filename to conventions of target platform.
       // Zip file always has / path separators
    +  OutputFileName:=StringReplace(Item.DiskFileName,'/',DirectorySeparator,[rfReplaceAll]);
    +
       IsCustomStream := Assigned(FOnCreateStream);
    -  GetOutputFileName;
    +
    +  if (IsCustomStream = False) and (FOutputPath<>'') then
    +    OutputFileName:=IncludeTrailingPathDelimiter(FOutputPath)+OutputFileName;
    +
       IsLink := Item.IsLink;
    +
     {$IFNDEF UNIX}
       if IsLink and Not IsCustomStream then
    -    begin
    -    {$warning TODO: Implement symbolic link creation for non-unix, e.g.
    -    Windows NTFS}
    +  begin
         IsLink := False;
    -    end;
    +  end;
     {$ENDIF}
    +
       if IsCustomStream then
    -    begin
    +  begin
         try
           OpenOutput(OutputFileName, FOutStream, Item);
           if (IsLink = False) and (Item.IsDirectory = False) then
    @@ -2601,48 +2440,69 @@ Begin
         Finally
           CloseOutput(Item, FOutStream);
         end;
    -    end
    +  end
       else
    -    begin
    +  begin
         if IsLink then
    +    begin
    +    {$IFDEF UNIX}
    +      LinkTargetStream := TStringStream.Create('');
    +      try
    +        DoUnzip(LinkTargetStream);
    +        fpSymlink(PChar(LinkTargetStream.DataString), PChar(OutputFileName));
    +      finally
    +        LinkTargetStream.Free;
    +      end;
    +    {$ENDIF}
    +    end
    +    else
    +    begin
    +      if Item.IsDirectory then
    +        CreateDir(OutputFileName)
    +      else
           begin
    -      {$IFDEF UNIX}
    -        LinkTargetStream := TStringStream.Create('');
             try
    -          DoUnzip(LinkTargetStream);
    -          fpSymlink(PChar(LinkTargetStream.DataString), PChar(OutputFileName));
    -        finally
    -          LinkTargetStream.Free;
    +          OpenOutput(OutputFileName, FOutStream, Item);
    +          DoUnzip(FOutStream);
    +        Finally
    +          CloseOutput(Item, FOutStream);
             end;
    -      {$ENDIF}
    -      end
    -    else if Item.IsDirectory then
    -      CreateDir(OutputFileName)
    -    else
    -      begin
    -      try
    -        OpenOutput(OutputFileName, FOutStream, Item);
    -        DoUnzip(FOutStream);
    -      Finally
    -        CloseOutput(Item, FOutStream);
    -      end;
           end;
    -    SetAttributes;
         end;
    -end;
    -
    +  end;
     
    -procedure TUnZipper.UnZipAllFiles;
    +  if Not IsCustomStream then
    +  begin
    +    // set attributes
    +    FileSetDate(OutputFileName, DateTimeToFileDate(Item.DateTime));
     
    -  Function IsMatch(I : TFullZipFileEntry) : Boolean;
    +    if (Item.Attributes <> 0) then
    +    begin
    +      Attrs := 0;
    +    {$IFDEF UNIX}
    +      if (Item.OS in [OS_UNIX,OS_OSX]) then Attrs := Item.Attributes;
    +      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then
    +        Attrs := ZipFatAttrsToUnixAttrs(Item.Attributes);
    +    {$ELSE}
    +      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then Attrs := Item.Attributes;
    +      if (Item.OS in [OS_UNIX,OS_OSX]) then
    +        Attrs := ZipUnixAttrsToFatAttrs(ExtractFileName(Item.ArchiveFileName), Item.Attributes);
    +    {$ENDIF}
     
    -  begin
    -    if UseUTF8 then
    -      Result:=(FFiles.IndexOf(I.UTF8ArchiveFileName)<>-1)
    -    else
    -      Result:=(FFiles.IndexOf(I.ArchiveFileName)<>-1)
    +      if Attrs <> 0 then
    +      begin
    +    {$IFDEF UNIX}
    +      FpChmod(OutputFileName, Attrs);
    +    {$ELSE}
    +      FileSetAttr(OutputFileName, Attrs);
    +    {$ENDIF}
    +      end;
    +    end;
       end;
    +end;
    +
     
    +procedure TUnZipper.UnZipAllFiles;
     Var
       Item : TFullZipFileEntry;
       I : integer; //Really QWord but limited to FEntries.Count
    @@ -2655,12 +2515,24 @@ Begin
         OpenInput;
         Try
           ReadZipDirectory;
    +      FTotPos := 0;
    +      FTotSize := 0;
           for i:=0 to FEntries.Count-1 do
    -        begin
    +      begin
    +        Item := FEntries[i];
    +        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
    +          FTotSize := FTotSize + TZipFileEntry(Item).Size;
    +      end;
    +      for i:=0 to FEntries.Count-1 do
    +      begin
             Item:=FEntries[i];
    -        if AllFiles or IsMatch(Item) then
    +        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
               UnZipOneFile(Item);
    -        end;
    +        if FNeedToBreak then
    +          Break;
    +      end;
    +      if Assigned(FOnProgressEx) then
    +        FOnProgressEx(Self, FTotSize, FTotSize);
         Finally
           CloseInput;
         end;
    @@ -2679,7 +2551,7 @@ begin
         FBufSize:=Value;
     end;
     
    -procedure TUnZipper.SetFileName(Value: RawByteString);
    +procedure TUnZipper.SetFileName(Value: String);
     
     begin
       If FUnZipping then
    @@ -2687,25 +2559,14 @@ begin
       FFileName:=Value;
     end;
     
    -procedure TUnZipper.SetOutputPath(Value: RawByteString);
    -
    -Var
    -  DS : RawByteString;
    -
    +procedure TUnZipper.SetOutputPath(Value: String);
     begin
       If FUnZipping then
         Raise EZipError.Create(SErrFileChange);
       FOutputPath:=Value;
    -  If (FOutputPath<>'') and (FoutputPath[Length(FoutputPath)]<>DirectorySeparator) then
    -    begin
    -    // Preserve codepage of outputpath
    -    DS:=DirectorySeparator;
    -    SetCodePage(DS,StringCodePage(FoutputPath),False);
    -    FOutputPath:=FoutputPath+DS;
    -    end;
     end;
     
    -procedure TUnZipper.UnZipFiles(AFileName: RawByteString; FileList: TStrings);
    +procedure TUnZipper.UnZipFiles(AFileName: String; FileList: TStrings);
     
     begin
       FFileName:=AFileName;
    @@ -2718,7 +2579,7 @@ begin
       UnZipAllFiles;
     end;
     
    -procedure TUnZipper.UnZipAllFiles(AFileName: RawByteString);
    +procedure TUnZipper.UnZipAllFiles(AFileName: String);
     
     begin
       FFileName:=AFileName;
    @@ -2784,6 +2645,46 @@ begin
       end;
     end;
     
    +function TUnZipper.GetZipSize(var IsDirZipped: Boolean; var ABaseDir: String): Int64;
    +var
    +  I: Integer;
    +  Item: TFullZipFileEntry;
    +  AllFiles: Boolean;
    +  P: Integer;
    +begin
    +  AllFiles := (FFiles.Count = 0);
    +  OpenInput;
    +  try
    +    ReadZipDirectory;
    +    Result := 0;
    +    if FEntries.Count > 0 then
    +    begin
    +      P := Pos('/', TZipFileEntry(FEntries.Items[0]).ArchiveFileName);
    +      if P = 0 then
    +        P := Pos('\', TZipFileEntry(FEntries.Items[0]).ArchiveFileName);
    +      if P <> 0 then
    +      ABaseDir := Copy(TZipFileEntry(FEntries.Items[0]).ArchiveFileName, 1, P);
    +    end;
    +    for i:=0 to FEntries.Count-1 do
    +    begin
    +      Item := FEntries[i];
    +      if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
    +      begin
    +        Result := Result + TZipFileEntry(Item).Size;
    +        if IsDirZipped then
    +          if Pos(ABaseDir, Item.ArchiveFileName) = 0 then
    +            IsDirZipped := False;
    +      end;
    +    end;
    +    if not IsDirZipped then
    +      ABaseDir := ''
    +    else
    +      ABaseDir := Copy(ABaseDir, 1, Length(ABaseDir) - 1);
    +  finally
    +    CloseInput;
    +  end;
    +end;
    +
     destructor TUnZipper.Destroy;
     
     begin
    @@ -2802,20 +2703,6 @@ begin
         Result:=FDiskFileName;
     end;
     
    -function TZipFileEntry.GetUTF8ArchiveFileName: UTF8String;
    -begin
    -  Result:=FUTF8FileName;
    -  If Result='' then
    -    Result:=ArchiveFileName;
    -end;
    -
    -function TZipFileEntry.GetUTF8DiskFileName: UTF8String;
    -begin
    -  Result:=FUTF8DiskFileName;
    -  If Result='' then
    -    Result:=DiskFileName;
    -end;
    -
     constructor TZipFileEntry.Create(ACollection: TCollection);
     
     begin
    @@ -2878,26 +2765,6 @@ begin
         FDiskFileName:=StringReplace(AValue,'/',DirectorySeparator,[rfReplaceAll]);
     end;
     
    -procedure TZipFileEntry.SetUTF8ArchiveFileName(AValue: UTF8String);
    -begin
    -  FUTF8FileName:=AValue;
    -  If ArchiveFileName='' then
    -    if DefaultSystemCodePage<>CP_UTF8 then
    -      ArchiveFileName:=Utf8ToAnsi(AValue)
    -    else
    -      ArchiveFileName:=AValue;
    -end;
    -
    -procedure TZipFileEntry.SetUTF8DiskFileName(AValue: UTF8String);
    -begin
    -  FUTF8DiskFileName:=AValue;
    -  If DiskFileName='' then
    -    if DefaultRTLFileSystemCodePage<>CP_UTF8 then
    -      DiskFileName:=Utf8ToAnsi(AValue)
    -    else
    -      DiskFileName:=AValue;
    -end;
    -
     
     procedure TZipFileEntry.Assign(Source: TPersistent);
     
    -- 
    2.12.0
    
    
  • zipper.patch (4,077 bytes)
    Index: zipper.pp
    ===================================================================
    --- zipper.pp	(revision 35505)
    +++ zipper.pp	(working copy)
    @@ -198,6 +198,7 @@
     Type
     
       TProgressEvent = Procedure(Sender : TObject; Const Pct : Double) of object;
    +  TProgressEventEx = Procedure(Sender : TObject; Const ATotPos, ATotSize: Int64) of object;
       TOnEndOfFileEvent = Procedure(Sender : TObject; Const Ratio : Double) of object;
       TOnStartFileEvent = Procedure(Sender : TObject; Const AFileName : String) of object;
     
    @@ -234,6 +235,10 @@
         FBufferSize : LongWord;
         FOnPercent  : Integer;
         FOnProgress : TProgressEvent;
    +    FOnProgressEx: TProgressEventEx;
    +    FTotPos      : Int64;
    +    FTotSize     : Int64;
    +    FNeedToBreak : Boolean;
         Procedure UpdC32(Octet: Byte);
       Public
         Constructor Create(AInFile, AOutFile : TStream; ABufSize : LongWord); virtual;
    @@ -242,6 +247,7 @@
         Property BufferSize : LongWord read FBufferSize;
         Property OnPercent : Integer Read FOnPercent Write FOnPercent;
         Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
    +    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
         Property Crc32Val : LongWord Read FCrc32Val Write FCrc32Val;
       end;
     
    @@ -511,9 +517,12 @@
         LocalHdr    : Local_File_Header_Type; //Local header, before compressed file data
         LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
         CentralHdr  : Central_File_Header_Type;
    -
    +    FTotPos     : Int64;
    +    FTotSize    : Int64;
    +    FNeedToBreak: Boolean;
         FOnPercent  : LongInt;
         FOnProgress : TProgressEvent;
    +    FOnProgressEx : TProgressEventEx;
         FOnEndOfFile : TOnEndOfFileEvent;
         FOnStartFile : TOnStartFileEvent;
       Protected
    @@ -551,6 +560,7 @@
         Property OnDoneStream : TOnCustomStreamEvent Read FOnDoneStream Write FOnDoneStream;
         Property OnPercent : Integer Read FOnPercent Write FOnPercent;
         Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
    +    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
         Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
         Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
         Property FileName : RawByteString Read FFileName Write SetFileName;
    @@ -559,6 +569,7 @@
         Property Files : TStrings Read FFiles;
         Property Entries : TFullZipFileEntries Read FEntries;
         Property UseUTF8 : Boolean Read FUseUTF8 Write FUseUTF8;
    +    Property NeedToBreak: Boolean Read FNeedToBreak Write FNeedToBreak;
       end;
     
       EZipError = Class(Exception);
    @@ -998,9 +1009,14 @@
                begin
                  if (FSize>0) and assigned(FOnProgress) Then
                    FOnProgress(self,100 * ( BytesNow / FSize));
    +             if assigned(FOnProgressEx) Then
    +               FOnProgressEx(Self, FTotPos + BytesNow, FTotSize);
                  inc(NextMark,OnBytes);
                end;
    +        if FNeedToBreak then
    +          Break;
           Until (Count=0);
    +      FTotPos := FTotPos + FOutFile.Size;
         Finally
           C.Free;
         end;
    @@ -1009,6 +1025,8 @@
       end;
      if assigned(FOnProgress) then
        fOnProgress(self,100.0);
    + if assigned(FOnProgressEx) then
    +   FOnProgressEx(Self, FTotPos, FTotSize);
       Crc32Val:=NOT Crc32Val;
     end;
     
    @@ -2655,12 +2673,25 @@
         OpenInput;
         Try
           ReadZipDirectory;
    +      FTotPos := 0;
    +      FTotSize := 0;
           for i:=0 to FEntries.Count-1 do
    -        begin
    +      begin
    +        Item := FEntries[i];
    +        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
    +          FTotSize := FTotSize + TZipFileEntry(Item).Size;
    +      end;
    +
    +      for i:=0 to FEntries.Count-1 do
    +      begin
             Item:=FEntries[i];
             if AllFiles or IsMatch(Item) then
               UnZipOneFile(Item);
    -        end;
    +        if FNeedToBreak then
    +          Break;
    +      end;
    +      if Assigned(FOnProgressEx) then
    +        FOnProgressEx(Self, FTotSize, FTotSize);
         Finally
           CloseInput;
         end;
    
    zipper.patch (4,077 bytes)
  • zipper1.patch (4,665 bytes)
    Index: zipper.pp
    ===================================================================
    --- zipper.pp	(revision 35506)
    +++ zipper.pp	(working copy)
    @@ -198,6 +198,7 @@
     Type
     
       TProgressEvent = Procedure(Sender : TObject; Const Pct : Double) of object;
    +  TProgressEventEx = Procedure(Sender : TObject; Const ATotPos, ATotSize: Int64) of object;
       TOnEndOfFileEvent = Procedure(Sender : TObject; Const Ratio : Double) of object;
       TOnStartFileEvent = Procedure(Sender : TObject; Const AFileName : String) of object;
     
    @@ -234,6 +235,10 @@
         FBufferSize : LongWord;
         FOnPercent  : Integer;
         FOnProgress : TProgressEvent;
    +    FOnProgressEx: TProgressEventEx;
    +    FTotPos      : Int64;
    +    FTotSize     : Int64;
    +    FNeedToBreak : Boolean;
         Procedure UpdC32(Octet: Byte);
       Public
         Constructor Create(AInFile, AOutFile : TStream; ABufSize : LongWord); virtual;
    @@ -242,6 +247,7 @@
         Property BufferSize : LongWord read FBufferSize;
         Property OnPercent : Integer Read FOnPercent Write FOnPercent;
         Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
    +    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
         Property Crc32Val : LongWord Read FCrc32Val Write FCrc32Val;
       end;
     
    @@ -511,9 +517,12 @@
         LocalHdr    : Local_File_Header_Type; //Local header, before compressed file data
         LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
         CentralHdr  : Central_File_Header_Type;
    -
    +    FTotPos     : Int64;
    +    FTotSize    : Int64;
    +    FNeedToBreak: Boolean;
         FOnPercent  : LongInt;
         FOnProgress : TProgressEvent;
    +    FOnProgressEx : TProgressEventEx;
         FOnEndOfFile : TOnEndOfFileEvent;
         FOnStartFile : TOnStartFileEvent;
       Protected
    @@ -551,6 +560,7 @@
         Property OnDoneStream : TOnCustomStreamEvent Read FOnDoneStream Write FOnDoneStream;
         Property OnPercent : Integer Read FOnPercent Write FOnPercent;
         Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
    +    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
         Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
         Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
         Property FileName : RawByteString Read FFileName Write SetFileName;
    @@ -559,6 +569,7 @@
         Property Files : TStrings Read FFiles;
         Property Entries : TFullZipFileEntries Read FEntries;
         Property UseUTF8 : Boolean Read FUseUTF8 Write FUseUTF8;
    +    Property NeedToBreak: Boolean Read FNeedToBreak Write FNeedToBreak;
       end;
     
       EZipError = Class(Exception);
    @@ -998,9 +1009,14 @@
                begin
                  if (FSize>0) and assigned(FOnProgress) Then
                    FOnProgress(self,100 * ( BytesNow / FSize));
    +             if assigned(FOnProgressEx) Then
    +               FOnProgressEx(Self, FTotPos + BytesNow, FTotSize);
                  inc(NextMark,OnBytes);
                end;
    +        if FNeedToBreak then
    +          Break;
           Until (Count=0);
    +      FTotPos := FTotPos + FOutFile.Size;
         Finally
           C.Free;
         end;
    @@ -1009,6 +1025,8 @@
       end;
      if assigned(FOnProgress) then
        fOnProgress(self,100.0);
    + if assigned(FOnProgressEx) then
    +   FOnProgressEx(Self, FTotPos, FTotSize);
       Crc32Val:=NOT Crc32Val;
     end;
     
    @@ -2536,9 +2554,15 @@
         else
           With CreateDecompressor(Item, ZMethod, FZipStream, Dest) do
             Try
    +          FTotPos := Self.FTotPos;
    +          FTotSize := Self.FTotSize;
               OnProgress:=Self.OnProgress;
    +          OnProgressEx := Self.OnProgressEx;
               OnPercent:=Self.OnPercent;
    +          OnProgress:=Self.OnProgress;
    +          OnPercent:=Self.OnPercent;
               DeCompress;
    +          Self.FTotPos := FTotPos; 
               if Item.CRC32 <> Crc32Val then
                 raise EZipError.CreateFmt(SErrInvalidCRC,[Item.ArchiveFileName]);
             Finally
    @@ -2655,12 +2679,25 @@
         OpenInput;
         Try
           ReadZipDirectory;
    +      FTotPos := 0;
    +      FTotSize := 0;
           for i:=0 to FEntries.Count-1 do
    -        begin
    +      begin
    +        Item := FEntries[i];
    +        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
    +          FTotSize := FTotSize + TZipFileEntry(Item).Size;
    +      end;
    +
    +      for i:=0 to FEntries.Count-1 do
    +      begin
             Item:=FEntries[i];
             if AllFiles or IsMatch(Item) then
               UnZipOneFile(Item);
    -        end;
    +        if FNeedToBreak then
    +          Break;
    +      end;
    +      if Assigned(FOnProgressEx) then
    +        FOnProgressEx(Self, FTotSize, FTotSize);
         Finally
           CloseInput;
         end;
    
    zipper1.patch (4,665 bytes)

Activities

Juha Manninen

2017-03-02 15:15

reporter  

0001-paszlib-Backport-changes-in-Zipper-used-in-OPM.patch (36,223 bytes)
From b770e96aa8763ebbfc35eb9ed92a5a0419e9c679 Mon Sep 17 00:00:00 2001
From: juha <juha.manninen62@gmail.com>
Date: Thu, 2 Mar 2017 15:38:27 +0200
Subject: [PATCH] paszlib: Backport changes in Zipper used in OPM to FPC
 project.

---
 packages/paszlib/src/zipper.pp | 555 ++++++++++++++++-------------------------
 1 file changed, 211 insertions(+), 344 deletions(-)

diff --git a/packages/paszlib/src/zipper.pp b/packages/paszlib/src/zipper.pp
index 753a64caa0..7c9ed40c74 100644
--- a/packages/paszlib/src/zipper.pp
+++ b/packages/paszlib/src/zipper.pp
@@ -32,8 +32,6 @@ Const
   LOCAL_FILE_HEADER_SIGNATURE                = $04034B50;
   CENTRAL_FILE_HEADER_SIGNATURE              = $02014B50;
   ZIP64_HEADER_ID                            = $0001;
-  // infozip unicode path
-  INFOZIP_UNICODE_PATH_ID                    = $7075;
 
 const
   OS_FAT  = 0; //MS-DOS and OS/2 (FAT/VFAT/FAT32)
@@ -71,7 +69,6 @@ Type
    Local_File_Header_Type = Packed Record //1 per zipped file
      Signature              :  LongInt; //4 bytes
      Extract_Version_Reqd   :  Word; //if zip64: >= 45
-     {$warning TODO implement EFS/language enooding using UTF-8}
      Bit_Flag               :  Word; //"General purpose bit flag in PKZip appnote
      Compress_Method        :  Word;
      Last_Mod_Time          :  Word;
@@ -198,6 +195,7 @@ Const
 Type
 
   TProgressEvent = Procedure(Sender : TObject; Const Pct : Double) of object;
+  TProgressEventEx = Procedure(Sender : TObject; Const ATotPos, ATotSize: Int64) of object;
   TOnEndOfFileEvent = Procedure(Sender : TObject; Const Ratio : Double) of object;
   TOnStartFileEvent = Procedure(Sender : TObject; Const AFileName : String) of object;
 
@@ -228,12 +226,16 @@ Type
   { TDeCompressor }
   TDeCompressor = Class(TObject)
   Protected
-    FInFile     : TStream;        { I/O file variables                         }
-    FOutFile    : TStream;
-    FCrc32Val   : LongWord;       { CRC calculation variable                   }
-    FBufferSize : LongWord;
-    FOnPercent  : Integer;
-    FOnProgress : TProgressEvent;
+    FInFile      : TStream;        { I/O file variables                         }
+    FOutFile     : TStream;
+    FCrc32Val    : LongWord;       { CRC calculation variable                   }
+    FBufferSize  : LongWord;
+    FOnPercent   : Integer;
+    FOnProgress  : TProgressEvent;
+    FOnProgressEx: TProgressEventEx;
+    FTotPos      : Int64;
+    FTotSize     : Int64;
+    FNeedToBreak : Boolean;
     Procedure UpdC32(Octet: Byte);
   Public
     Constructor Create(AInFile, AOutFile : TStream; ABufSize : LongWord); virtual;
@@ -242,6 +244,7 @@ Type
     Property BufferSize : LongWord read FBufferSize;
     Property OnPercent : Integer Read FOnPercent Write FOnPercent;
     Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
+    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
     Property Crc32Val : LongWord Read FCrc32Val Write FCrc32Val;
   end;
 
@@ -341,8 +344,6 @@ Type
   TZipFileEntry = Class(TCollectionItem)
   private
     FArchiveFileName: String; //Name of the file as it appears in the zip file list
-    FUTF8FileName : UTF8String;
-    FUTF8DiskFileName : UTF8String;
     FAttributes: LongWord;
     FDateTime: TDateTime;
     FDiskFileName: String; {Name of the file on disk (i.e. uncompressed. Can be empty if based on a stream.);
@@ -354,12 +355,8 @@ Type
     FStream: TStream;
     FCompressionLevel: TCompressionlevel;
     function GetArchiveFileName: String;
-    function GetUTF8ArchiveFileName: UTF8String;
-    function GetUTF8DiskFileName: UTF8String;
     procedure SetArchiveFileName(Const AValue: String);
     procedure SetDiskFileName(Const AValue: String);
-    procedure SetUTF8ArchiveFileName(AValue: UTF8String);
-    procedure SetUTF8DiskFileName(AValue: UTF8String);
   Protected
     // For multi-disk support, a disk number property could be added here.
     Property HdrPos : int64 Read FHeaderPos Write FheaderPos;
@@ -372,9 +369,7 @@ Type
     Property Stream : TStream Read FStream Write FStream;
   Published
     Property ArchiveFileName : String Read GetArchiveFileName Write SetArchiveFileName;
-    Property UTF8ArchiveFileName : UTF8String Read GetUTF8ArchiveFileName Write SetUTF8ArchiveFileName;
     Property DiskFileName : String Read FDiskFileName Write SetDiskFileName;
-    Property UTF8DiskFileName : UTF8String Read GetUTF8DiskFileName Write SetUTF8DiskFileName;
     Property Size : Int64 Read FSize Write FSize;
     Property DateTime : TDateTime Read FDateTime Write FDateTime;
     property OS: Byte read FOS write FOS;
@@ -403,7 +398,7 @@ Type
     FEntries        : TZipFileEntries;
     FZipping        : Boolean;
     FBufSize        : LongWord;
-    FFileName       : RawByteString;         { Name of resulting Zip file                 }
+    FFileName       : String;         { Name of resulting Zip file                 }
     FFileComment    : String;
     FFiles          : TStrings;
     FInMemSize      : Int64;
@@ -415,8 +410,10 @@ Type
     LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
     CentralHdr      : Central_File_Header_Type;
     EndHdr          : End_of_Central_Dir_Type;
+    FNeedToBreak    : Boolean;
     FOnPercent      : LongInt;
     FOnProgress     : TProgressEvent;
+    FOnProgressEx   : TProgressEventEx;
     FOnEndOfFile    : TOnEndOfFileEvent;
     FOnStartFile    : TOnStartFileEvent;
     function CheckEntries: Integer;
@@ -431,7 +428,7 @@ Type
     Function  OpenInput(Item : TZipFileEntry) : Boolean;
     Procedure GetFileInfo;
     Procedure SetBufSize(Value : LongWord);
-    Procedure SetFileName(Value : RawByteString);
+    Procedure SetFileName(Value : String);
     Function CreateCompressor(Item : TZipFileEntry; AinFile,AZipStream : TStream) : TCompressor; virtual;
     Property NeedsZip64 : boolean Read FZipFileNeedsZip64 Write FZipFileNeedsZip64;
   Public
@@ -439,40 +436,40 @@ Type
     Destructor Destroy;override;
     Procedure ZipAllFiles; virtual;
     // Saves zip to file and changes FileName
-    Procedure SaveToFile(AFileName: RawByteString);
+    Procedure SaveToFile(AFileName: string);
     // Saves zip to stream
     Procedure SaveToStream(AStream: TStream);
     // Zips specified files into a zip with name AFileName
-    Procedure ZipFiles(AFileName : RawByteString; FileList : TStrings);
+    Procedure ZipFiles(AFileName : String; FileList : TStrings);
     Procedure ZipFiles(FileList : TStrings);
     // Zips specified entries into a zip with name AFileName
-    Procedure ZipFiles(AFileName : RawByteString; Entries : TZipFileEntries);
+    Procedure ZipFiles(AFileName : String; Entries : TZipFileEntries);
     Procedure ZipFiles(Entries : TZipFileEntries);
     Procedure Clear;
   Public
     Property BufferSize : LongWord Read FBufSize Write SetBufSize;
     Property OnPercent : Integer Read FOnPercent Write FOnPercent;
     Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
+    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
     Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
     Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
-    Property FileName : RawByteString Read FFileName Write SetFileName;
+    Property FileName : String Read FFileName Write SetFileName;
     Property FileComment: String Read FFileComment Write FFileComment;
     // Deprecated. Use Entries.AddFileEntry(FileName) or Entries.AddFileEntries(List) instead.
     Property Files : TStrings Read FFiles; deprecated;
     Property InMemSize : Int64 Read FInMemSize Write FInMemSize;
     Property Entries : TZipFileEntries Read FEntries Write SetEntries;
+    Property NeedToBreak : Boolean Read FNeedToBreak Write FNeedToBreak;
   end;
 
   { TFullZipFileEntry }
 
   TFullZipFileEntry = Class(TZipFileEntry)
   private
-    FBitFlags: Word;
     FCompressedSize: QWord;
     FCompressMethod: Word;
     FCRC32: LongWord;
   Public
-    Property BitFlags : Word Read FBitFlags;
     Property CompressMethod : Word Read FCompressMethod;
     Property CompressedSize : QWord Read FCompressedSize;
     property CRC32: LongWord read FCRC32 write FCRC32;
@@ -501,19 +498,21 @@ Type
     FOnOpenInputStream: TCustomInputStreamEvent;
     FUnZipping  : Boolean;
     FBufSize    : LongWord;
-    FFileName   : RawByteString;         { Name of resulting Zip file                 }
-    FOutputPath : RawByteString;
+    FFileName   : String;         { Name of resulting Zip file                 }
+    FOutputPath : String;
     FFileComment: String;
     FEntries    : TFullZipFileEntries;
     FFiles      : TStrings;
-    FUseUTF8: Boolean;
     FZipStream  : TStream;     { I/O file variables                         }
     LocalHdr    : Local_File_Header_Type; //Local header, before compressed file data
     LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
     CentralHdr  : Central_File_Header_Type;
-
+    FTotPos     : Int64;
+    FTotSize    : Int64;
+    FNeedToBreak: Boolean;
     FOnPercent  : LongInt;
     FOnProgress : TProgressEvent;
+    FOnProgressEx : TProgressEventEx;
     FOnEndOfFile : TOnEndOfFileEvent;
     FOnStartFile : TOnStartFileEvent;
   Protected
@@ -529,20 +528,21 @@ Type
     Procedure ReadZipHeader(Item : TFullZipFileEntry; out AMethod : Word);
     Procedure DoEndOfFile;
     Procedure UnZipOneFile(Item : TFullZipFileEntry); virtual;
-    Function  OpenOutput(OutFileName : RawByteString; Out OutStream: TStream; Item : TFullZipFileEntry) : Boolean;
+    Function  OpenOutput(OutFileName : String; var OutStream: TStream; Item : TFullZipFileEntry) : Boolean;
     Procedure SetBufSize(Value : LongWord);
-    Procedure SetFileName(Value : RawByteString);
-    Procedure SetOutputPath(Value: RawByteString);
+    Procedure SetFileName(Value : String);
+    Procedure SetOutputPath(Value:String);
     Function CreateDeCompressor(Item : TZipFileEntry; AMethod : Word;AZipFile,AOutFile : TStream) : TDeCompressor; virtual;
   Public
     Constructor Create;
     Destructor Destroy;override;
     Procedure UnZipAllFiles; virtual;
-    Procedure UnZipFiles(AFileName : RawByteString; FileList : TStrings);
+    Procedure UnZipFiles(AFileName : String; FileList : TStrings);
     Procedure UnZipFiles(FileList : TStrings);
-    Procedure UnZipAllFiles(AFileName : RawByteString);
+    Procedure UnZipAllFiles(AFileName : String);
     Procedure Clear;
     Procedure Examine;
+    Function GetZipSize(var IsDirZipped: Boolean; var ABaseDir: String): Int64;
   Public
     Property BufferSize : LongWord Read FBufSize Write SetBufSize;
     Property OnOpenInputStream: TCustomInputStreamEvent read FOnOpenInputStream write FOnOpenInputStream;
@@ -551,28 +551,27 @@ Type
     Property OnDoneStream : TOnCustomStreamEvent Read FOnDoneStream Write FOnDoneStream;
     Property OnPercent : Integer Read FOnPercent Write FOnPercent;
     Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
+    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
     Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
     Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
-    Property FileName : RawByteString Read FFileName Write SetFileName;
-    Property OutputPath : RawByteString Read FOutputPath Write SetOutputPath;
+    Property FileName : String Read FFileName Write SetFileName;
+    Property OutputPath : String Read FOutputPath Write SetOutputPath;
     Property FileComment: String Read FFileComment;
     Property Files : TStrings Read FFiles;
     Property Entries : TFullZipFileEntries Read FEntries;
-    Property UseUTF8 : Boolean Read FUseUTF8 Write FUseUTF8;
+    Property NeedToBreak: Boolean Read FNeedToBreak Write FNeedToBreak;
   end;
 
   EZipError = Class(Exception);
 
 Implementation
 
-uses rtlconsts;
-
 ResourceString
   SErrBufsizeChange = 'Changing buffer size is not allowed while (un)zipping.';
   SErrFileChange = 'Changing output file name is not allowed while (un)zipping.';
   SErrInvalidCRC = 'Invalid CRC checksum while unzipping %s.';
   SErrCorruptZIP = 'Corrupt ZIP file %s.';
-  SErrUnsupportedCompressionFormat = 'Unsupported compression format %d';
+  SErrUnsupportedCompressionFormat = 'Unsupported compression format %d.';
   SErrUnsupportedMultipleDisksCD = 'A central directory split over multiple disks is unsupported.';
   SErrMaxEntries = 'Encountered %d file entries; maximum supported is %d.';
   SErrMissingFileName = 'Missing filename in entry %d.';
@@ -581,56 +580,10 @@ ResourceString
   SErrPosTooLarge = 'Position/offset %d is larger than maximum supported %d.';
   SErrNoFileName = 'No archive filename for examine operation.';
   SErrNoStream = 'No stream is opened.';
-  SErrEncryptionNotSupported = 'Cannot unzip item "%s" : encryption is not supported.';
-  SErrPatchSetNotSupported = 'Cannot unzip item "%s" : Patch sets are not supported.';
 
 { ---------------------------------------------------------------------
     Auxiliary
   ---------------------------------------------------------------------}
-Type
-  // A local version of TFileStream which uses rawbytestring. It
-  TFileStream = class(THandleStream)
-  Private
-    FFileName : RawBytestring;
-  public
-    constructor Create(const AFileName: RawBytestring; Mode: Word);
-    constructor Create(const AFileName: RawBytestring; Mode: Word; Rights: Cardinal);
-    destructor Destroy; override;
-    property FileName : RawBytestring Read FFilename;
-  end;
-  constructor TFileStream.Create(const AFileName: rawbytestring; Mode: Word);
-
-  begin
-    Create(AFileName,Mode,438);
-  end;
-
-
-  constructor TFileStream.Create(const AFileName: rawbytestring; Mode: Word; Rights: Cardinal);
-
-  Var
-    H : Thandle;
-
-  begin
-    FFileName:=AFileName;
-    If (Mode and fmCreate) > 0 then
-      H:=FileCreate(AFileName,Mode,Rights)
-    else
-      H:=FileOpen(AFileName,Mode);
-
-    If (THandle(H)=feInvalidHandle) then
-      If Mode=fmcreate then
-        raise EFCreateError.createfmt(SFCreateError,[AFileName])
-      else
-        raise EFOpenError.Createfmt(SFOpenError,[AFilename]);
-    Inherited Create(H);
-  end;
-
-
-  destructor TFileStream.Destroy;
-
-  begin
-    FileClose(Handle);
-  end;
 
 {$IFDEF FPC_BIG_ENDIAN}
 function SwapLFH(const Values: Local_File_Header_Type): Local_File_Header_Type;
@@ -819,17 +772,6 @@ begin
       Result := Result or UNIX_FILE;
 end;
 
-function CRC32Str(const s:string):DWord;
-var
-  i:Integer;
-begin
-  Result:=$FFFFFFFF;
-  if Length(S)>0 then
-    for i:=1 to Length(s) do
-      Result:=Crc_32_Tab[Byte(Result XOR LongInt(s[i]))] XOR ((Result SHR 8) AND $00FFFFFF);
-  Result:=not Result;
-end;
-
 { ---------------------------------------------------------------------
     TDeCompressor
   ---------------------------------------------------------------------}
@@ -988,6 +930,8 @@ begin
   Try
     C:=TDeCompressionStream.Create(FInFile,True);
     Try
+      if assigned(FOnProgress) then
+        fOnProgress(self,0);
       Repeat
         Count:=C.Read(Buf^,FBufferSize);
         For I:=0 to Count-1 do
@@ -998,9 +942,14 @@ begin
            begin
              if (FSize>0) and assigned(FOnProgress) Then
                FOnProgress(self,100 * ( BytesNow / FSize));
+             if assigned(FOnProgressEx) Then
+               FOnProgressEx(Self, FTotPos + BytesNow, FTotSize);
              inc(NextMark,OnBytes);
            end;
+        if FNeedToBreak then
+          Break;
       Until (Count=0);
+      FTotPos := FTotPos + FOutFile.Size;
     Finally
       C.Free;
     end;
@@ -1009,6 +958,8 @@ begin
   end;
  if assigned(FOnProgress) then
    fOnProgress(self,100.0);
+ if assigned(FOnProgressEx) then
+   FOnProgressEx(Self, FTotPos, FTotSize);
   Crc32Val:=NOT Crc32Val;
 end;
 
@@ -1488,7 +1439,7 @@ Begin
   With LocalHdr do
     begin
     Signature := LOCAL_FILE_HEADER_SIGNATURE;
-    Extract_Version_Reqd := 20; //default value, v2.0
+    Extract_Version_Reqd := 10; //default value, v1.0
     Bit_Flag := 0;
     Compress_Method := 1;
     DateTimeToZipDateTime(Item.DateTime,Last_Mod_Date,Last_Mod_Time);
@@ -1578,13 +1529,12 @@ Begin
     LocalHdr.Extra_Field_Length:=SizeOf(LocalZip64ExtHdr)+SizeOf(LocalZip64Fld);
   FOutStream.WriteBuffer({$IFDEF ENDIAN_BIG}SwapLFH{$ENDIF}(LocalHdr),SizeOf(LocalHdr));
   // Append extensible field header+zip64 extensible field if needed:
-  FOutStream.WriteBuffer(ZFileName[1],Length(ZFileName));
   if IsZip64 then
   begin
-    LocalZip64ExtHdr.Header_ID:=ZIP64_HEADER_ID;
     FOutStream.WriteBuffer({$IFDEF ENDIAN_BIG}SwapEDFH{$ENDIF}(LocalZip64ExtHdr),SizeOf(LocalZip64ExtHdr));
     FOutStream.WriteBuffer({$IFDEF ENDIAN_BIG}SwapZ64EIF{$ENDIF}(LocalZip64Fld),SizeOf(LocalZip64Fld));
   end;
+  FOutStream.WriteBuffer(ZFileName[1],Length(ZFileName));
 End;
 
 
@@ -1660,7 +1610,6 @@ Begin
     {$IFDEF OS2}
       MadeBy_Version := MadeBy_Version or (OS_OS2 shl 8);
     {$ENDIF}
-      {$warning TODO: find a way to recognize VFAT and NTFS}
       // Copy over extract_version_reqd..extra_field_length
       Move(LocalHdr.Extract_Version_Reqd, Extract_Version_Reqd, 26);
       if (IsZip64) and (Extract_Version_Reqd<45) then
@@ -1702,7 +1651,7 @@ Begin
 
     Inc(ACount);
     // Move past compressed file data to next header:
-    if Iszip64 then
+    if LocalHdr.Compressed_Size=$FFFFFFFF then
       FOutStream.Seek(SavePos + LocalZip64Fld.Compressed_Size,soBeginning)
     else
       FOutStream.Seek(SavePos + LocalHdr.Compressed_Size,soBeginning);
@@ -1826,6 +1775,7 @@ Begin
       With CreateCompressor(Item, FinFile,ZipStream) do
         Try
           OnProgress:=Self.OnProgress;
+          OnProgressEx := Self.OnProgressEx;
           OnPercent:=Self.OnPercent;
           Compress;
           CRC:=Crc32Val;
@@ -1860,7 +1810,7 @@ begin
   SaveToFile(FileName);
 end;
 
-procedure TZipper.SaveToFile(AFileName: RawByteString);
+procedure TZipper.SaveToFile(AFileName: string);
 var
   lStream: TFileStream;
 begin
@@ -1905,7 +1855,7 @@ begin
     FBufSize:=Value;
 end;
 
-Procedure TZipper.SetFileName(Value : RawByteString);
+Procedure TZipper.SetFileName(Value : String);
 
 begin
   If FZipping then
@@ -1913,7 +1863,7 @@ begin
   FFileName:=Value;
 end;
 
-Procedure TZipper.ZipFiles(AFileName : RawByteString; FileList : TStrings);
+Procedure TZipper.ZipFiles(AFileName : String; FileList : TStrings);
 
 begin
   FFileName:=AFileName;
@@ -1926,7 +1876,7 @@ begin
   ZipAllFiles;
 end;
 
-procedure TZipper.ZipFiles(AFileName: RawByteString; Entries: TZipFileEntries);
+procedure TZipper.ZipFiles(AFileName: String; Entries: TZipFileEntries);
 begin
   FFileName:=AFileName;
   ZipFiles(Entries);
@@ -2028,12 +1978,12 @@ Begin
 End;
 
 
-function TUnZipper.OpenOutput(OutFileName: RawByteString;
-  out OutStream: TStream; Item: TFullZipFileEntry): Boolean;
+function TUnZipper.OpenOutput(OutFileName: String; var OutStream: TStream;
+  Item: TFullZipFileEntry): Boolean;
 Var
-  Path: RawByteString;
+  Path: String;
   OldDirectorySeparators: set of char;
-  
+
 Begin
   { the default RTL behavior is broken on Unix platforms
     for Windows compatibility: it allows both '/' and '\'
@@ -2068,9 +2018,8 @@ Begin
       ForceDirectories(Path);
     AllowDirectorySeparators:=OldDirectorySeparators;
     OutStream:=TFileStream.Create(OutFileName,fmCreate);
-	
     end;
-	
+
   AllowDirectorySeparators:=OldDirectorySeparators;
   Result:=True;
   If Assigned(FOnStartFile) then
@@ -2078,8 +2027,7 @@ Begin
 End;
 
 
-procedure TUnZipper.CloseOutput(Item: TFullZipFileEntry; var OutStream: TStream
-  );
+procedure TUnZipper.CloseOutput(Item: TFullZipFileEntry; var OutStream: TStream);
 
 Begin
   if Assigned(FOnDoneStream) then
@@ -2105,13 +2053,9 @@ end;
 procedure TUnZipper.ReadZipHeader(Item: TFullZipFileEntry; out AMethod: Word);
 Var
   S : String;
-  U : UTF8String;
   D : TDateTime;
   ExtraFieldHdr: Extensible_Data_Field_Header_Type;
   SavePos: int64; //could be qword but limited by stream
-  // Infozip unicode path
-  Infozip_Unicode_Path_Ver:Byte;
-  Infozip_Unicode_Path_CRC32:DWord;
 Begin
   FZipStream.Seek(Item.HdrPos,soBeginning);
   FZipStream.ReadBuffer(LocalHdr,SizeOf(LocalHdr));
@@ -2121,7 +2065,6 @@ Begin
   FillChar(LocalZip64Fld,SizeOf(LocalZip64Fld),0); //ensure no erroneous info
   With LocalHdr do
     begin
-      Item.FBitFlags:=Bit_Flag;
       SetLength(S,Filename_Length);
       FZipStream.ReadBuffer(S[1],Filename_Length);
       Item.ArchiveFileName:=S;
@@ -2130,7 +2073,7 @@ Begin
       if Extra_Field_Length>0 then
         begin
         SavePos := FZipStream.Position;
-        if (LocalHdr.Extra_Field_Length>=SizeOf(ExtraFieldHdr)) then
+        if (LocalHdr.Extra_Field_Length>=SizeOf(ExtraFieldHdr)+SizeOf(LocalZip64Fld)) then
           while FZipStream.Position<SavePos+LocalHdr.Extra_Field_Length do
             begin
             FZipStream.ReadBuffer(ExtraFieldHdr, SizeOf(ExtraFieldHdr));
@@ -2143,32 +2086,7 @@ Begin
             {$IFDEF FPC_BIG_ENDIAN}
               LocalZip64Fld := SwapZ64EIF(LocalZip64Fld);
             {$ENDIF}
-              end
-            // Infozip unicode path
-            else if ExtraFieldHdr.Header_ID=INFOZIP_UNICODE_PATH_ID then
-              begin
-              FZipStream.ReadBuffer(Infozip_Unicode_Path_Ver,1);
-              if Infozip_Unicode_Path_Ver=1 then
-                begin
-                FZipStream.ReadBuffer(Infozip_Unicode_Path_CRC32,sizeof(Infozip_Unicode_Path_CRC32));
-                {$IFDEF FPC_BIG_ENDIAN}
-                Infozip_Unicode_Path_CRC32:=SwapEndian(Infozip_Unicode_Path_CRC32);
-                {$ENDIF}
-                if CRC32Str(S)=Infozip_Unicode_Path_CRC32 then
-                  begin
-                  SetLength(U,ExtraFieldHdr.Data_Size-5);
-                  FZipStream.ReadBuffer(U[1],Length(U));
-                  Item.UTF8ArchiveFileName:=U;
-                  Item.UTF8DiskFileName:=U;
-                  end
-                else
-                  FZipStream.Seek(ExtraFieldHdr.Data_Size-5,soFromCurrent);
-                end
-              else
-                FZipStream.Seek(ExtraFieldHdr.Data_Size-1,soFromCurrent);
-              end
-            else
-              FZipStream.Seek(ExtraFieldHdr.Data_Size,soFromCurrent);
+              end;
             end;
         // Move past extra fields
         FZipStream.Seek(SavePos+Extra_Field_Length,soFromBeginning);
@@ -2336,10 +2254,6 @@ Var
   NewNode   : TFullZipFileEntry;
   D : TDateTime;
   S : String;
-  U : UTF8String;
-  // infozip unicode path
-  Infozip_unicode_path_ver : byte; // always 1
-  Infozip_unicode_path_crc32 : DWord;
 Begin
   FindEndHeaders(EndHdr, EndHdrPos,
     EndZip64Hdr, EndZip64HdrPos);
@@ -2387,7 +2301,6 @@ Begin
       NewNode:=FEntries.Add as TFullZipFileEntry;
       // Header position will be corrected later with zip64 version, if needed..
       NewNode.HdrPos := Local_Header_Offset;
-      NewNode.FBitFlags:=Bit_Flag;
       SetLength(S,Filename_Length);
       FZipStream.ReadBuffer(S[1],Filename_Length);
       SavePos:=FZipStream.Position; //After fixed part of central directory...
@@ -2431,28 +2344,6 @@ Begin
               NewNode.HdrPos := Zip64Field.Relative_Hdr_Offset;
               end;
             end
-            // infozip unicode path extra field
-          else if ExtraFieldHeader.Header_ID = INFOZIP_UNICODE_PATH_ID then
-            begin
-            FZipStream.ReadBuffer(Infozip_unicode_path_ver,1);
-            if Infozip_unicode_path_ver=1 then
-              begin
-              FZipStream.ReadBuffer(Infozip_unicode_path_crc32,sizeof(Infozip_unicode_path_crc32));
-              {$IFDEF FPC_BIG_ENDIAN}
-              Infozip_unicode_path_crc32:=SwapEndian(Infozip_unicode_path_crc32);
-              {$ENDIF}
-              if CRC32Str(S)=Infozip_unicode_path_crc32 then
-                begin
-                SetLength(U,ExtraFieldHeader.Data_Size-5);
-				FZipStream.ReadBuffer(U[1],Length(U));
-                NewNode.UTF8ArchiveFileName:=U;
-                end
-              else
-                FZipStream.Seek(ExtraFieldHeader.Data_Size-5,soFromCurrent);
-              end
-            else
-              FZipStream.Seek(ExtraFieldHeader.Data_Size-1,soFromCurrent);
-            end
           else
             begin
               // Read past non-Zip64 extra field
@@ -2480,120 +2371,68 @@ end;
 procedure TUnZipper.UnZipOneFile(Item: TFullZipFileEntry);
 
 Var
+  Attrs: Longint;
   ZMethod : Word;
-{$ifdef unix}
-  LinkTargetStream: TStringStream;
-{$endif}
-  OutputFileName: RawByteString;
+  OutputFileName: string;
   FOutStream: TStream;
   IsLink: Boolean;
   IsCustomStream: Boolean;
-  U : UnicodeString;
-
-  Procedure SetAttributes;
-  Var
-    Attrs : Longint;
-  begin
-    // set attributes
-    FileSetDate(OutputFileName, DateTimeToFileDate(Item.DateTime));
-    if (Item.Attributes <> 0) then
-      begin
-      Attrs := 0;
-      {$IFDEF UNIX}
-      if (Item.OS in [OS_UNIX,OS_OSX]) then Attrs := Item.Attributes;
-      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then
-        Attrs := ZipFatAttrsToUnixAttrs(Item.Attributes);
-      {$ELSE}
-      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then Attrs := Item.Attributes;
-      if (Item.OS in [OS_UNIX,OS_OSX]) then
-        Attrs := ZipUnixAttrsToFatAttrs(ExtractFileName(Item.ArchiveFileName), Item.Attributes);
-      {$ENDIF}
-      if Attrs <> 0 then
-        begin
-        {$IFDEF UNIX}
-        FpChmod(OutputFileName, Attrs);
-        {$ELSE}
-        FileSetAttr(OutputFileName, Attrs);
-        {$ENDIF}
-        end;
-      end;
-  end;
-
+  {$IFDEF UNIX}LinkTargetStream: TStringStream;{$ENDIF}
   procedure DoUnzip(const Dest: TStream);
-
+  var
+    Count: Int64;
   begin
     if ZMethod=0 then
       begin
       if (LocalHdr.Compressed_Size<>0) then
         begin
-        if LocalZip64Fld.Compressed_Size>0 then
-          Dest.CopyFrom(FZipStream,LocalZip64Fld.Compressed_Size)
-        else
-          Dest.CopyFrom(FZipStream,LocalHdr.Compressed_Size);
-        {$warning TODO: Implement CRC Check}
-        end;
-      end
-    else
-      With CreateDecompressor(Item, ZMethod, FZipStream, Dest) do
-        Try
-          OnProgress:=Self.OnProgress;
-          OnPercent:=Self.OnPercent;
-          DeCompress;
-          if Item.CRC32 <> Crc32Val then
-            raise EZipError.CreateFmt(SErrInvalidCRC,[Item.ArchiveFileName]);
-        Finally
-          Free;
-        end;
-  end;
-
-  Procedure GetOutputFileName;
-
-  Var
-    I : Integer;
-
-  begin
-    if Not UseUTF8 then
-      OutputFileName:=StringReplace(Item.DiskFileName,'/',DirectorySeparator,[rfReplaceAll])
+          if LocalZip64Fld.Compressed_Size>0 then
+            Count:=Dest.CopyFrom(FZipStream,LocalZip64Fld.Compressed_Size)
+          else
+            Count:=Dest.CopyFrom(FZipStream,LocalHdr.Compressed_Size);
+        end
+      else
+        Count:=0;
+    end
     else
-      begin
-      // Sets codepage.
-      OutputFileName:=Item.UTF8DiskFileName;
-      U:=UTF8Decode(OutputFileName);
-      // Do not use stringreplace, it will mess up the codepage.
-      if '/'<>DirectorySeparator then
-        For I:=1 to Length(U) do
-          if U[i]='/' then
-            U[i]:=DirectorySeparator;
-      OutputFileName:=UTF8Encode(U);
-      end;
-    if (Not IsCustomStream) and (FOutputPath<>'') then
-      begin
-      // Do not use IncludeTrailingPathdelimiter
-      OutputFileName:=FOutputPath+OutputFileName;
+    With CreateDecompressor(Item, ZMethod, FZipStream, Dest) do
+      Try
+        FTotPos := Self.FTotPos;
+        FTotSize := Self.FTotSize;
+        OnProgress:=Self.OnProgress;
+        OnProgressEx := Self.OnProgressEx;
+        OnPercent:=Self.OnPercent;
+        DeCompress;
+        Self.FTotPos := FTotPos;
+        if Item.CRC32 <> Crc32Val then
+          raise EZipError.CreateFmt(SErrInvalidCRC,[Item.ArchiveFileName]);
+      Finally
+        Free;
       end;
   end;
 
 Begin
   ReadZipHeader(Item, ZMethod);
-  if (Item.BitFlags and 1)<>0 then
-    Raise EZipError.CreateFmt(SErrEncryptionNotSupported,[Item.ArchiveFileName]);
-  if (Item.BitFlags and (1 shl 5))<>0 then
-    Raise EZipError.CreateFmt(SErrPatchSetNotSupported,[Item.ArchiveFileName]);
   // Normalize output filename to conventions of target platform.
   // Zip file always has / path separators
+  OutputFileName:=StringReplace(Item.DiskFileName,'/',DirectorySeparator,[rfReplaceAll]);
+
   IsCustomStream := Assigned(FOnCreateStream);
-  GetOutputFileName;
+
+  if (IsCustomStream = False) and (FOutputPath<>'') then
+    OutputFileName:=IncludeTrailingPathDelimiter(FOutputPath)+OutputFileName;
+
   IsLink := Item.IsLink;
+
 {$IFNDEF UNIX}
   if IsLink and Not IsCustomStream then
-    begin
-    {$warning TODO: Implement symbolic link creation for non-unix, e.g.
-    Windows NTFS}
+  begin
     IsLink := False;
-    end;
+  end;
 {$ENDIF}
+
   if IsCustomStream then
-    begin
+  begin
     try
       OpenOutput(OutputFileName, FOutStream, Item);
       if (IsLink = False) and (Item.IsDirectory = False) then
@@ -2601,48 +2440,69 @@ Begin
     Finally
       CloseOutput(Item, FOutStream);
     end;
-    end
+  end
   else
-    begin
+  begin
     if IsLink then
+    begin
+    {$IFDEF UNIX}
+      LinkTargetStream := TStringStream.Create('');
+      try
+        DoUnzip(LinkTargetStream);
+        fpSymlink(PChar(LinkTargetStream.DataString), PChar(OutputFileName));
+      finally
+        LinkTargetStream.Free;
+      end;
+    {$ENDIF}
+    end
+    else
+    begin
+      if Item.IsDirectory then
+        CreateDir(OutputFileName)
+      else
       begin
-      {$IFDEF UNIX}
-        LinkTargetStream := TStringStream.Create('');
         try
-          DoUnzip(LinkTargetStream);
-          fpSymlink(PChar(LinkTargetStream.DataString), PChar(OutputFileName));
-        finally
-          LinkTargetStream.Free;
+          OpenOutput(OutputFileName, FOutStream, Item);
+          DoUnzip(FOutStream);
+        Finally
+          CloseOutput(Item, FOutStream);
         end;
-      {$ENDIF}
-      end
-    else if Item.IsDirectory then
-      CreateDir(OutputFileName)
-    else
-      begin
-      try
-        OpenOutput(OutputFileName, FOutStream, Item);
-        DoUnzip(FOutStream);
-      Finally
-        CloseOutput(Item, FOutStream);
-      end;
       end;
-    SetAttributes;
     end;
-end;
-
+  end;
 
-procedure TUnZipper.UnZipAllFiles;
+  if Not IsCustomStream then
+  begin
+    // set attributes
+    FileSetDate(OutputFileName, DateTimeToFileDate(Item.DateTime));
 
-  Function IsMatch(I : TFullZipFileEntry) : Boolean;
+    if (Item.Attributes <> 0) then
+    begin
+      Attrs := 0;
+    {$IFDEF UNIX}
+      if (Item.OS in [OS_UNIX,OS_OSX]) then Attrs := Item.Attributes;
+      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then
+        Attrs := ZipFatAttrsToUnixAttrs(Item.Attributes);
+    {$ELSE}
+      if (Item.OS in [OS_FAT,OS_NTFS,OS_OS2,OS_VFAT]) then Attrs := Item.Attributes;
+      if (Item.OS in [OS_UNIX,OS_OSX]) then
+        Attrs := ZipUnixAttrsToFatAttrs(ExtractFileName(Item.ArchiveFileName), Item.Attributes);
+    {$ENDIF}
 
-  begin
-    if UseUTF8 then
-      Result:=(FFiles.IndexOf(I.UTF8ArchiveFileName)<>-1)
-    else
-      Result:=(FFiles.IndexOf(I.ArchiveFileName)<>-1)
+      if Attrs <> 0 then
+      begin
+    {$IFDEF UNIX}
+      FpChmod(OutputFileName, Attrs);
+    {$ELSE}
+      FileSetAttr(OutputFileName, Attrs);
+    {$ENDIF}
+      end;
+    end;
   end;
+end;
+
 
+procedure TUnZipper.UnZipAllFiles;
 Var
   Item : TFullZipFileEntry;
   I : integer; //Really QWord but limited to FEntries.Count
@@ -2655,12 +2515,24 @@ Begin
     OpenInput;
     Try
       ReadZipDirectory;
+      FTotPos := 0;
+      FTotSize := 0;
       for i:=0 to FEntries.Count-1 do
-        begin
+      begin
+        Item := FEntries[i];
+        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
+          FTotSize := FTotSize + TZipFileEntry(Item).Size;
+      end;
+      for i:=0 to FEntries.Count-1 do
+      begin
         Item:=FEntries[i];
-        if AllFiles or IsMatch(Item) then
+        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
           UnZipOneFile(Item);
-        end;
+        if FNeedToBreak then
+          Break;
+      end;
+      if Assigned(FOnProgressEx) then
+        FOnProgressEx(Self, FTotSize, FTotSize);
     Finally
       CloseInput;
     end;
@@ -2679,7 +2551,7 @@ begin
     FBufSize:=Value;
 end;
 
-procedure TUnZipper.SetFileName(Value: RawByteString);
+procedure TUnZipper.SetFileName(Value: String);
 
 begin
   If FUnZipping then
@@ -2687,25 +2559,14 @@ begin
   FFileName:=Value;
 end;
 
-procedure TUnZipper.SetOutputPath(Value: RawByteString);
-
-Var
-  DS : RawByteString;
-
+procedure TUnZipper.SetOutputPath(Value: String);
 begin
   If FUnZipping then
     Raise EZipError.Create(SErrFileChange);
   FOutputPath:=Value;
-  If (FOutputPath<>'') and (FoutputPath[Length(FoutputPath)]<>DirectorySeparator) then
-    begin
-    // Preserve codepage of outputpath
-    DS:=DirectorySeparator;
-    SetCodePage(DS,StringCodePage(FoutputPath),False);
-    FOutputPath:=FoutputPath+DS;
-    end;
 end;
 
-procedure TUnZipper.UnZipFiles(AFileName: RawByteString; FileList: TStrings);
+procedure TUnZipper.UnZipFiles(AFileName: String; FileList: TStrings);
 
 begin
   FFileName:=AFileName;
@@ -2718,7 +2579,7 @@ begin
   UnZipAllFiles;
 end;
 
-procedure TUnZipper.UnZipAllFiles(AFileName: RawByteString);
+procedure TUnZipper.UnZipAllFiles(AFileName: String);
 
 begin
   FFileName:=AFileName;
@@ -2784,6 +2645,46 @@ begin
   end;
 end;
 
+function TUnZipper.GetZipSize(var IsDirZipped: Boolean; var ABaseDir: String): Int64;
+var
+  I: Integer;
+  Item: TFullZipFileEntry;
+  AllFiles: Boolean;
+  P: Integer;
+begin
+  AllFiles := (FFiles.Count = 0);
+  OpenInput;
+  try
+    ReadZipDirectory;
+    Result := 0;
+    if FEntries.Count > 0 then
+    begin
+      P := Pos('/', TZipFileEntry(FEntries.Items[0]).ArchiveFileName);
+      if P = 0 then
+        P := Pos('\', TZipFileEntry(FEntries.Items[0]).ArchiveFileName);
+      if P <> 0 then
+      ABaseDir := Copy(TZipFileEntry(FEntries.Items[0]).ArchiveFileName, 1, P);
+    end;
+    for i:=0 to FEntries.Count-1 do
+    begin
+      Item := FEntries[i];
+      if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
+      begin
+        Result := Result + TZipFileEntry(Item).Size;
+        if IsDirZipped then
+          if Pos(ABaseDir, Item.ArchiveFileName) = 0 then
+            IsDirZipped := False;
+      end;
+    end;
+    if not IsDirZipped then
+      ABaseDir := ''
+    else
+      ABaseDir := Copy(ABaseDir, 1, Length(ABaseDir) - 1);
+  finally
+    CloseInput;
+  end;
+end;
+
 destructor TUnZipper.Destroy;
 
 begin
@@ -2802,20 +2703,6 @@ begin
     Result:=FDiskFileName;
 end;
 
-function TZipFileEntry.GetUTF8ArchiveFileName: UTF8String;
-begin
-  Result:=FUTF8FileName;
-  If Result='' then
-    Result:=ArchiveFileName;
-end;
-
-function TZipFileEntry.GetUTF8DiskFileName: UTF8String;
-begin
-  Result:=FUTF8DiskFileName;
-  If Result='' then
-    Result:=DiskFileName;
-end;
-
 constructor TZipFileEntry.Create(ACollection: TCollection);
 
 begin
@@ -2878,26 +2765,6 @@ begin
     FDiskFileName:=StringReplace(AValue,'/',DirectorySeparator,[rfReplaceAll]);
 end;
 
-procedure TZipFileEntry.SetUTF8ArchiveFileName(AValue: UTF8String);
-begin
-  FUTF8FileName:=AValue;
-  If ArchiveFileName='' then
-    if DefaultSystemCodePage<>CP_UTF8 then
-      ArchiveFileName:=Utf8ToAnsi(AValue)
-    else
-      ArchiveFileName:=AValue;
-end;
-
-procedure TZipFileEntry.SetUTF8DiskFileName(AValue: UTF8String);
-begin
-  FUTF8DiskFileName:=AValue;
-  If DiskFileName='' then
-    if DefaultRTLFileSystemCodePage<>CP_UTF8 then
-      DiskFileName:=Utf8ToAnsi(AValue)
-    else
-      DiskFileName:=AValue;
-end;
-
 
 procedure TZipFileEntry.Assign(Source: TPersistent);
 
-- 
2.12.0

Marco van de Voort

2017-03-02 16:29

manager   ~0098570

1. probably because string is not UTF8string in free pascal, and assignment to string would mangle foreign encodings.

Michael Van Canneyt

2017-03-02 18:35

administrator   ~0098576

You'll have to revert the RawByteString to String change.

It kills the unicode support.

Really Juha, you should check the patches more carefully.

Juha Manninen

2017-03-02 18:44

reporter   ~0098577

> ... string is not UTF8string in free pascal, and assignment to string would mangle foreign encodings.

I thought the dynamic string encoding would take care of all that.
We are talking about filenames here, not about the compressed data.
But ok, apparently I don't understand all details involved.

Marco van de Voort

2017-03-02 23:15

manager   ~0098580

The currentcode loads the paths in a rawbytestring and then manually converts them to utf8 (and vice versa)

Your patch loads them in a string, which means all characters not in the local encoding get mangled, and THEN convert the mangled result with a lot of squares in it to utf8.

Balázs Székely

2017-03-03 08:37

reporter   ~0098584

Last edited: 2017-03-03 08:44

View 4 revisions

I think the essence of my changes was lost somewhere along the line. I will try to describe the issues that I found last year, then a solution as a patch against the current trunk.
1. The issue
When unzipping a large archive(multiple sub directories + lot of files) there is no way to show a progress to the user. TProgressEvent is not good enough, since it deals only with individual files not with the archive as a whole. More over there is no possibility to cancel the unzip process.
2. The solution is to add another event TProgressEventEx + the FNeedToBreak boolean. TProgressEventEx has two parameters ATotPos and ATotSize. The first one shows how many bytes where extracted, the second one the total size of the items after extraction. For more details please see zipper.patch.

Balázs Székely

2017-03-03 08:37

reporter  

zipper.patch (4,077 bytes)
Index: zipper.pp
===================================================================
--- zipper.pp	(revision 35505)
+++ zipper.pp	(working copy)
@@ -198,6 +198,7 @@
 Type
 
   TProgressEvent = Procedure(Sender : TObject; Const Pct : Double) of object;
+  TProgressEventEx = Procedure(Sender : TObject; Const ATotPos, ATotSize: Int64) of object;
   TOnEndOfFileEvent = Procedure(Sender : TObject; Const Ratio : Double) of object;
   TOnStartFileEvent = Procedure(Sender : TObject; Const AFileName : String) of object;
 
@@ -234,6 +235,10 @@
     FBufferSize : LongWord;
     FOnPercent  : Integer;
     FOnProgress : TProgressEvent;
+    FOnProgressEx: TProgressEventEx;
+    FTotPos      : Int64;
+    FTotSize     : Int64;
+    FNeedToBreak : Boolean;
     Procedure UpdC32(Octet: Byte);
   Public
     Constructor Create(AInFile, AOutFile : TStream; ABufSize : LongWord); virtual;
@@ -242,6 +247,7 @@
     Property BufferSize : LongWord read FBufferSize;
     Property OnPercent : Integer Read FOnPercent Write FOnPercent;
     Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
+    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
     Property Crc32Val : LongWord Read FCrc32Val Write FCrc32Val;
   end;
 
@@ -511,9 +517,12 @@
     LocalHdr    : Local_File_Header_Type; //Local header, before compressed file data
     LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
     CentralHdr  : Central_File_Header_Type;
-
+    FTotPos     : Int64;
+    FTotSize    : Int64;
+    FNeedToBreak: Boolean;
     FOnPercent  : LongInt;
     FOnProgress : TProgressEvent;
+    FOnProgressEx : TProgressEventEx;
     FOnEndOfFile : TOnEndOfFileEvent;
     FOnStartFile : TOnStartFileEvent;
   Protected
@@ -551,6 +560,7 @@
     Property OnDoneStream : TOnCustomStreamEvent Read FOnDoneStream Write FOnDoneStream;
     Property OnPercent : Integer Read FOnPercent Write FOnPercent;
     Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
+    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
     Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
     Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
     Property FileName : RawByteString Read FFileName Write SetFileName;
@@ -559,6 +569,7 @@
     Property Files : TStrings Read FFiles;
     Property Entries : TFullZipFileEntries Read FEntries;
     Property UseUTF8 : Boolean Read FUseUTF8 Write FUseUTF8;
+    Property NeedToBreak: Boolean Read FNeedToBreak Write FNeedToBreak;
   end;
 
   EZipError = Class(Exception);
@@ -998,9 +1009,14 @@
            begin
              if (FSize>0) and assigned(FOnProgress) Then
                FOnProgress(self,100 * ( BytesNow / FSize));
+             if assigned(FOnProgressEx) Then
+               FOnProgressEx(Self, FTotPos + BytesNow, FTotSize);
              inc(NextMark,OnBytes);
            end;
+        if FNeedToBreak then
+          Break;
       Until (Count=0);
+      FTotPos := FTotPos + FOutFile.Size;
     Finally
       C.Free;
     end;
@@ -1009,6 +1025,8 @@
   end;
  if assigned(FOnProgress) then
    fOnProgress(self,100.0);
+ if assigned(FOnProgressEx) then
+   FOnProgressEx(Self, FTotPos, FTotSize);
   Crc32Val:=NOT Crc32Val;
 end;
 
@@ -2655,12 +2673,25 @@
     OpenInput;
     Try
       ReadZipDirectory;
+      FTotPos := 0;
+      FTotSize := 0;
       for i:=0 to FEntries.Count-1 do
-        begin
+      begin
+        Item := FEntries[i];
+        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
+          FTotSize := FTotSize + TZipFileEntry(Item).Size;
+      end;
+
+      for i:=0 to FEntries.Count-1 do
+      begin
         Item:=FEntries[i];
         if AllFiles or IsMatch(Item) then
           UnZipOneFile(Item);
-        end;
+        if FNeedToBreak then
+          Break;
+      end;
+      if Assigned(FOnProgressEx) then
+        FOnProgressEx(Self, FTotSize, FTotSize);
     Finally
       CloseInput;
     end;
zipper.patch (4,077 bytes)

Juha Manninen

2017-03-03 10:39

reporter   ~0098589

Last edited: 2017-03-03 10:52

View 2 revisions

Now I understand the file name handling code. I thought it could be simpler with dynamic encoding but maybe it is all needed.
Sorry for the hassle.

I also understand the code was not so general after all, and not designed for backporting. I hope Balázs can make a patch with essential changes. The RawByteString stuff must stay.
I can then refactor the code, maybe move some variables to protected section and move an application specific function TUnZipper.GetZipSize to an inherited class. It has many parameters which may not be of general interest.

Michael Van Canneyt

2017-03-03 11:11

administrator   ~0098591

I will look at integrating zipper.patch.
I will use the Terminate and Terminated approach as in httpclient

Unless something pops up, normally this weekend it should be done.

Balázs Székely

2017-03-03 11:51

reporter  

zipper1.patch (4,665 bytes)
Index: zipper.pp
===================================================================
--- zipper.pp	(revision 35506)
+++ zipper.pp	(working copy)
@@ -198,6 +198,7 @@
 Type
 
   TProgressEvent = Procedure(Sender : TObject; Const Pct : Double) of object;
+  TProgressEventEx = Procedure(Sender : TObject; Const ATotPos, ATotSize: Int64) of object;
   TOnEndOfFileEvent = Procedure(Sender : TObject; Const Ratio : Double) of object;
   TOnStartFileEvent = Procedure(Sender : TObject; Const AFileName : String) of object;
 
@@ -234,6 +235,10 @@
     FBufferSize : LongWord;
     FOnPercent  : Integer;
     FOnProgress : TProgressEvent;
+    FOnProgressEx: TProgressEventEx;
+    FTotPos      : Int64;
+    FTotSize     : Int64;
+    FNeedToBreak : Boolean;
     Procedure UpdC32(Octet: Byte);
   Public
     Constructor Create(AInFile, AOutFile : TStream; ABufSize : LongWord); virtual;
@@ -242,6 +247,7 @@
     Property BufferSize : LongWord read FBufferSize;
     Property OnPercent : Integer Read FOnPercent Write FOnPercent;
     Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
+    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
     Property Crc32Val : LongWord Read FCrc32Val Write FCrc32Val;
   end;
 
@@ -511,9 +517,12 @@
     LocalHdr    : Local_File_Header_Type; //Local header, before compressed file data
     LocalZip64Fld   : Zip64_Extended_Info_Field_Type; //header is in LocalZip64ExtHdr
     CentralHdr  : Central_File_Header_Type;
-
+    FTotPos     : Int64;
+    FTotSize    : Int64;
+    FNeedToBreak: Boolean;
     FOnPercent  : LongInt;
     FOnProgress : TProgressEvent;
+    FOnProgressEx : TProgressEventEx;
     FOnEndOfFile : TOnEndOfFileEvent;
     FOnStartFile : TOnStartFileEvent;
   Protected
@@ -551,6 +560,7 @@
     Property OnDoneStream : TOnCustomStreamEvent Read FOnDoneStream Write FOnDoneStream;
     Property OnPercent : Integer Read FOnPercent Write FOnPercent;
     Property OnProgress : TProgressEvent Read FOnProgress Write FOnProgress;
+    Property OnProgressEx : TProgressEventEx Read FOnProgressEx Write FOnProgressEx;
     Property OnStartFile : TOnStartFileEvent Read FOnStartFile Write FOnStartFile;
     Property OnEndFile : TOnEndOfFileEvent Read FOnEndOfFile Write FOnEndOfFile;
     Property FileName : RawByteString Read FFileName Write SetFileName;
@@ -559,6 +569,7 @@
     Property Files : TStrings Read FFiles;
     Property Entries : TFullZipFileEntries Read FEntries;
     Property UseUTF8 : Boolean Read FUseUTF8 Write FUseUTF8;
+    Property NeedToBreak: Boolean Read FNeedToBreak Write FNeedToBreak;
   end;
 
   EZipError = Class(Exception);
@@ -998,9 +1009,14 @@
            begin
              if (FSize>0) and assigned(FOnProgress) Then
                FOnProgress(self,100 * ( BytesNow / FSize));
+             if assigned(FOnProgressEx) Then
+               FOnProgressEx(Self, FTotPos + BytesNow, FTotSize);
              inc(NextMark,OnBytes);
            end;
+        if FNeedToBreak then
+          Break;
       Until (Count=0);
+      FTotPos := FTotPos + FOutFile.Size;
     Finally
       C.Free;
     end;
@@ -1009,6 +1025,8 @@
   end;
  if assigned(FOnProgress) then
    fOnProgress(self,100.0);
+ if assigned(FOnProgressEx) then
+   FOnProgressEx(Self, FTotPos, FTotSize);
   Crc32Val:=NOT Crc32Val;
 end;
 
@@ -2536,9 +2554,15 @@
     else
       With CreateDecompressor(Item, ZMethod, FZipStream, Dest) do
         Try
+          FTotPos := Self.FTotPos;
+          FTotSize := Self.FTotSize;
           OnProgress:=Self.OnProgress;
+          OnProgressEx := Self.OnProgressEx;
           OnPercent:=Self.OnPercent;
+          OnProgress:=Self.OnProgress;
+          OnPercent:=Self.OnPercent;
           DeCompress;
+          Self.FTotPos := FTotPos; 
           if Item.CRC32 <> Crc32Val then
             raise EZipError.CreateFmt(SErrInvalidCRC,[Item.ArchiveFileName]);
         Finally
@@ -2655,12 +2679,25 @@
     OpenInput;
     Try
       ReadZipDirectory;
+      FTotPos := 0;
+      FTotSize := 0;
       for i:=0 to FEntries.Count-1 do
-        begin
+      begin
+        Item := FEntries[i];
+        if AllFiles or (FFiles.IndexOf(Item.ArchiveFileName)<>-1) then
+          FTotSize := FTotSize + TZipFileEntry(Item).Size;
+      end;
+
+      for i:=0 to FEntries.Count-1 do
+      begin
         Item:=FEntries[i];
         if AllFiles or IsMatch(Item) then
           UnZipOneFile(Item);
-        end;
+        if FNeedToBreak then
+          Break;
+      end;
+      if Assigned(FOnProgressEx) then
+        FOnProgressEx(Self, FTotSize, FTotSize);
     Finally
       CloseInput;
     end;
zipper1.patch (4,665 bytes)

Balázs Székely

2017-03-03 11:57

reporter   ~0098594

>Unless something pops up, normally this weekend it should be done.
Great! Thanks. Please work with zipper1.patch since I forget something from the original patch.

Michael Van Canneyt

2017-03-04 15:03

administrator   ~0098625

Implemented. Also for zip operations I implemented it.

I have added finer grained operation control: the terminate is passed on to compressor/decompressor objects so they will stop more soon.

The FonProgressEx was called with FTotSize twice, I changed this to FTotPos,FTotSize

Please test and close if OK.

Juha Manninen

2017-03-06 14:23

reporter   ~0098684

Thank you.
It is now used also by the Lazarus online package manager.

Issue History

Date Modified Username Field Change
2017-03-02 15:15 Juha Manninen New Issue
2017-03-02 15:15 Juha Manninen File Added: 0001-paszlib-Backport-changes-in-Zipper-used-in-OPM.patch
2017-03-02 16:29 Marco van de Voort Note Added: 0098570
2017-03-02 18:33 Michael Van Canneyt Assigned To => Michael Van Canneyt
2017-03-02 18:33 Michael Van Canneyt Status new => assigned
2017-03-02 18:35 Michael Van Canneyt Note Added: 0098576
2017-03-02 18:35 Michael Van Canneyt Status assigned => feedback
2017-03-02 18:44 Juha Manninen Note Added: 0098577
2017-03-02 18:44 Juha Manninen Status feedback => assigned
2017-03-02 23:15 Marco van de Voort Note Added: 0098580
2017-03-03 08:37 Balázs Székely Note Added: 0098584
2017-03-03 08:37 Balázs Székely File Added: zipper.patch
2017-03-03 08:39 Balázs Székely Note Edited: 0098584 View Revisions
2017-03-03 08:43 Balázs Székely Note Edited: 0098584 View Revisions
2017-03-03 08:44 Balázs Székely Note Edited: 0098584 View Revisions
2017-03-03 10:39 Juha Manninen Note Added: 0098589
2017-03-03 10:52 Juha Manninen Note Edited: 0098589 View Revisions
2017-03-03 11:11 Michael Van Canneyt Note Added: 0098591
2017-03-03 11:51 Balázs Székely File Added: zipper1.patch
2017-03-03 11:57 Balázs Székely Note Added: 0098594
2017-03-04 15:03 Michael Van Canneyt Fixed in Revision => 35517
2017-03-04 15:03 Michael Van Canneyt Note Added: 0098625
2017-03-04 15:03 Michael Van Canneyt Status assigned => resolved
2017-03-04 15:03 Michael Van Canneyt Fixed in Version => 3.1.1
2017-03-04 15:03 Michael Van Canneyt Resolution open => fixed
2017-03-04 15:03 Michael Van Canneyt Target Version => 3.2.0
2017-03-06 14:23 Juha Manninen Note Added: 0098684
2017-03-06 14:23 Juha Manninen Status resolved => closed