DotNetZip created backup sometimes causes issues in (only) some unzip programs

[ic2]

Not sure if this rings a bell for any of you, but I can always give it a try.
I created a backup program based upon DotNetZip. This allows me a.o. to create AES 256 password protected zips.

Two recent full backups (zips of 40 GB) did not open correctly in a program like 7-Zip. I see only the first directory and then the first file of it. A test of the zip says
ERRORS: Headers Error Unconfirmed start of archive
WARNINGS: There are data after the end of archive

With a couple of more tests I found:
1 It is (now) always (reproducible) creating the zip again with the same selection (and probably a few more files)
2 Windows Explorer sees all directories (but can't open file because it does not support AES 256)
3 Winzip also can open everything the same zip
4 After creating some parts of the original backup the problem remained only in file from which I further found:
- It is not the size (one of the other 'partly' backups is 4x larger
- It does not seem to be the content either): one separate backup with only the root of that disk and one with only the subdirs with content resulted in 2 readable backups (=all the same files of the non working combined backup)
- When I create the same backup as differentials (so on the 'problem' disk the same root + subdirs but less files) this zip also opens in all programs.

So there does not seem to be a logical reason nor a solution. Maybe Fabrice has an idea (as X# FabZip uses DotNetZip too)?

Dick

[Fabrice]

Hi Dick,
unfortunately, I have not idea...
Have you tried with the same set of files (same condition), without AES ? or with another encryption method ?

just a shoot in the dark...

Fab

[ic2]

Hello Fabrice,

I should have added that to the list of what I tried. The problem is the same regardless the use of password protection.

Thanks for your suggestion though.

Hopefully I get an idea soon....

Dick

[Fabrice]

Hi Dick,
So ... Could it be related to the name of the Zipped files ?
Special Names, Special chars or char that are not/badly supported ??

Fab

[ic2]

Hello Fabrice,

The strange thing is that if I split the faulty content in 1 zip with the root content and one zip with the content fo all subdirectories, both zips are problem free. This while having the same content as the 1 zip file with both.
And it can't be the size either, because 1 of the other 'partly' zip files is 4 times bigger.

I also try to find some logic, so far in vain. There must be something which is written into the zip file in this specific combination which causes some zip programs to stop reading more than the first directory/file while others (like Explorer) still see the correct content in the same file. But what...

Dick

Dick

[Meinhard]

Hi Dick,
this indeed rings a bell to me. Can you show us the code you are using to create the zip?
Regards
Meinhard

[ArneOrtlinghaus]

The original old ZIP-File formats had limits on the file sizes of single files and on the file size of the zipped file of 4 GB. Perhaps it is this the reason?

Arne

[ic2]

Hello Meinhard,

this indeed rings a bell to me. Can you show us the code you are using to create the zip?

I would be great if a simple addition would do the trick!

It's actually a C# program and I think this is basically what I do. First I assign List<FileInfo> liFilesToBackup to which files to be zipped are added from an XML file.
collect. Then I call:

Code: Select all

DoZipping(cZipName,FileMode.Open,liFilesToBackup,dLastChanged,lDifferential,lIncremental);

This method then goes through the List, I stripped anything which I think is irrelevant, like checks on files dates or attributes like differentials, progress info and logs:

Code: Select all

			try
			{
				using (Ionic.Zip.ZipFile zip = new Ionic.Zip.ZipFile())
				{
					string cPW = this.PW.Password.ToString();
					if (!string.IsNullOrEmpty(cPW))
					{
						if (this.pkzip.IsChecked == true)
						{
							zip.Encryption = Ionic.Zip.EncryptionAlgorithm.PkzipWeak;
						}
						if (this.aes128.IsChecked == true)
						{
							zip.Encryption = Ionic.Zip.EncryptionAlgorithm.WinZipAes128;
						}
						if (this.aes256.IsChecked == true)
						{
							zip.Encryption = Ionic.Zip.EncryptionAlgorithm.WinZipAes256;
						}
						zip.Password = cPW;
					}
					zip.ZipErrorAction=ZipErrorAction.Skip;												
					zip.UseZip64WhenSaving = Zip64Option.AsNecessary;										
					string cFilesToBackup = liFilesToBackup.Count.ToString();
					tStart = DateTime.Now;														
					foreach (FileInfo file in liFilesToBackup)											
					{
						cFileFull = file.FullName;												
						bool lResult = CreateZipEntry(zip, file, cFileFull);														
					}   // ForEach

					try																					
					{
						zip.SaveProgress += ZipSaveProgress;																
						zip.Save(cZipName);
					} // try
					catch (Exception e)
					{
					}
				} // using
			} // try
			catch (System.Exception e)
			{
			}

The CreateZipEntry as called within that loop looks as follows, again the important code.

Is there anything you may see which is missing or wrong?

Code: Select all

		private bool CreateZipEntry(Ionic.Zip.ZipFile zip, FileInfo file, string cFileFull)
			{
			{
				string cFileToBackup;
				string cPath;
				string cFileOnly;
				string cRoot;
				string cPathOnly;
				int nLen;
				try
				{
					DateTime dtNow = File.GetLastWriteTime(cFileFull);															
					cPath = System.IO.Path.GetDirectoryName(cFileFull);
					cFileOnly = System.IO.Path.GetFileName(cFileFull);															
					cFileToBackup = cPath + "" + cFileOnly;
					cRoot = System.IO.Path.GetPathRoot(cFileFull);																
					nLen = cRoot.Length;																			
					cPathOnly = cPath.Remove(0, nLen - 1);																	

					cFileToBackup = cPathOnly + "" + cFileOnly;																
					if (cFileToBackup.Substring(0, 1) == "")																
						cFileToBackup = cFileToBackup.Substring(1);															

					System.Text.StringBuilder builder = new System.Text.StringBuilder(cFileToBackup);											
					builder.Replace("", "/");																		
					string cPathToAdd = builder.ToString();

					try
					{
						{
							var zipEntry=zip.AddFile(cFileFull, cPath);														
							zipEntry.Comment=cPath.Substring(0,1);															
						}
						if (lResetArchivebit)																		
						{
							bool lArchive = ((File.GetAttributes(cFileFull) & FileAttributes.Archive) == FileAttributes.Archive);							
							File.SetAttributes(cFileFull, File.GetAttributes(cFileFull) & ~FileAttributes.Archive);									
						}
					}
					catch (Exception e)
					{
					}
					return true;
				}		// try mainloop
				catch (Exception e)
				{
				}

			};
			// return taskZip1File.Result;
		}

Dick

[ic2]

Hello Arne,

The original old ZIP-File formats had limits on the file sizes of single files and on the file size of the zipped file of 4 GB. Perhaps it is this the reason?

No, that is not the problem. A full backup, over 4 different (logical) disks is 42 GB. When I create a separate backup for each of the disks, the one with the error is <6GB while one other without problems is 25 GB.

Dick

[ArneOrtlinghaus]

if Only 7-zip does not manage to read the zips then perhaps 7-zip is the problem. 7-zip surely has reprogrammed the zip-format and does not use another DLL.

Otherwise perhaps some files/directories are changing while trying to zip the files? The bigger the list of files to save the bigger gets this risk.
Or what is not excluded: Interaction with other tasks on the same computer, for example an Antivirus.

Arne