• 🏆 Texturing Contest #33 is OPEN! Contestants must re-texture a SD unit model found in-game (Warcraft 3 Classic), recreating the unit into a peaceful NPC version. 🔗Click here to enter!
  • It's time for the first HD Modeling Contest of 2024. Join the theme discussion for Hive's HD Modeling Contest #6! Click here to post your idea!

Compressing Software

Status
Not open for further replies.

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,192
I still see a lot of rar archives right now... specially splitted rar archives...
For the following reasons.
1. Rar is a liscenced format and I am sure has some form of liability if the tool goes wrong. 7zip has no liability as it is free (open source) software.
2. People keep recommending others to use rar because they refuse or are not aware of 7zip. The advertisement from the makers of rar is not helping kill the format.

Rar is not the only form of split-able archive. The 7z format also supports it. There is even an extension for .zip to support split archives.

It can open them just fine, just not create them. I saw a RAR recently, used by a keyboard driver that also had in it thousands of BMP images...good companions, huh? two dead formats that shouldn't be used.
Rar is a commercial format so some companies are tricked or feel obliged to use it over open source alternatives. It is the same reason why thousands of games were made with CD protection when clearly it did nothing to stop hackers (often cracked days before release), purely that stupid people felt obliged to use it.

thousands of BMP images
I would not say bmp is a dead format. It might not be the best format but there certainly are worse.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,192
No format on earth can be as bad as OBJ, not even if you force yourself to make a terrible format can you compete with it.
What about AoE 2 terrain blend data? You know, the format that was half assembly and half byte code? The one that no one knew wtf it was until someone who made Age of Empires 2 told them? Why the hell did they make this format?! Well for those of you who are young it was because all graphics were done by the CPU so to make them as fast as possible they made a absolutely stupid highly optimized assembly system that took an image and compiled it into byte code that in turn generated some assembly when the game loaded.
 

pyf

pyf

Level 32
Joined
Mar 21, 2016
Messages
2,985

What was said above is probably what your looking for. However, there are different programs for compressing different things. Depends if your trying to compress music, videos, pictures or other files.

So true...


Please look at the Multiple Files Compression tests here:
Maximum Compression (lossless data compression software)

For "best", I personally would consider two main things:
- the size of the final compressed archive
- the time and resources needed to decompress the data in it afterwards

For general daily use, I suggest:
- WinRAR (the RAR5 format is not that great imho)
- 7-Zip (v9.20 even works with W95 osr2, yay!)

For testing/experimental purposes, I suggest:
- FreeArc (audio tracks in a cue/bin are not compressed the best way / too bad development halted)
- NanoZip (too bad development halted too)


For jpg or mp3 *lossless* compression, I suggest Stuffit Deluxe (the sitx decompressor/expander is free).

Free alternatives, with or without GUI:
packJPG Options
packJPG | official developer blog for packJPG & other software by Matthias Stirner

About jpg lossless compression:
ACT JPEG Compression Test
Lossless jpg/jpeg compression test


Archive optimizers / recompressors:
- for zip archives: ZipMax (set it up with at least 7za 9.20 an deflopt)
- for 7z archives: Ultra7z Optimizer (ugly, but works)


Lastly, I point out UltraISO's isz file format, allows to directly read such compressed data from a compatible CD-Rom emulator. Use the fastest compression method to create your isz.


For those curious, I have attached my compression tests notes from 2011.
 

Attachments

  • pyf's compression tests (2011).txt
    2.4 KB · Views: 127
Last edited:

pyf

pyf

Level 32
Joined
Mar 21, 2016
Messages
2,985
Level 19
Joined
Jul 2, 2011
Messages
2,162
you could just create your own compression software

security wise java probably has one of the best compression and encryption software's since files encrypted using java code can only be decrypted by java because it doesn't pack the resulting encryption in a Zip file
 
Last edited:

pyf

pyf

Level 32
Joined
Mar 21, 2016
Messages
2,985
you could just create your own compression software
Well, the source code for AdvanceCOMP, zopfi and libdeflate is available on GitHub:
- GitHub - advancecomp
- GitHub - zopfli
- GitHub - libdeflate


On THW, we have the nice MPQ Compressor tool. But it is currently buggy, because it can not compress stock Blizzard maps.

Maybe @LeP will want to have a look at AdvanceCOMP's source code?

chat_potte-pub.jpg

** update **

Fixed!
 
Last edited:
Level 19
Joined
Jul 2, 2011
Messages
2,162
Well, the source code for AdvanceCOMP, zopfi and libdeflate is available on GitHub:
- GitHub - advancecomp
- GitHub - zopfli
- GitHub - libdeflate


On THW, we have the nice MPQ Compressor tool. But it is currently buggy, because it can not compress stock Blizzard maps.

Maybe @LeP will want to have a look at AdvanceCOMP's source code?

View attachment 260699
I can quickly write up a encryption decryption program if you guys want one.

as for compression I'm not sure if it automatically packs it in a smaller out put file. I guess we will find out after I encrypt it =)

#edit

the answer is no, encrypting with java does not automatically make the file smaller. good news is that it is a perfect encryption, meaning the file is no smaller or bigger
 
Last edited:

pyf

pyf

Level 32
Joined
Mar 21, 2016
Messages
2,985
(Re)compression first, encryption later I would say...


** update **

Which MS-DOS compressor produces the smallest archive file, regardless of both the memory and time required to process the uncompressed data:
DOS Archiver Benchmark

The year on the right speaks volumes imho.

Please keep in mind computers, data and networks have evolved a *lot* since the 1980s. And their use was certainly not as mainstream (if not 'hip') back then as it is now.

way back machine.jpg way back machine2.jpg

Please also keep in mind, our current computers will feel as outdated in 30 years. In a not so distant future, computer technology will have evolved beyond what we currently feel is possible / can be achieved.


Also worth reading imho:
The future of lossless data compression

Suffice to understand, data can still be compressed further.
 
Last edited:

pyf

pyf

Level 32
Joined
Mar 21, 2016
Messages
2,985
For the sake of completeness, I am mentioning FileOptimizer.


Everything written below is quoted from its official site:

"FileOptimizer is an advanced file optimizer featuring a lossless (no quality loss) file size reduction [...]. It keeps the behavior of the file untouched, but with its size reduced thanks to several recompression and optimization techniques [...]. Free and open source."

"- More than 45 third party tools integrated in one same tool (plugins).
- More than 275 file extensions supported, being the most complete tool available of this type.
- Smart detection of file types by analyzing its contents, no matter of what extension they have.
- Simple program interface.
- Easy to use.
- Powerful automation via command-line.
- Free and open source."
"- Suitable for home users that need to speedup file transfers no matter if they are in email attachments, P2P or shared upload servers.
- Suitable for webmasters to increase the page load speed.
- Suitable for web developers to reduce content weight.
- Suitable for desktop developers in any platform (Windows, Linux, MacOS, ...) to reduce distribution sizes and reduce load times.
- Suitable for mobile developers (Android, iOS, Windows Phone, ...) to reduce distribution sizes and reduce load times.
- Suitable for server administrators that could integrate FileOptimizer via command-line.
- Suitable for content creators and distributors to reduce content weights"
".3GP, .A, .AAI, .AC, .ACC, .ADP, .AI, .AIR, .APK, .APNG, .APPX, .APR, .ART, .AVS, .BAR, .BMP, .BPL, .BSZ, .CBT, .CBZ, .CDR, .CDT, .CHI, .CHM, .CHQ, .CHS, .CHW, .CPL, .CSL, .CSS, .DB, .DCX, .DEB, .DES, .DIB, .DLL, .DOC, .DOCM, .DOCX, .DOT, .DOTM, .DOTX, .DRV, .DWF, .DWFX, .EASM, .EML, .EMLX, .EPDF, .EPRT, .EPUB, .EXE, .FAX, .FB2, .FDF, .FITS, .FLA, .FLAC, .FPX, .FXG, .GALLERY, .GALLERYCOLLECTION, .GALLERYITEM, .GIF, .GRS, .GZ, .HDR, .HRZ, .HTM, .HTML, .ICO, .INK, .IPA, .IPK, .IPSW, .ITA, .ITS, .ITZ, .JAR, .JFI, .JFIF, .JIF, .JNG, .JP2, .JPC, .JPE, .JPEG, .JPG, .JPS, .JS, .JSON, .KML, .KMZ, .KSF, .LIB, .LIT, .LUA, .LUAC, .LXF, .LZL, .M4A, .M4B, .M4P, .M4R, .M4V, .MAX, .MBX, .MCE, .MDB, .MDT, .MDZ, .MHT, .MHTML, .MIC, .MIF, .MIFF, .MIX, .MIZ, .MKA, .MKS, .MKV, .MMIP, .MNG, .MP3, .MP4, .MPD, .MPO, .MPP, .MPT, .MSC, .MSG, .MSI, .MSP, .MST, .MSZ, .MTV, .MTW, .MVZ, .NAR, .NBK, .NOTEBOOK, .O, .OBJ, .OCX, .ODB, .ODF, .ODG, .ODP, .ODS, .ODT, .OEX, .OGA, .OGG, .OGV, .OGX, .OLE, .OLE2, .ONE, .OPT, .OSK, .OST, .OTB, .OXPS, .P7, .PALM, .PBM, .PCC, .PCD, .PCDS, .PCX, .PDB, .PDF, .PFM, .PGM, .PIC, .PICON, .PICT, .PK3, .PNG, .PNM, .PNS, .POTM, .POTX, .PPAM, .PPM, .PPS, .PPSM, .PPSX, .PPT, .PPTM, .PPTX, .PSB, .PSD, .PTIF, .PTIFF, .PUB, .PUZ, .R2SKIN, .RDB, .RFA, .RFT, .RMSKIN, .RTE, .RVT, .S3Z, .SCR, .SGML, .SLDASM, .SLDDRW, .SLDM, .SLDPRT, .SLDX, .SOU, .SPL, .SPO, .STZ, .SUN, .SVG, .SVGZ, .SWC, .SWF, .SYS, .TAR, .TGZ, .THM, .TIF, .TIFF, .VBX, .VCARD, .VCF, .VDX, .VICAR, .VIFF, .VLT, .VSD, .VSS, .VST, .VSX, .VTX, .WAL, .WBA, .WBMP, .WEBP, .WIZ, .WMZ, .WPS, .WSZ, .XAP, .XBM, .XHTML, .XL, .XLA, .XLAM, .XLC, .XLM, .XLS, .XLSM, .XLSX, .XLTM, .XLTX, .XLW, .XML, .XMZ, .XNK, .XPI, .XPM, .XPS, .XSF, .XSL, .XSLT, .XSN, .XWD, .ZIP, STICKYNOTES.SNT, and THUMBS.DB file formats among others."
- AdvanceCOMP 1.23
- APNG Optimizer 1.4
- Coherent PDF Command Line Tools (cpdf) 2.2
- CSSTidy 1.3
- DeflOpt 2.07
- defluff 0.3.2
- Document Press Version 6.01
- ECT 0.6.2.0 (511fa96)
- FLACOUT 07/03/2014
- Flasm 1.62
- Gifsicle 1.88
- Gifsicle Lossy 1.88
- Ghostscript 9.20
- GNU Strip 2.27
- Guetzli 0.2
- ImageMagick 7.0.4.7
- ImageWorsener 1.3.0
- jhead 3.00
- jpeg-archive 2.1.1
- jpegoptim 1.4.4
- jpegtran 9
- jsmin 2013-03-29
- Leanify 0.4.3.180
- libwebp 0.6.0 RC3
- mkclean 0.8.7
- mozjpeg 3.2
- MP3packer 2.04
- mp4v2 2.0.0 r504
- mutool 1.10a
- OptiPNG 0.7.6
- PCXLite 1.0
- PETrim 4.00 beta
- pingo 0.59
- PngOptimizer 2.5
- PNGOUT 02/13/2015
- pngquant 2.8.1
- pngrewrite 1.4.0
- pngwolf-zopli 1.1.1
- rehuff
- Tidy 5.2.0
- TruePNG 0.6.2.2
- UPX 3.93
- ZLib 1.2.8
- zRecompress 2.12
 
Status
Not open for further replies.
Top