gzip

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

gzip
Gzip-Logo.svg
Original author(s)
Developer(s)GNU Project
Initial release31 October 1992; 28 years ago (1992-10-31)
Stable release
1.11 (GNU Gzip)[1] / 3 September 2021; 27 days ago (2021-09-03)
Repositorygit.savannah.gnu.org/cgit/gzip.git
Written inC
Operating systemUnix-like, Plan 9, Inferno
TypeData compression
LicenseGPLv3
Websitewww.gnu.org/software/gzip/

gzip is a file format and a software application used for file compression and decompression. The program was created by Jean-loup Gailly and Mark Adler as a free software replacement for the compress program used in early Unix systems, and intended for use by GNU (the "g" is from "GNU"). Version 0.1 was first publicly released on 31 October 1992, and version 1.0 followed in February 1993.

The decompression of the gzip format can be implemented as a streaming algorithm, an important feature for Web protocols, data interchange and ETL (in standard pipes) applications.

File format[edit]

gzip
Filename extension
.gz
Internet media type
application/gzip[2]
Uniform Type Identifier (UTI)org.gnu.gnu-zip-archive
Magic number1f 8b
Developed byJean-loup Gailly and Mark Adler
Type of formatData compression
Open format?Yes
Websitegzip.org (obsolete)

gzip is based on the DEFLATE algorithm, which is a combination of LZ77 and Huffman coding. DEFLATE was intended as a replacement for LZW and other patent-encumbered data compression algorithms which, at the time, limited the usability of compress and other popular archivers.

"gzip" is often also used to refer to the gzip file format, which is:

  • a 10-byte header, containing a magic number (1f 8b), the compression method (08 for DEFLATE), 1-byte of header flags, a 4-byte timestamp, compression flags and the operating system ID.
  • optional extra headers as allowed by the header flags, including the original filename, a comment field, an "extra" field, and the lower half of a CRC-32 checksum for the header section.[3]
  • a body, containing a DEFLATE-compressed payload
  • an 8-byte footer, containing a CRC-32 checksum and the length of the original uncompressed data, modulo 232.[4]
Targzip.svg

Although its file format also allows for multiple such streams to be concatenated (gzipped files are simply decompressed concatenated as if they were originally one file),[5] gzip is normally used to compress just single files.[6] Compressed archives are typically created by assembling collections of files into a single tar archive (also called tarball),[7] and then compressing that archive with gzip. The final compressed file usually has the extension .tar.gz or .tgz.

gzip is not to be confused with the ZIP archive format, which also uses DEFLATE. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed tarballs holding the same data, because it compresses files individually and cannot take advantage of redundancy between files (solid compression).

Implementations[edit]

NetBSD Gzip / FreeBSD Gzip
Developer(s)The NetBSD Foundation
Repositorycvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/gzip/
Written inC
TypeData compression
LicenseSimplified BSD License

Various implementations of the program have been written. The most commonly known is the GNU Project's implementation using Lempel-Ziv coding (LZ77). OpenBSD's version of gzip is actually the compress program, to which support for the gzip format was added in OpenBSD 3.4. The 'g' in this specific version stands for gratis.[8] FreeBSD, DragonFly BSD and NetBSD use a BSD-licensed implementation instead of the GNU version; it is actually a command-line interface for zlib intended to be compatible with the GNU implementation's options.[9] These implementations originally come from NetBSD, and support decompression of bzip2 and the Unix pack format.

An alternative compression program achieving 3-8% better compression is Zopfli. It achieves gzip-compatible compression using more exhaustive algorithms, at the expense of compression time required. It does not affect decompression time.

pigz, written by Mark Adler, is compatible with gzip and speeds up compression by using all available CPU cores and threads.[10]

gzip uses block-based compression with blocks up to 64 kilobytes in accordance to the "BZGF" specification.[11]

Damage recovery[edit]

Data in blocks prior to the first damaged part of the archive is usually fully readable. Data from blocks not demolished by damage that are located afterwards may be recoverable through difficult workarounds.[12]

Derivatives and other uses[edit]

The tar utility included in most Linux distributions can extract .tar.gz files by passing the z option, e.g., tar -zxf file.tar.gz.

zlib is an abstraction of the DEFLATE algorithm in library form which includes support both for the gzip file format and a lightweight data stream format in its API. The zlib stream format, DEFLATE, and the gzip file format were standardized respectively as RFC 1950, RFC 1951, and RFC 1952.

The gzip format is used in HTTP compression, a technique used to speed up the sending of HTML and other content on the World Wide Web. It is one of the three standard formats for HTTP compression as specified in RFC 2616. This RFC also specifies a zlib format (called "DEFLATE"), which is equal to the gzip format except that gzip adds eleven bytes of overhead in the form of headers and trailers. Still, the gzip format is sometimes recommended over zlib because Internet Explorer does not implement the standard correctly and cannot handle the zlib format as specified in RFC 1950.[13]

zlib DEFLATE is used internally by the Portable Network Graphics (PNG) format.

Since the late 1990s, bzip2, a file compression utility based on a block-sorting algorithm, has gained some popularity as a gzip replacement. It produces considerably smaller files (especially for source code and other structured text), but at the cost of memory and processing time (up to a factor of 4).[14]

AdvanceCOMP and 7-Zip can produce gzip-compatible files, using an internal DEFLATE implementation with better compression ratios than gzip itself—at the cost of more processor time compared to the reference implementation.[citation needed]

See also[edit]

Notes[edit]

  1. ^ Meyering, Jim (3 September 2021). "gzip-1.11 released (stable)". The Free Software Foundation. Retrieved 4 September 2021.
  2. ^ The 'application/zlib' and 'application/gzip' Media Types. Tools.ietf.org. doi:10.17487/RFC6713. RFC 6713. Retrieved 1 March 2014.
  3. ^ Deutsch <[email protected]>, L. Peter. "GZIP file format specification version 4.3". tools.ietf.org. Retrieved 23 July 2019.
  4. ^ Jean-loup Gailly. "GNU Gzip". Gnu.org. Retrieved 11 October 2015.
  5. ^ "GNU Gzip: Advanced usage". Gnu.org. Retrieved 28 November 2012.
  6. ^ "Can gzip compress several files into a single archive?". Gnu.org. Retrieved 27 January 2010.
  7. ^ "tarball, The Jargon File, version 4.4.7". Catb.org. Retrieved 27 January 2010.
  8. ^ "OpenBSD gzip(1) manual page". Openbsd.org. OpenBSD. Retrieved 4 February 2018.
  9. ^ "gzip". Man.freebsd.org. 9 October 2011. Retrieved 1 March 2014.
  10. ^ Mark Adler (2017). "pigz: A parallel implementation of gzip for modern multi-processor, multi-core machines". zlib.net.
  11. ^ Compress::BGZF - Read/write blocked GZIP (BGZF) files – "According to the BGZF specification, each GZIP block is limited to 64kb in size"
  12. ^ Recovering a damaged .gz file – Jean-loup Gailly, GZip.org
  13. ^ Lawrence, Eric (21 November 2014). "Compressing the Web". MSDN Blogs > IEInternals. Microsoft.
  14. ^ "Comparison Tool: 7-zip vs bzip2 vs gzip". compressionratings.com. Archived from the original on 1 November 2014. Retrieved 1 November 2014.

References[edit]

  • RFC 1952 – GZIP file format specification version 4.3

External links[edit]

Retrieved from "https://en.wikipedia.org/w/index.php?title=Gzip&oldid=1042222348"