5 Q: What are LZMA, LZMA Utils, lzma, .lzma, liblzma, LZMA SDK, LZMA_Alone,
8 A: LZMA stands for Lempel-Ziv-Markov chain-Algorithm. LZMA is the name
9 of the compression algorithm designed by Igor Pavlov. He is the author
10 of 7-Zip, which is a great LGPL'd compression tool for Microsoft
11 Windows operating systems. In addition to 7-Zip itself, also LZMA SDK
12 is available on the website of 7-Zip. LZMA SDK contains LZMA
13 implementations in C++, Java and C#. The C++ version is the original
14 implementation which is used also in 7-Zip itself.
16 Excluding the unrar plugin, 7-Zip is free software (free as in
17 freedom). Thanks to this, it was possible to port it to POSIX
18 platforms. The port was done and is maintained by myspace (TODO:
19 myspace's real name?). p7zip is a port of 7-Zip's command line version;
20 p7zip doesn't include the 7-Zip's GUI.
22 In POSIX world, users are used to gzip and bzip2 command line tools.
23 Developers know APIs of zlib and libbzip2. LZMA Utils try to ease
24 adoption of LZMA on free operating systems by providing a compression
25 library and a set of command line tools. The library is called liblzma.
26 It provides a zlib-like API making it easy to adapt LZMA compression in
27 existing applications. The main command line tool is known as lzma,
28 whose command line syntax is very similar to that of gzip and bzip2.
30 The original command line tool from LZMA SDK (lzma.exe) was found from
31 a directory called LZMA_Alone in the LZMA SDK. It used a simple header
32 format in .lzma files. This format was also used by LZMA Utils up to
33 and including 4.32.x. In LZMA Utils documentation, LZMA_Alone refers
34 to both the file format and the command line tool from LZMA SDK.
36 Because of various limitations of the LZMA_Alone file format, a new
37 file format was developed. Extending some existing format such as .gz
38 used by gzip was considered, but these formats were found to be too
39 limited. The filename suffix for the new .lzma format is `.lzma'. The
40 same suffix is also used for files in the LZMA_Alone format. To make
41 the transition to the new format as transparent as possible, LZMA Utils
42 support both the new and old formats transparently.
44 7-Zip and LZMA SDK: <http://7-zip.org/>
45 p7zip: <http://p7zip.sourceforge.net/>
46 LZMA Utils: <http://tukaani.org/lzma/>
49 Q: What LZMA implementations there are available?
51 A: LZMA SDK contains implementations in C++, Java and C#. The C++ version
52 is the original implementation which is part of 7-Zip. LZMA SDK
53 contains also a small LZMA decoder in C.
55 A port of LZMA SDK to Pascal was made by Alan Birtles
56 <http://www.birtles.org.uk/programming/>. It should work with
57 multiple Pascal programming language implementations.
59 LZMA Utils includes liblzma, which is directly based on LZMA SDK.
60 liblzma is written in C (C99, not C89). In contrast to C++ callback
61 API used by LZMA SDK, liblzma uses zlib-like stateful C API. I do not
62 want to comment whether both/former/latter/neither API(s) are good or
63 bad. The only reason to implement a zlib-like API was, that many
64 developers are already familiar with zlib, and very many applications
65 already use zlib. Having a similar API makes it easier to include LZMA
66 support in existing applications.
68 See also <http://en.wikipedia.org/wiki/LZMA#External_links>.
71 Q: Which file formats are supported by LZMA Utils?
73 A: Even when the raw LZMA stream is always the same, it can be wrapped
74 in different container formats. The preferred format is the new .lzma
75 format. It has magic bytes (the first six bytes: 0xFF 'L' 'Z' 'M'
76 'A' 0x00). The format supports chaining up to seven filters, splitting
77 data to multiple blocks for easier multi-threading and rough
78 random-access reading. The file integrity is verified using CRC32,
79 CRC64, or SHA256, and by verifying the uncompressed size of the file.
81 LZMA SDK includes a tool called LZMA_Alone. It supports uses a
82 primitive header which includes only the mandatory stream information
83 required by the LZMA decoder. This format can be both read and
84 written by liblzma and the command line tool (use --format=alone to
87 .7z is the native archive format used by 7-Zip. This format is not
88 supported by liblzma, and probably will never be supported. You
89 should use e.g. p7zip to extract .7z files.
91 It is possible to implement custom file formats by using raw filter
92 mode in liblzma. In this mode the application needs to store the filter
93 properties and provide them to liblzma before starting to uncompress
97 Q: How can I identify files containing LZMA compressed data?
99 A: The preferred filename suffix for .lzma files is `.lzma'. `.tar.lzma'
100 may be abbreviated to `.tlz'. The same suffixes are used for files in
101 LZMA_Alone format. In practice this should be no problem since tools
102 included in LZMA Utils support both formats transparently.
104 Checking the magic bytes is easy way to detect files in the new .lzma
105 format (the first six bytes: 0xFF 'L' 'Z' 'M' 'A' 0x00). The "file"
106 command version FIXME contains magic strings for this format.
108 The old LZMA_Alone format has no magic bytes. Its header cannot contain
109 arbitrary bytes, thus it is possible to make a guess. Unfortunately the
110 guessing is usually too hard to be reliable, so don't try it unless you
114 Q: Does the lzma command line tool support sparse files?
116 A: Sparse files can (of course) be compressed like normal files, but
117 uncompression will not restore sparseness of the file. Use an archiver
118 tool to take care of sparseness before compressing the data with lzma.
120 The reason for this is that archiver tools handle files, while
121 compression tools handle streams or buffers. Being a sparse file is
122 a property of the file on the disk, not a property of the stream or
126 Q: Can I recover parts of a broken LZMA file (e.g. corrupted CD-R)?
128 A: With LZMA_Alone and single-block .lzma files, you can uncompress the
129 file until you hit the first broken byte. The data after the broken
130 position is lost. LZMA relies on the uncompression history, and if
131 bytes are missing in the middle of the file, it is impossible to
132 reliably continue after the broken section.
134 With multi-block .lzma files it may be possible to locale the next
135 block in the file and continue decoding there. A limited recovery
136 tool for this kind of situations is planned.
141 A: No, the authors are not aware of any patents that could affect LZMA.
142 However, due to nature of software patents, the authors cannot
143 guarantee, that LZMA isn't affected by any third party patent.
146 Q: Where can I find documentation about how LZMA works as an algorithm?
148 A: Read the source code, Luke. There is no documentation about LZMA
149 internals. It is possible that Igor Pavlov is the only person on
150 the Earth that completely knows and understands the algorithm.
152 You could begin by downloading LZMA SDK, and start reading from
153 the LZMA decoder to get some idea about the bitstream format.
154 Before you begin, you should know the basics of LZ77 and
155 range coding algorithms. LZMA is based on LZ77, but LZMA is
156 *a lot* more complex. Range coding is used to compress the
157 final bitstream like Huffman coding is used in Deflate.
162 A: In context of .lzma files, a filter means an implementation of a
163 compression algorithm. The primary filter is LZMA, which is why
164 the names of the tools contain the letters LZMA.
166 liblzma and the new .lzma format support also other filters than LZMA.
167 There are different types of filters, which are suitable for different
168 types of data. Thus, to select the optimal filter and settings, the
169 type of the input data being compressed needs to be known.
171 Some filters are most useful when combined with another filter like
172 LZMA. These filters increase redundancy in the data, without changing
173 the size of the data, by taking advantage of properties specific to
174 the data being compressed.
176 So far, all the filters are always reversible. That is, no matter what
177 data you pass to a filter encoder, it can be always defiltered back to
178 the original form. Because of this, it is safe to compress for example
179 a software package that contains other file types than executables
180 using a filter specific to the architechture of the package being
183 The old LZMA_Alone format supports only the LZMA filter.
186 Q: I cannot find BCJ and BCJ2 filters. Don't they exist in liblzma?
188 A: BCJ filter is called "x86" in liblzma. BCJ2 is not included,
189 because it requires using more than one encoded output stream.
192 Q: Can I use LZMA in proprietary, non-free applications?
194 A: Yes. See the file COPYING for details.
197 Q: I would like to help. What can I do?
199 A: See the TODO file. Please contact Lasse Collin before starting to do
200 anything, because it is possible that someone else is already working
204 Q: How can I contact the authors?
206 A: Lasse Collin is the maintainer of LZMA Utils. You can contact him
207 either via IRC (Larhzu on #tukaani at Freenode or IRCnet). Email
208 should work too, <lasse.collin@tukaani.org>.
210 Igor Pavlov is the father of LZMA. He is the author of 7-Zip
211 and LZMA SDK. <http://7-zip.org/>
213 NOTE: Please don't bother Igor Pavlov with questions specific