3 .\" Author: Lasse Collin
5 .\" This file has been put into the public domain.
6 .\" You can do whatever you want with this file.
8 .TH XZ 1 "2009-08-10" "Tukaani" "XZ Utils"
10 xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files
18 .BR "xz \-\-decompress" .
22 .BR "xz \-\-decompress \-\-stdout" .
26 .BR "xz \-\-format=lzma" .
30 .BR "xz \-\-format=lzma \-\-decompress" .
34 .BR "xz \-\-format=lzma \-\-decompress \-\-stdout" .
36 When writing scripts that need to decompress files, it is recommended to
39 with appropriate arguments
49 is a general-purpose data compression tool with command line syntax similar to
53 The native file format is the
55 format, but also the legacy
57 format and raw compressed streams with no container format headers
61 compresses or decompresses each
63 according to the selected operation mode.
71 reads from standard input and writes the processed data to standard output.
73 will refuse (display an error and skip the
75 to write compressed data to standard output if it is a terminal. Similarly,
77 will refuse to read compressed data from standard input if it is a terminal.
85 are written to a new file whose name is derived from the source
89 When compressing, the suffix of the target file format
93 is appended to the source filename to get the target filename.
95 When decompressing, the
99 suffix is removed from the filename to get the target filename.
101 also recognizes the suffixes
105 and replaces them with the
109 If the target file already exists, an error is displayed and the
113 Unless writing to standard output,
115 will display a warning and skip the
117 if any of the following applies:
120 is not a regular file. Symbolic links are not followed, thus they
121 are never considered to be regular files.
124 has more than one hardlink.
127 has setuid, setgid, or sticky bit set.
129 The operation mode is set to compress, and the
131 already has a suffix of the target file format
135 when compressing to the
141 when compressing to the
145 The operation mode is set to decompress, and the
147 doesn't have a suffix of any of the supported file formats
154 After successfully compressing or decompressing the
157 copies the owner, group, permissions, access time, and modification time
160 to the target file. If copying the group fails, the permissions are modified
161 so that the target file doesn't become accessible to users who didn't have
162 permission to access the source
165 doesn't support copying other metadata like access control lists
166 or extended attributes yet.
168 Once the target file has been successfully closed, the source
172 was specified. The source
174 is never removed if the output is written to standard output.
182 process makes it print progress information to standard error.
183 This has only limited use since when standard error is a terminal, using
185 will display an automatically updating progress indicator.
189 varies from a few hundred kilobytes to several gigabytes depending on
190 the compression settings. The settings used when compressing a file
191 affect also the memory usage of the decompressor. Typically the decompressor
192 needs only 5\ % to 20\ % of the amount of RAM that the compressor needed when
193 creating the file. Still, the worst-case memory usage of the decompressor
194 is several gigabytes.
196 To prevent uncomfortable surprises caused by huge memory usage,
198 has a built-in memory usage limiter. The default limit is 40 % of total
199 physical RAM. While operating systems provide ways to limit the memory usage
200 of processes, relying on it wasn't deemed to be flexible enough.
202 When compressing, if the selected compression settings exceed the memory
203 usage limit, the settings are automatically adjusted downwards and a notice
204 about this is displayed. As an exception, if the memory usage limit is
205 exceeded when compressing with
207 an error is displayed and
209 will exit with exit status
214 cannot be decompressed without exceeding the memory usage limit, an error
215 message is displayed and the file is skipped. Note that compressed files
216 may contain many blocks, which may have been compressed with different
217 settings. Typically all blocks will have roughly the same memory requirements,
218 but it is possible that a block later in the file will exceed the memory usage
219 limit, and an error about too low memory usage limit gets displayed after some
220 data has already been decompressed.
222 The absolute value of the active memory usage limit can be seen near
223 the bottom of the output of
225 The default limit can be overriden with
226 \fB\-\-memory=\fIlimit\fR.
228 .SS "Integer suffixes and special values"
229 In most places where an integer argument is expected, an optional suffix
230 is supported to easily indicate large integers. There must be no space
231 between the integer and the suffix.
234 The integer is multiplied by 1,000 (10^3). For example,
242 The integer is multiplied by 1,024 (2^10).
245 The integer is multiplied by 1,000,000 (10^6).
248 The integer is multiplied by 1,048,576 (2^20).
251 The integer is multiplied by 1,000,000,000 (10^9).
254 The integer is multiplied by 1,073,741,824 (2^30).
258 can be used to indicate the maximum integer value supported by the option.
260 If multiple operation mode options are given, the last one takes effect.
262 .BR \-z ", " \-\-compress
263 Compress. This is the default operation mode when no operation mode option
264 is specified, and no other operation mode is implied from the command name
268 .BR \-\-decompress ).
270 .BR \-d ", " \-\-decompress ", " \-\-uncompress
273 .BR \-t ", " \-\-test
274 Test the integrity of compressed
276 No files are created or removed. This option is equivalent to
277 .B "\-\-decompress \-\-stdout"
278 except that the decompressed data is discarded instead of being
279 written to standard output.
281 .BR \-l ", " \-\-list
282 View information about the compressed files. No uncompressed output is
283 produced, and no files are created or removed. In list mode, the program
284 cannot read the compressed data from standard input or from other
287 .B "This feature has not been implemented yet."
288 .SS "Operation modifiers"
290 .BR \-k ", " \-\-keep
291 Keep (don't delete) the input files.
293 .BR \-f ", " \-\-force
294 This option has several effects:
297 If the target file already exists, delete it before compressing or
300 Compress or decompress even if the input is not a regular file,
301 has more than one hardlink, or has setuid, setgid, or sticky bit set.
302 The setuid, setgid, and sticky bits are not copied to the target file.
309 doesn't recognize the type of the source file,
311 will copy the source file as is to standard output. This allows using
316 for files that have not been compressed with
320 might support new compressed file formats, which may make
322 decompress more types of files instead of copying them as is to
324 .BI \-\-format= format
325 can be used to restrict
327 to decompress only a single file format.
329 Allow writing compressed data to a terminal, and reading compressed data
333 .BR \-c ", " \-\-stdout ", " \-\-to-stdout
334 Write the compressed or decompressed data to standard output instead of
338 \fB\-S\fR \fI.suf\fR, \fB\-\-suffix=\fI.suf
339 When compressing, use
341 as the suffix for the target file instead of
345 If not writing to standard output and the source file already has the suffix
347 a warning is displayed and the file is skipped.
349 When decompressing, recognize also files with the suffix
351 in addition to files with the
357 suffix. If the source file has the suffix
359 the suffix is removed to get the target filename.
361 When compressing or decompressing raw streams
362 .RB ( \-\-format=raw ),
363 the suffix must always be specified unless writing to standard output,
364 because there is no default suffix for raw streams.
366 \fB\-\-files\fR[\fB=\fIfile\fR]
367 Read the filenames to process from
371 is omitted, filenames are read from standard input. Filenames must be
372 terminated with the newline character. If filenames are given also as
373 command line arguments, they are processed before the filenames read from
376 \fB\-\-files0\fR[\fB=\fIfile\fR]
377 This is identical to \fB\-\-files\fR[\fB=\fIfile\fR] except that the
378 filenames must be terminated with the null character.
379 .SS "Basic file format and compression options"
381 \fB\-F\fR \fIformat\fR, \fB\-\-format=\fIformat
382 Specify the file format to compress or decompress:
386 This is the default. When compressing,
390 When decompressing, the format of the input file is autodetected. Note that
391 raw streams (created with
393 cannot be autodetected.
398 file format, or accept only
400 files when decompressing.
405 Compress to the legacy
407 file format, or accept only
409 files when decompressing. The alternative name
411 is provided for backwards compatibility with LZMA Utils.
414 Compress or uncompress a raw stream (no headers). This is meant for advanced
415 users only. To decode raw streams, you need to set not only
417 but also specify the filter chain, which would normally be stored in the
418 container format headers.
421 \fB\-C\fR \fIcheck\fR, \fB\-\-check=\fIcheck
422 Specify the type of the integrity check, which is calculated from the
423 uncompressed data. This option has an effect only when compressing into the
427 format doesn't support integrity checks.
428 The integrity check (if any) is verified when the
430 file is decompressed.
438 Don't calculate an integrity check at all. This is usually a bad idea. This
439 can be useful when integrity of the data is verified by other means anyway.
442 Calculate CRC32 using the polynomial from IEEE-802.3 (Ethernet).
445 Calculate CRC64 using the polynomial from ECMA-182. This is the default, since
446 it is slightly better than CRC32 at detecting damaged files and the speed
447 difference is negligible.
450 Calculate SHA-256. This is somewhat slower than CRC32 and CRC64.
455 headers is always verified with CRC32. It is not possible to change or
459 Select compression preset. If a preset level is specified multiple times,
460 the last one takes effect.
462 The compression preset levels can be categorised roughly into three
465 .IP "\fB\-0\fR ... \fB\-2"
466 Fast presets with relatively low memory usage.
470 should give compression speed and ratios comparable to
477 is not very good (not much faster than
479 but much worse compression). In future,
481 may be indicate some fast algorithm instead of LZMA2.
482 .IP "\fB\-3\fR ... \fB\-5"
483 Good compression ratio with low to medium memory usage.
484 These are significantly slower than levels 0\-2.
485 .IP "\fB\-6\fR ... \fB\-9"
486 Excellent compression with medium to high memory usage. These are also
487 slower than the lower preset levels. The default is
489 Unless you want to maximize the compression ratio, you probably don't want
490 a higher preset level than
492 due to speed and memory usage.
495 The exact compression settings (filter chain) used by each preset may
498 versions. The settings may also vary between files being compressed, if
500 determines that modified settings will probably give better compression
501 ratio without significantly affecting compression time or memory usage.
503 Because the settings may vary, the memory usage may vary too. The following
504 table lists the maximum memory usage of each preset level, which won't be
505 exceeded even in future versions of
508 .B "FIXME: The table below is just a rough idea."
515 Preset;Compression;Decompression
532 automatically adjusts the compression settings downwards if
533 the memory usage limit would be exceeded, so it is safe to specify
534 a high preset level even on systems that don't have lots of RAM.
536 .BR \-\-fast " and " \-\-best
537 These are somewhat misleading aliases for
542 These are provided only for backwards compatibility with LZMA Utils.
543 Avoid using these options.
545 Especially the name of
547 is misleading, because the definition of best depends on the input data,
548 and that usually people don't want the very best compression ratio anyway,
549 because it would be very slow.
551 .BR \-e ", " \-\-extreme
552 Modify the compression preset (\fB\-0\fR ... \fB\-9\fR) so that a little bit
553 better compression ratio can be achieved without increasing memory usage
554 of the compressor or decompressor (exception: compressor memory usage may
555 increase a little with presets \fB\-0\fR ... \fB\-2\fR). The downside is that
556 the compression time will increase dramatically (it can easily double).
558 \fB\-M\fR \fIlimit\fR, \fB\-\-memory=\fIlimit
559 Set the memory usage limit. If this option is specied multiple times,
560 the last one takes effect. The
562 can be specified in multiple ways:
567 can be an absolute value in bytes. Using an integer suffix like
569 can be useful. Example:
570 .B "\-\-memory=80MiB"
574 can be specified as a percentage of physical RAM. Example:
579 can be reset back to its default value (currently 40 % of physical RAM)
583 The memory usage limiting can be effectively disabled by setting
587 This isn't recommended. It's usually better to use, for example,
593 can be seen near the bottom of the output of the
597 \fB\-T\fR \fIthreads\fR, \fB\-\-threads=\fIthreads
598 Specify the maximum number of worker threads to use. The default is
599 the number of available CPU cores. You can see the current value of
601 near the end of the output of the
605 The actual number of worker threads can be less than
607 if using more threads would exceed the memory usage limit.
608 In addition to CPU-intensive worker threads,
610 may use a few auxiliary threads, which don't use a lot of CPU time.
612 .B "Multithreaded compression and decompression are not implemented yet,"
613 .B "so this option has no effect for now."
614 .SS Custom compressor filter chains
615 A custom filter chain allows specifying the compression settings in detail
616 instead of relying on the settings associated to the preset levels.
617 When a custom filter chain is specified, the compression preset level options
618 (\fB\-0\fR ... \fB\-9\fR and \fB\-\-extreme\fR) are silently ignored.
620 A filter chain is comparable to piping on the UN*X command line.
621 When compressing, the uncompressed input goes to the first filter, whose
622 output goes to the next filter (if any). The output of the last filter
623 gets written to the compressed file. The maximum number of filters in
624 the chain is four, but typically a filter chain has only one or two filters.
626 Many filters have limitations where they can be in the filter chain:
627 some filters can work only as the last filter in the chain, some only
628 as a non-last filter, and some work in any position in the chain. Depending
629 on the filter, this limitation is either inherent to the filter design or
630 exists to prevent security issues.
632 A custom filter chain is specified by using one or more filter options in
633 the order they are wanted in the filter chain. That is, the order of filter
634 options is significant! When decoding raw streams
635 .RB ( \-\-format=raw ),
636 the filter chain is specified in the same order as it was specified when
639 Filters take filter-specific
641 as a comma-separated list. Extra commas in
643 are ignored. Every option has a default value, so you need to
644 specify only those you want to change.
646 \fB\-\-lzma1\fR[\fB=\fIoptions\fR], \fB\-\-lzma2\fR[\fB=\fIoptions\fR]
647 Add LZMA1 or LZMA2 filter to the filter chain. These filter can be used
648 only as the last filter in the chain.
650 LZMA1 is a legacy filter, which is supported almost solely due to the legacy
652 file format, which supports only LZMA1. LZMA2 is an updated
653 version of LZMA1 to fix some practical issues of LZMA1. The
655 format uses LZMA2, and doesn't support LZMA1 at all. Compression speed and
656 ratios of LZMA1 and LZMA2 are practically the same.
658 LZMA1 and LZMA2 share the same set of
663 Reset all LZMA1 or LZMA2
668 consist of an integer, which may be followed by single-letter preset
669 modifiers. The integer can be from
673 matching the command line options \fB\-0\fR ... \fB\-9\fR.
674 The only supported modifier is currently
683 from which the default values for the rest of the LZMA1 or LZMA2
688 Dictionary (history buffer) size indicates how many bytes of the recently
689 processed uncompressed data is kept in memory. One method to reduce size of
690 the uncompressed data is to store distance-length pairs, which
691 indicate what data to repeat from the dictionary buffer. The bigger
692 the dictionary, the better the compression ratio usually is,
693 but dictionaries bigger than the uncompressed data are waste of RAM.
695 Typical dictionary size is from 64 KiB to 64 MiB. The minimum is 4 KiB.
696 The maximum for compression is currently 1.5 GiB. The decompressor already
697 supports dictionaries up to one byte less than 4 GiB, which is the
698 maximum for LZMA1 and LZMA2 stream formats.
700 Dictionary size has the biggest effect on compression ratio.
701 Dictionary size and match finder together determine the memory usage of
702 the LZMA1 or LZMA2 encoder. The same dictionary size is required
703 for decompressing that was used when compressing, thus the memory usage of
704 the decoder is determined by the dictionary size used when compressing.
707 Specify the number of literal context bits.
710 Specify the number of literal position bits.
713 Specify the number of position bits.
718 specifies the function used to analyze the data produced by the match finder.
737 Match finder has a major effect on encoder speed, memory usage, and
738 compression ratio. Usually Hash Chain match finders are faster than
739 Binary Tree match finders. The memory usage formulas are only rough
740 estimates, which are closest to reality when
748 Hash Chain with 2- and 3-byte hashing
766 Hash Chain with 2-, 3-, and 4-byte hashing
777 Binary Tree with 2-byte hashing
788 Binary Tree with 2- and 3-byte hashing
806 Binary Tree with 2-, 3-, and 4-byte hashing
818 Specify what is considered to be a nice length for a match. Once a match
821 bytes is found, the algorithm stops looking for possibly better matches.
824 can be 2\-273 bytes. Higher values tend to give better compression ratio
825 at expense of speed. The default depends on the
830 Specify the maximum search depth in the match finder. The default is the
833 which makes the compressor determine a reasonable
840 Using very high values for
842 can make the encoder extremely slow with carefully crafted files.
845 over 1000 unless you are prepared to interrupt the compression in case it
849 When decoding raw streams
850 .RB ( \-\-format=raw ),
851 LZMA2 needs only the value of
859 \fB\-\-x86\fR[\fB=\fIoptions\fR]
861 \fB\-\-powerpc\fR[\fB=\fIoptions\fR]
863 \fB\-\-ia64\fR[\fB=\fIoptions\fR]
865 \fB\-\-arm\fR[\fB=\fIoptions\fR]
867 \fB\-\-armthumb\fR[\fB=\fIoptions\fR]
869 \fB\-\-sparc\fR[\fB=\fIoptions\fR]
870 Add a branch/call/jump (BCJ) filter to the filter chain. These filters
871 can be used only as non-last filter in the filter chain.
873 A BCJ filter converts relative addresses in the machine code to their
874 absolute counterparts. This doesn't change the size of the data, but
875 it increases redundancy, which allows e.g. LZMA2 to get better
878 The BCJ filters are always reversible, so using a BCJ filter for wrong
879 type of data doesn't cause any data loss. However, applying a BCJ filter
880 for wrong type of data is a bad idea, because it tends to make the
881 compression ratio worse.
883 Different instruction sets have have different alignment:
890 Filter;Alignment;Notes
891 x86;1;32-bit and 64-bit x86
892 PowerPC;4;Big endian only
893 ARM;4;Little endian only
894 ARM-Thumb;2;Little endian only
895 IA-64;16;Big or little endian
896 SPARC;4;Big or little endian
901 Since the BCJ-filtered data is usually compressed with LZMA2, the compression
902 ratio may be improved slightly if the LZMA2 options are set to match the
903 alignment of the selected BCJ filter. For example, with the IA-64 filter,
906 with LZMA2 (2^4=16). The x86 filter is an exception; it's usually good to
907 stick to LZMA2's default four-byte alignment when compressing x86 executables.
909 All BCJ filters support the same
916 that is used when converting between relative and absolute addresses.
919 must be a multiple of the alignment of the filter (see the table above).
920 The default is zero. In practice, the default is good; specifying
923 is almost never useful.
925 Specifying a non-zero start
927 is probably useful only if the executable has multiple sections, and there
928 are many cross-section jumps or calls. Applying a BCJ filter separately for
929 each section with proper start offset and then compressing the result as
930 a single chunk may give some improvement in compression ratio compared
931 to applying the BCJ filter with the default
933 for the whole executable.
936 \fB\-\-delta\fR[\fB=\fIoptions\fR]
937 Add Delta filter to the filter chain. The Delta filter
938 can be used only as non-last filter in the filter chain.
940 Currently only simple byte-wise delta calculation is supported. It can
941 be useful when compressing e.g. uncompressed bitmap images or uncompressed
942 PCM audio. However, special purpose algorithms may give significantly better
943 results than Delta + LZMA2. This is true especially with audio, which
944 compresses faster and better e.g. with FLAC.
953 of the delta calculation as bytes.
955 must be 1\-256. The default is 1.
959 and eight-byte input A1 B1 A2 B3 A3 B5 A4 B7, the output will be
960 A1 B1 01 02 01 02 01 02.
964 .BR \-q ", " \-\-quiet
965 Suppress warnings and notices. Specify this twice to suppress errors too.
966 This option has no effect on the exit status. That is, even if a warning
967 was suppressed, the exit status to indicate a warning is still used.
969 .BR \-v ", " \-\-verbose
970 Be verbose. If standard error is connected to a terminal,
972 will display a progress indicator.
975 twice will give even more verbose output (useful mostly for debugging).
977 .BR \-Q ", " \-\-no\-warn
978 Don't set the exit status to
980 even if a condition worth a warning was detected. This option doesn't affect
981 the verbosity level, thus both
985 have to be used to not display warnings and to not alter the exit status.
987 .BR \-h ", " \-\-help
988 Display a help message describing the most commonly used options,
989 and exit successfully.
991 .BR \-H ", " \-\-long\-help
992 Display a help message describing all features of
994 and exit successfully
996 .BR \-V ", " \-\-version
997 Display the version number of
1009 Something worth a warning occurred, but no actual errors occurred.
1011 Notices (not warnings or errors) printed on standard error don't affect
1016 A space-separated list of options is parsed from
1018 before parsing the options given on the command line. Note that only
1019 options are parsed from
1021 all non-options are silently ignored. Parsing is done with
1023 which is used also for the command line arguments.
1024 .SH "LZMA UTILS COMPATIBILITY"
1025 The command line syntax of
1027 is practically a superset of
1032 as found from LZMA Utils 4.32.x. In most cases, it is possible to replace
1033 LZMA Utils with XZ Utils without breaking existing scripts. There are some
1034 incompatibilities though, which may sometimes cause problems.
1035 .SS "Compression preset levels"
1036 The numbering of the compression level presets is not identical in
1038 and LZMA Utils. Compressor memory usage:
1046 Level;xz;LZMA Utils 4.32.x
1058 .SS "Streamed vs. non-streamed .lzma files"
1059 Uncompressed size of the file can be stored in the
1061 header. LZMA Utils does that when compressing regular files.
1062 The alternative is to mark that uncompressed size is unknown and
1063 use end of payload marker to indicate where the decompressor should stop.
1064 LZMA Utils uses this method when uncompressed size isn't known, which is
1065 the case for example in pipes.
1068 supports decompressing
1070 files with or without end of payload marker, but all
1074 will use end of payload marker and have uncompressed size marked as unknown
1077 header. This may be a problem in some (uncommon) situations. For example, a
1079 decompressor in an embedded device might work only with files that have known
1080 uncompressed size. If you hit this problem, you need to use LZMA Utils or
1083 files with known uncompressed size.
1084 .SS "Unsupported .lzma files"
1091 values up to 4. LZMA Utils can decompress files with any
1095 but always creates files with
1099 Creating files with other
1107 The implementation of the LZMA1 filter in liblzma requires
1112 must not exceed 4. Thus,
1114 files which exceed this limitation, cannot be decompressed with
1117 LZMA Utils creates only
1119 files which have dictionary size of
1121 (a power of 2), but accepts files with any dictionary size.
1122 liblzma accepts only
1124 files which have dictionary size of
1127 .RI "2^" n " + 2^(" n "\-1)."
1128 This is to decrease false positives when autodetecting
1132 These limitations shouldn't be a problem in practice, since practically all
1134 files have been compressed with settings that liblzma will accept.
1135 .SS "Trailing garbage"
1136 When decompressing, LZMA Utils silently ignore everything after the first
1138 stream. In most situations, this is a bug. This also means that LZMA Utils
1139 don't support decompressing concatenated
1143 If there is data left after the first
1147 considers the file to be corrupt. This may break obscure scripts which have
1148 assumed that trailing garbage is ignored.
1150 .SS Builds with disabled features
1152 can be built with some features disabled, which can make some features
1153 unavailable. This is never the case with normal non-embedded builds.
1155 The exact compressed output produced from the same uncompressed input file
1156 may vary between XZ Utils versions even if compression options are identical.
1157 This is because the encoder can be improved (faster or better compression)
1158 without affecting the file format. The output can vary even between different
1159 builds of XZ Utils, if different build options are used or if the endianness
1160 of the hardware is different for different builds.
1162 The above means that implementing
1166 files is not going to happen without freezing a part of the encoder
1167 implementation, which can then be used with
1169 .SS Embedded .xz decompressors
1172 decompressor implementations like XZ Embedded don't necessarily support files
1179 Since the default is \fB\-\-check=\fIcrc64\fR, you must use
1183 when creating files for embedded systems.
1185 Outside embedded systems, all
1187 format decompressors support all the
1189 types, or at least are able to decompress the file without verifying the
1190 integrity check if the particular
1194 XZ Embedded supports BCJ filters, but only with the default start offset.
1200 XZ Utils: <http://tukaani.org/xz/>
1202 XZ Embedded: <http://tukaani.org/xz/embedded.html>
1204 LZMA SDK: <http://7-zip.org/sdk.html>