7 This document gives some overall information about the internals of
8 liblzma, which should make it easier to start reading and modifying
12 1. Programming language
14 liblzma was written in C99. If you use GCC, this means that you need
15 at least GCC 3.x.x. GCC 2 isn't and won't be supported.
17 Some GCC-specific extensions are used *conditionally*. They aren't
18 required to build a full-featured library. Don't make the code rely
19 on any non-standard compiler extensions or even C99 features that
20 aren't portable between almost-C99 compatible compilers (for example
23 The public API headers are in C89. This is to avoid frustrating those
24 who maintain programs, which are strictly in C89 or C++.
26 An assumption about sizeof(size_t) is made. If this assumption is
27 wrong, some porting is probably needed:
29 sizeof(uint32_t) <= sizeof(size_t) <= sizeof(uint64_t)
32 2. Internal vs. external API
38 | liblzma public API |
47 `-- liblzma public API
49 |-- Stream info handler
50 |-- Stream Header coder
51 |-- Block Header coder
52 | `-- Filter Flags coder
66 x. Designing new filters
68 All filters must be designed so that the decoder cannot consume
69 arbitrary amount input without producing any decoded output. Failing
70 to follow this rule makes liblzma vulnerable to DoS attacks if
71 untrusted files are decoded (usually they are untrusted).
73 An example should clarify the reason behind this requirement: There
74 are two filters in the chain. The decoder of the first filter produces
75 huge amount of output (many gigabytes or more) with a few bytes of
76 input, which gets passed to the decoder of the second filter. If the
77 data passed to the second filter is interpreted as something that
78 produces no output (e.g. padding), the filter chain as a whole
79 produces no output and consumes no input for a long period of time.
81 The above problem was present in the first versions of the Subblock
82 filter. A tiny .lzma file could have taken several years to decode
83 while it wouldn't produce any output at all. The problem was fixed
84 by adding limits for number of consecutive Padding bytes, and requiring
85 that some decoded output must be produced between Set Subfilter and
89 x. Implementing new filters
91 If the filter supports embedding End of Payload Marker, make sure that
92 when your filter detects End of Payload Marker,
93 - the usage of End of Payload Marker is actually allowed (i.e. End
94 of Input isn't used); and
95 - it also checks that there is no more input coming from the next
98 The second requirement is slightly tricky. It's possible that the next
99 filter hasn't returned LZMA_STREAM_END yet. It may even need a few
100 bytes more input before it will do so. You need to give it as much
101 input as it needs, and verify that it doesn't produce any output.
103 Don't call the next filter in the chain after it has returned
104 LZMA_STREAM_END (except in encoder if action == LZMA_SYNC_FLUSH).
105 It will result undefined behavior.
107 Be pedantic. If the input data isn't exactly valid, reject it.
109 At the moment, liblzma isn't modular. You will need to edit several
110 files in src/liblzma/common to include support for a new filter. grep
111 for LZMA_FILTER_LZMA to locate the files needing changes.