NEWS 66 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407140814091410141114121413141414151416141714181419142014211422142314241425142614271428142914301431143214331434143514361437143814391440144114421443144414451446144714481449145014511452145314541455145614571458145914601461146214631464146514661467146814691470147114721473147414751476147714781479148014811482148314841485148614871488148914901491149214931494149514961497149814991500150115021503150415051506150715081509151015111512151315141515151615171518151915201521152215231524152515261527152815291530153115321533153415351536153715381539154015411542154315441545154615471548154915501551155215531554155515561557155815591560156115621563156415651566156715681569157015711572157315741575157615771578157915801581158215831584158515861587158815891590159115921593159415951596159715981599160016011602160316041605160616071608160916101611161216131614161516161617161816191620162116221623162416251626162716281629163016311632163316341635163616371638163916401641164216431644164516461647164816491650165116521653165416551656165716581659166016611662166316641665166616671668166916701671
  1. XZ Utils Release Notes
  2. ======================
  3. 5.4.2 (2023-03-18)
  4. * All fixes from 5.2.11 that were not included in 5.4.1.
  5. * If xz is built with support for the Capsicum sandbox but running
  6. in an environment that doesn't support Capsicum, xz now runs
  7. normally without sandboxing instead of exiting with an error.
  8. * liblzma:
  9. - Documentation was updated to improve the style, consistency,
  10. and completeness of the liblzma API headers.
  11. - The Doxygen-generated HTML documentation for the liblzma API
  12. header files is now included in the source release and is
  13. installed as part of "make install". All JavaScript is
  14. removed to simplify license compliance and to reduce the
  15. install size.
  16. - Fixed a minor bug in lzma_str_from_filters() that produced
  17. too many filters in the output string instead of reporting
  18. an error if the input array had more than four filters. This
  19. bug did not affect xz.
  20. * Build systems:
  21. - autogen.sh now invokes the doxygen tool via the new wrapper
  22. script doxygen/update-doxygen, unless the command line option
  23. --no-doxygen is used.
  24. - Added microlzma_encoder.c and microlzma_decoder.c to the
  25. VS project files for Windows and to the CMake build. These
  26. should have been included in 5.3.2alpha.
  27. * Tests:
  28. - Added a test to the CMake build that was forgotten in the
  29. previous release.
  30. - Added and refactored a few tests.
  31. * Translations:
  32. - Updated the Brazilian Portuguese translation.
  33. - Added Brazilian Portuguese man page translation.
  34. 5.4.1 (2023-01-11)
  35. * liblzma:
  36. - Fixed the return value of lzma_microlzma_encoder() if the
  37. LZMA options lc/lp/pb are invalid. Invalid lc/lp/pb options
  38. made the function return LZMA_STREAM_END without encoding
  39. anything instead of returning LZMA_OPTIONS_ERROR.
  40. - Windows / Visual Studio: Workaround a possible compiler bug
  41. when targeting 32-bit x86 and compiling the CLMUL version of
  42. the CRC64 code. The CLMUL code isn't enabled by the Windows
  43. project files but it is in the CMake-based builds.
  44. * Build systems:
  45. - Windows-specific CMake changes:
  46. * Don't try to enable CLMUL CRC64 code if _mm_set_epi64x()
  47. isn't available. This fixes CMake-based build with Visual
  48. Studio 2013.
  49. * Created a workaround for a build failure with windres
  50. from GNU binutils. It is used only when the C compiler
  51. is GCC (not Clang). The workaround is incompatible
  52. with llvm-windres, resulting in "XZx20Utils" instead
  53. of "XZ Utils" in the resource file, but without the
  54. workaround llvm-windres works correctly. See the
  55. comment in CMakeLists.txt for details.
  56. * Included the resource files in the xz and xzdec build
  57. rules. Building the command line tools is still
  58. experimental but possible with MinGW-w64.
  59. - Visual Studio: Added stream_decoder_mt.c to the project
  60. files. Now the threaded decompressor lzma_stream_decoder_mt()
  61. gets built. CMake-based build wasn't affected.
  62. - Updated windows/INSTALL-MSVC.txt to mention that CMake-based
  63. build is now the preferred method with Visual Studio. The
  64. project files will probably be removed after 5.4.x releases.
  65. - Changes to #defines in config.h:
  66. * HAVE_DECL_CLOCK_MONOTONIC was replaced by
  67. HAVE_CLOCK_MONOTONIC. The old macro was always defined
  68. in configure-generated config.h to either 0 or 1. The
  69. new macro is defined (to 1) only if the declaration of
  70. CLOCK_MONOTONIC is available. This matches the way most
  71. other config.h macros work and makes things simpler with
  72. other build systems.
  73. * HAVE_DECL_PROGRAM_INVOCATION_NAME was replaced by
  74. HAVE_PROGRAM_INVOCATION_NAME for the same reason.
  75. * Tests:
  76. - Fixed test script compatibility with ancient /bin/sh
  77. versions. Now the five test_compress_* tests should
  78. no longer fail on Solaris 10.
  79. - Added and refactored a few tests.
  80. * Translations:
  81. - Updated the Catalan and Esperanto translations.
  82. - Added Korean and Ukrainian man page translations.
  83. 5.4.0 (2022-12-13)
  84. This bumps the minor version of liblzma because new features were
  85. added. The API and ABI are still backward compatible with liblzma
  86. 5.2.x and 5.0.x.
  87. Since 5.3.5beta:
  88. * All fixes from 5.2.10.
  89. * The ARM64 filter is now stable. The xz option is now --arm64.
  90. Decompression requires XZ Utils 5.4.0. In the future the ARM64
  91. filter will be supported by XZ for Java, XZ Embedded (including
  92. the version in Linux), LZMA SDK, and 7-Zip.
  93. * Translations:
  94. - Updated Catalan, Croatian, German, Romanian, and Turkish
  95. translations.
  96. - Updated German man page translations.
  97. - Added Romanian man page translations.
  98. Summary of new features added in the 5.3.x development releases:
  99. * liblzma:
  100. - Added threaded .xz decompressor lzma_stream_decoder_mt().
  101. It can use multiple threads with .xz files that have multiple
  102. Blocks with size information in Block Headers. The threaded
  103. encoder in xz has always created such files.
  104. Single-threaded encoder cannot store the size information in
  105. Block Headers even if one used LZMA_FULL_FLUSH to create
  106. multiple Blocks, so this threaded decoder cannot use multiple
  107. threads with such files.
  108. If there are multiple Streams (concatenated .xz files), one
  109. Stream will be decompressed completely before starting the
  110. next Stream.
  111. - A new decoder flag LZMA_FAIL_FAST was added. It makes the
  112. threaded decompressor report errors soon instead of first
  113. flushing all pending data before the error location.
  114. - New Filter IDs:
  115. * LZMA_FILTER_ARM64 is for ARM64 binaries.
  116. * LZMA_FILTER_LZMA1EXT is for raw LZMA1 streams that don't
  117. necessarily use the end marker.
  118. - Added lzma_str_to_filters(), lzma_str_from_filters(), and
  119. lzma_str_list_filters() to convert a preset or a filter chain
  120. string to a lzma_filter[] and vice versa. These should make
  121. it easier to write applications that allow users to specify
  122. custom compression options.
  123. - Added lzma_filters_free() which can be convenient for freeing
  124. the filter options in a filter chain (an array of lzma_filter
  125. structures).
  126. - lzma_file_info_decoder() to makes it a little easier to get
  127. the Index field from .xz files. This helps in getting the
  128. uncompressed file size but an easy-to-use random access
  129. API is still missing which has existed in XZ for Java for
  130. a long time.
  131. - Added lzma_microlzma_encoder() and lzma_microlzma_decoder().
  132. It is used by erofs-utils and may be used by others too.
  133. The MicroLZMA format is a raw LZMA stream (without end marker)
  134. whose first byte (always 0x00) has been replaced with
  135. bitwise-negation of the LZMA properties (lc/lp/pb). It was
  136. created for use in EROFS but may be used in other contexts
  137. as well where it is important to avoid wasting bytes for
  138. stream headers or footers. The format is also supported by
  139. XZ Embedded (the XZ Embedded version in Linux got MicroLZMA
  140. support in Linux 5.16).
  141. The MicroLZMA encoder API in liblzma can compress into a
  142. fixed-sized output buffer so that as much data is compressed
  143. as can be fit into the buffer while still creating a valid
  144. MicroLZMA stream. This is needed for EROFS.
  145. - Added lzma_lzip_decoder() to decompress the .lz (lzip) file
  146. format version 0 and the original unextended version 1 files.
  147. Also lzma_auto_decoder() supports .lz files.
  148. - lzma_filters_update() can now be used with the multi-threaded
  149. encoder (lzma_stream_encoder_mt()) to change the filter chain
  150. after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH.
  151. - In lzma_options_lzma, allow nice_len = 2 and 3 with the match
  152. finders that require at least 3 or 4. Now it is internally
  153. rounded up if needed.
  154. - CLMUL-based CRC64 on x86-64 and E2K with runtime processor
  155. detection. On 32-bit x86 it currently isn't available unless
  156. --disable-assembler is used which can make the non-CLMUL
  157. CRC64 slower; this might be fixed in the future.
  158. - Building with --disable-threads --enable-small
  159. is now thread-safe if the compiler supports
  160. __attribute__((__constructor__)).
  161. * xz:
  162. - Using -T0 (--threads=0) will now use multi-threaded encoder
  163. even on a single-core system. This is to ensure that output
  164. from the same xz binary is identical on both single-core and
  165. multi-core systems.
  166. - --threads=+1 or -T+1 is now a way to put xz into
  167. multi-threaded mode while using only one worker thread.
  168. The + is ignored if the number is not 1.
  169. - A default soft memory usage limit is now used for compression
  170. when -T0 is used and no explicit limit has been specified.
  171. This soft limit is used to restrict the number of threads
  172. but if the limit is exceeded with even one thread then xz
  173. will continue with one thread using the multi-threaded
  174. encoder and this limit is ignored. If the number of threads
  175. is specified manually then no default limit will be used;
  176. this affects only -T0.
  177. This change helps on systems that have very many cores and
  178. using all of them for xz makes no sense. Previously xz -T0
  179. could run out of memory on such systems because it attempted
  180. to reserve memory for too many threads.
  181. This also helps with 32-bit builds which don't have a large
  182. amount of address space that would be required for many
  183. threads. The default soft limit for -T0 is at most 1400 MiB
  184. on all 32-bit platforms.
  185. - Previously a low value in --memlimit-compress wouldn't cause
  186. xz to switch from multi-threaded mode to single-threaded mode
  187. if the limit cannot otherwise be met; xz failed instead. Now
  188. xz can switch to single-threaded mode and then, if needed,
  189. scale down the LZMA2 dictionary size too just like it already
  190. did when it was started in single-threaded mode.
  191. - The option --no-adjust no longer prevents xz from scaling down
  192. the number of threads as that doesn't affect the compressed
  193. output (only performance). Now --no-adjust only prevents
  194. adjustments that affect compressed output, that is, with
  195. --no-adjust xz won't switch from multi-threaded mode to
  196. single-threaded mode and won't scale down the LZMA2
  197. dictionary size.
  198. - Added a new option --memlimit-mt-decompress=LIMIT. This is
  199. used to limit the number of decompressor threads (possibly
  200. falling back to single-threaded mode) but it will never make
  201. xz refuse to decompress a file. This has a system-specific
  202. default value because without any limit xz could end up
  203. allocating memory for the whole compressed input file, the
  204. whole uncompressed output file, multiple thread-specific
  205. decompressor instances and so on. Basically xz could
  206. attempt to use an insane amount of memory even with fairly
  207. common files. The system-specific default value is currently
  208. the same as the one used for compression with -T0.
  209. The new option works together with the existing option
  210. --memlimit-decompress=LIMIT. The old option sets a hard limit
  211. that must not be exceeded (xz will refuse to decompress)
  212. while the new option only restricts the number of threads.
  213. If the limit set with --memlimit-mt-decompress is greater
  214. than the limit set with --memlimit-compress, then the latter
  215. value is used also for --memlimit-mt-decompress.
  216. - Added new information to the output of xz --info-memory and
  217. new fields to the output of xz --robot --info-memory.
  218. - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders
  219. now that liblzma handles it.
  220. - Don't mention endianness for ARM and ARM-Thumb filters in
  221. --long-help. The filters only work for little endian
  222. instruction encoding but modern ARM processors using
  223. big endian data access still use little endian
  224. instruction encoding. So the help text was misleading.
  225. In contrast, the PowerPC filter is only for big endian
  226. 32/64-bit PowerPC code. Little endian PowerPC would need
  227. a separate filter.
  228. - Added decompression support for the .lz (lzip) file format
  229. version 0 and the original unextended version 1. It is
  230. autodetected by default. See also the option --format on
  231. the xz man page.
  232. - Sandboxing enabled by default:
  233. * Capsicum (FreeBSD)
  234. * pledge(2) (OpenBSD)
  235. * Scripts now support the .lz format using xz.
  236. * A few new tests were added.
  237. * The liblzma-specific tests are now supported in CMake-based
  238. builds too ("make test").
  239. 5.3.5beta (2022-12-01)
  240. * All fixes from 5.2.9.
  241. * liblzma:
  242. - Added new LZMA_FILTER_LZMA1EXT for raw encoder and decoder to
  243. handle raw LZMA1 streams that don't have end of payload marker
  244. (EOPM) alias end of stream (EOS) marker. It can be used in
  245. filter chains, for example, with the x86 BCJ filter.
  246. - Added lzma_str_to_filters(), lzma_str_from_filters(), and
  247. lzma_str_list_filters() to make it easier for applications
  248. to get custom compression options from a user and convert
  249. it to an array of lzma_filter structures.
  250. - Added lzma_filters_free().
  251. - lzma_filters_update() can now be used with the multi-threaded
  252. encoder (lzma_stream_encoder_mt()) to change the filter chain
  253. after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH.
  254. - In lzma_options_lzma, allow nice_len = 2 and 3 with the match
  255. finders that require at least 3 or 4. Now it is internally
  256. rounded up if needed.
  257. - ARM64 filter was modified. It is still experimental.
  258. - Fixed LTO build with Clang if -fgnuc-version=10 or similar
  259. was used to make Clang look like GCC >= 10. Now it uses
  260. __has_attribute(__symver__) which should be reliable.
  261. * xz:
  262. - --threads=+1 or -T+1 is now a way to put xz into multi-threaded
  263. mode while using only one worker thread.
  264. - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders
  265. now that liblzma handles it.
  266. * Updated translations: Chinese (simplified), Korean, and Turkish.
  267. 5.3.4alpha (2022-11-15)
  268. * All fixes from 5.2.7 and 5.2.8.
  269. * liblzma:
  270. - Minor improvements to the threaded decoder.
  271. - Added CRC64 implementation that uses SSSE3, SSE4.1, and CLMUL
  272. instructions on 32/64-bit x86 and E2K. On 32-bit x86 it's
  273. not enabled unless --disable-assembler is used but then
  274. the non-CLMUL code might be slower. Processor support is
  275. detected at runtime so this is built by default on x86-64
  276. and E2K. On these platforms, if compiler flags indicate
  277. unconditional CLMUL support (-msse4.1 -mpclmul) then the
  278. generic version is not built, making liblzma 8-9 KiB smaller
  279. compared to having both versions included.
  280. With extremely compressible files this can make decompression
  281. up to twice as fast but with typical files 5 % improvement
  282. is a more realistic expectation.
  283. The CLMUL version is slower than the generic version with
  284. tiny inputs (especially at 1-8 bytes per call, but up to
  285. 16 bytes). In normal use in xz this doesn't matter at all.
  286. - Added an experimental ARM64 filter. This is *not* the final
  287. version! Files created with this experimental version won't
  288. be supported in the future versions! The filter design is
  289. a compromise where improving one use case makes some other
  290. cases worse.
  291. - Added decompression support for the .lz (lzip) file format
  292. version 0 and the original unextended version 1. See the
  293. API docs of lzma_lzip_decoder() for details. Also
  294. lzma_auto_decoder() supports .lz files.
  295. - Building with --disable-threads --enable-small
  296. is now thread-safe if the compiler supports
  297. __attribute__((__constructor__))
  298. * xz:
  299. - Added support for OpenBSD's pledge(2) as a sandboxing method.
  300. - Don't mention endianness for ARM and ARM-Thumb filters in
  301. --long-help. The filters only work for little endian
  302. instruction encoding but modern ARM processors using
  303. big endian data access still use little endian
  304. instruction encoding. So the help text was misleading.
  305. In contrast, the PowerPC filter is only for big endian
  306. 32/64-bit PowerPC code. Little endian PowerPC would need
  307. a separate filter.
  308. - Added --experimental-arm64. This will be renamed once the
  309. filter is finished. Files created with this experimental
  310. filter will not be supported in the future!
  311. - Added new fields to the output of xz --robot --info-memory.
  312. - Added decompression support for the .lz (lzip) file format
  313. version 0 and the original unextended version 1. It is
  314. autodetected by default. See also the option --format on
  315. the xz man page.
  316. * Scripts now support the .lz format using xz.
  317. * Build systems:
  318. - New #defines in config.h: HAVE_ENCODER_ARM64,
  319. HAVE_DECODER_ARM64, HAVE_LZIP_DECODER, HAVE_CPUID_H,
  320. HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR, HAVE_USABLE_CLMUL
  321. - New configure options: --disable-clmul-crc,
  322. --disable-microlzma, --disable-lzip-decoder, and
  323. 'pledge' is now an option in --enable-sandbox (but
  324. it's autodetected by default anyway).
  325. - INSTALL was updated to document the new configure options.
  326. - PACKAGERS now lists also --disable-microlzma and
  327. --disable-lzip-decoder as configure options that must
  328. not be used in builds for non-embedded use.
  329. * Tests:
  330. - Fix some of the tests so that they skip instead of fail if
  331. certain features have been disabled with configure options.
  332. It's still not perfect.
  333. - Other improvements to tests.
  334. * Updated translations: Croatian, Finnish, Hungarian, Polish,
  335. Romanian, Spanish, Swedish, and Ukrainian.
  336. 5.3.3alpha (2022-08-22)
  337. * All fixes from 5.2.6.
  338. * liblzma:
  339. - Fixed 32-bit build.
  340. - Added threaded .xz decompressor lzma_stream_decoder_mt().
  341. It can use multiple threads with .xz files that have multiple
  342. Blocks with size information in Block Headers. The threaded
  343. encoder in xz has always created such files.
  344. Single-threaded encoder cannot store the size information in
  345. Block Headers even if one used LZMA_FULL_FLUSH to create
  346. multiple Blocks, so this threaded decoder cannot use multiple
  347. threads with such files.
  348. If there are multiple Streams (concatenated .xz files), one
  349. Stream will be decompressed completely before starting the
  350. next Stream.
  351. - A new decoder flag LZMA_FAIL_FAST was added. It makes the
  352. threaded decompressor report errors soon instead of first
  353. flushing all pending data before the error location.
  354. * xz:
  355. - Using -T0 (--threads=0) will now use multi-threaded encoder
  356. even on a single-core system. This is to ensure that output
  357. from the same xz binary is identical on both single-core and
  358. multi-core systems.
  359. - A default soft memory usage limit is now used for compression
  360. when -T0 is used and no explicit limit has been specified.
  361. This soft limit is used to restrict the number of threads
  362. but if the limit is exceeded with even one thread then xz
  363. will continue with one thread using the multi-threaded
  364. encoder and this limit is ignored. If the number of threads
  365. is specified manually then no default limit will be used;
  366. this affects only -T0.
  367. This change helps on systems that have very many cores and
  368. using all of them for xz makes no sense. Previously xz -T0
  369. could run out of memory on such systems because it attempted
  370. to reserve memory for too many threads.
  371. This also helps with 32-bit builds which don't have a large
  372. amount of address space that would be required for many
  373. threads. The default limit is 1400 MiB on all 32-bit
  374. platforms with -T0.
  375. Now xz -T0 should just work. It might use too few threads
  376. in some cases but at least it shouldn't easily run out of
  377. memory. It's possible that this will be tweaked before 5.4.0.
  378. - Changes to --memlimit-compress and --no-adjust:
  379. In single-threaded mode, --memlimit-compress can make xz
  380. scale down the LZMA2 dictionary size to meet the memory usage
  381. limit. This obviously affects the compressed output. However,
  382. if xz was in threaded mode, --memlimit-compress could make xz
  383. reduce the number of threads but it wouldn't make xz switch
  384. from multi-threaded mode to single-threaded mode or scale
  385. down the LZMA2 dictionary size. This seemed illogical.
  386. Now --memlimit-compress can make xz switch to single-threaded
  387. mode if one thread in multi-threaded mode uses too much
  388. memory. If memory usage is still too high, then the LZMA2
  389. dictionary size can be scaled down too.
  390. The option --no-adjust was also changed so that it no longer
  391. prevents xz from scaling down the number of threads as that
  392. doesn't affect compressed output (only performance). After
  393. this commit --no-adjust only prevents adjustments that affect
  394. compressed output, that is, with --no-adjust xz won't switch
  395. from multithreaded mode to single-threaded mode and won't
  396. scale down the LZMA2 dictionary size.
  397. - Added a new option --memlimit-mt-decompress=LIMIT. This is
  398. used to limit the number of decompressor threads (possibly
  399. falling back to single-threaded mode) but it will never make
  400. xz refuse to decompress a file. This has a system-specific
  401. default value because without any limit xz could end up
  402. allocating memory for the whole compressed input file, the
  403. whole uncompressed output file, multiple thread-specific
  404. decompressor instances and so on. Basically xz could
  405. attempt to use an insane amount of memory even with fairly
  406. common files.
  407. The new option works together with the existing option
  408. --memlimit-decompress=LIMIT. The old option sets a hard limit
  409. that must not be exceeded (xz will refuse to decompress)
  410. while the new option only restricts the number of threads.
  411. If the limit set with --memlimit-mt-decompress is greater
  412. than the limit set with --memlimit-compress, then the latter
  413. value is used also for --memlimit-mt-decompress.
  414. * Tests:
  415. - Added a few more tests.
  416. - Added tests/code_coverage.sh to create a code coverage report
  417. of the tests.
  418. * Build systems:
  419. - Automake's parallel test harness is now used to make tests
  420. finish faster.
  421. - Added the CMake files to the distribution tarball. These were
  422. supposed to be in 5.2.5 already.
  423. - Added liblzma tests to the CMake build.
  424. - Windows: Fix building of liblzma.dll with the included
  425. Visual Studio project files.
  426. 5.3.2alpha (2021-10-28)
  427. This release was made on short notice so that recent erofs-utils can
  428. be built with LZMA support without needing a snapshot from xz.git.
  429. Thus many pending things were not included, not even updated
  430. translations (which would need to be updated for the new --list
  431. strings anyway).
  432. * All fixes from 5.2.5.
  433. * xz:
  434. - When copying metadata from the source file to the destination
  435. file, don't try to set the group (GID) if it is already set
  436. correctly. This avoids a failure on OpenBSD (and possibly on
  437. a few other OSes) where files may get created so that their
  438. group doesn't belong to the user, and fchown(2) can fail even
  439. if it needs to do nothing.
  440. - The --keep option now accepts symlinks, hardlinks, and
  441. setuid, setgid, and sticky files. Previously this required
  442. using --force.
  443. - Split the long strings used in --list and --info-memory modes
  444. to make them much easier for translators.
  445. - If built with sandbox support and enabling the sandbox fails,
  446. xz will now immediately exit with exit status of 1. Previously
  447. it would only display a warning if -vv was used.
  448. - Cap --memlimit-compress to 2000 MiB on MIPS32 because on
  449. MIPS32 userspace processes are limited to 2 GiB of address
  450. space.
  451. * liblzma:
  452. - Added lzma_microlzma_encoder() and lzma_microlzma_decoder().
  453. The API is in lzma/container.h.
  454. The MicroLZMA format is a raw LZMA stream (without end marker)
  455. whose first byte (always 0x00) has been replaced with
  456. bitwise-negation of the LZMA properties (lc/lp/pb). It was
  457. created for use in EROFS but may be used in other contexts
  458. as well where it is important to avoid wasting bytes for
  459. stream headers or footers. The format is also supported by
  460. XZ Embedded.
  461. The MicroLZMA encoder API in liblzma can compress into a
  462. fixed-sized output buffer so that as much data is compressed
  463. as can be fit into the buffer while still creating a valid
  464. MicroLZMA stream. This is needed for EROFS.
  465. - Added fuzzing support.
  466. - Support Intel Control-flow Enforcement Technology (CET) in
  467. 32-bit x86 assembly files.
  468. - Visual Studio: Use non-standard _MSVC_LANG to detect C++
  469. standard version in the lzma.h API header. It's used to
  470. detect when "noexcept" can be used.
  471. * Scripts:
  472. - Fix exit status of xzdiff/xzcmp. Exit status could be 2 when
  473. the correct value is 1.
  474. - Fix exit status of xzgrep.
  475. - Detect corrupt .bz2 files in xzgrep.
  476. - Add zstd support to xzgrep and xzdiff/xzcmp.
  477. - Fix less(1) version detection in xzless. It failed if the
  478. version number from "less -V" contained a dot.
  479. * Fix typos and technical issues in man pages.
  480. * Build systems:
  481. - Windows: Fix building of resource files when config.h isn't
  482. used. CMake + Visual Studio can now build liblzma.dll.
  483. - Various fixes to the CMake support. It might still need a few
  484. more fixes even for liblzma-only builds.
  485. 5.3.1alpha (2018-04-29)
  486. * All fixes from 5.2.4.
  487. * Add lzma_file_info_decoder() into liblzma and use it in xz to
  488. implement the --list feature.
  489. * Capsicum sandbox support is enabled by default where available
  490. (FreeBSD >= 10).
  491. 5.2.10 (2022-12-13)
  492. * xz: Don't modify argv[] when parsing the --memlimit* and
  493. --block-list command line options. This fixes confusing
  494. arguments in process listing (like "ps auxf").
  495. * GNU/Linux only: Use __has_attribute(__symver__) to detect if
  496. that attribute is supported. This fixes build on Mandriva where
  497. Clang is patched to define __GNUC__ to 11 by default (instead
  498. of 4 as used by Clang upstream).
  499. 5.2.9 (2022-11-30)
  500. * liblzma:
  501. - Fixed an infinite loop in LZMA encoder initialization
  502. if dict_size >= 2 GiB. (The encoder only supports up
  503. to 1536 MiB.)
  504. - Fixed two cases of invalid free() that can happen if
  505. a tiny allocation fails in encoder re-initialization
  506. or in lzma_filters_update(). These bugs had some
  507. similarities with the bug fixed in 5.2.7.
  508. - Fixed lzma_block_encoder() not allowing the use of
  509. LZMA_SYNC_FLUSH with lzma_code() even though it was
  510. documented to be supported. The sync-flush code in
  511. the Block encoder was already used internally via
  512. lzma_stream_encoder(), so this was just a missing flag
  513. in the lzma_block_encoder() API function.
  514. - GNU/Linux only: Don't put symbol versions into static
  515. liblzma as it breaks things in some cases (and even if
  516. it didn't break anything, symbol versions in static
  517. libraries are useless anyway). The downside of the fix
  518. is that if the configure options --with-pic or --without-pic
  519. are used then it's not possible to build both shared and
  520. static liblzma at the same time on GNU/Linux anymore;
  521. with those options --disable-static or --disable-shared
  522. must be used too.
  523. * New email address for bug reports is <xz@tukaani.org> which
  524. forwards messages to Lasse Collin and Jia Tan.
  525. 5.2.8 (2022-11-13)
  526. * xz:
  527. - If xz cannot remove an input file when it should, this
  528. is now treated as a warning (exit status 2) instead of
  529. an error (exit status 1). This matches GNU gzip and it
  530. is more logical as at that point the output file has
  531. already been successfully closed.
  532. - Fix handling of .xz files with an unsupported check type.
  533. Previously such printed a warning message but then xz
  534. behaved as if an error had occurred (didn't decompress,
  535. exit status 1). Now a warning is printed, decompression
  536. is done anyway, and exit status is 2. This used to work
  537. slightly before 5.0.0. In practice this bug matters only
  538. if xz has been built with some check types disabled. As
  539. instructed in PACKAGERS, such builds should be done in
  540. special situations only.
  541. - Fix "xz -dc --single-stream tests/files/good-0-empty.xz"
  542. which failed with "Internal error (bug)". That is,
  543. --single-stream was broken if the first .xz stream in
  544. the input file didn't contain any uncompressed data.
  545. - Fix displaying file sizes in the progress indicator when
  546. working in passthru mode and there are multiple input files.
  547. Just like "gzip -cdf", "xz -cdf" works like "cat" when the
  548. input file isn't a supported compressed file format. In
  549. this case the file size counters weren't reset between
  550. files so with multiple input files the progress indicator
  551. displayed an incorrect (too large) value.
  552. * liblzma:
  553. - API docs in lzma/container.h:
  554. * Update the list of decoder flags in the decoder
  555. function docs.
  556. * Explain LZMA_CONCATENATED behavior with .lzma files
  557. in lzma_auto_decoder() docs.
  558. - OpenBSD: Use HW_NCPUONLINE to detect the number of
  559. available hardware threads in lzma_physmem().
  560. - Fix use of wrong macro to detect x86 SSE2 support.
  561. __SSE2_MATH__ was used with GCC/Clang but the correct
  562. one is __SSE2__. The first one means that SSE2 is used
  563. for floating point math which is irrelevant here.
  564. The affected SSE2 code isn't used on x86-64 so this affects
  565. only 32-bit x86 builds that use -msse2 without -mfpmath=sse
  566. (there is no runtime detection for SSE2). It improves LZMA
  567. compression speed (not decompression).
  568. - Fix the build with Intel C compiler 2021 (ICC, not ICX)
  569. on Linux. It defines __GNUC__ to 10 but doesn't support
  570. the __symver__ attribute introduced in GCC 10.
  571. * Scripts: Ignore warnings from xz by using --quiet --no-warn.
  572. This is needed if the input .xz files use an unsupported
  573. check type.
  574. * Translations:
  575. - Updated Croatian and Turkish translations.
  576. - One new translations wasn't included because it needed
  577. technical fixes. It will be in upcoming 5.4.0. No new
  578. translations will be added to the 5.2.x branch anymore.
  579. - Renamed the French man page translation file from
  580. fr_FR.po to fr.po and thus also its install directory
  581. (like /usr/share/man/fr_FR -> .../fr).
  582. - Man page translations for upcoming 5.4.0 are now handled
  583. in the Translation Project.
  584. * Update doc/faq.txt a little so it's less out-of-date.
  585. 5.2.7 (2022-09-30)
  586. * liblzma:
  587. - Made lzma_filters_copy() to never modify the destination
  588. array if an error occurs. lzma_stream_encoder() and
  589. lzma_stream_encoder_mt() already assumed this. Before this
  590. change, if a tiny memory allocation in lzma_filters_copy()
  591. failed it would lead to a crash (invalid free() or invalid
  592. memory reads) in the cleanup paths of these two encoder
  593. initialization functions.
  594. - Added missing integer overflow check to lzma_index_append().
  595. This affects xz --list and other applications that decode
  596. the Index field from .xz files using lzma_index_decoder().
  597. Normal decompression of .xz files doesn't call this code
  598. and thus most applications using liblzma aren't affected
  599. by this bug.
  600. - Single-threaded .xz decoder (lzma_stream_decoder()): If
  601. lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible
  602. to use lzma_memlimit_set() to increase the limit and continue
  603. decoding. This was supposed to work from the beginning
  604. but there was a bug. With other decoders (.lzma or
  605. threaded .xz decoder) this already worked correctly.
  606. - Fixed accumulation of integrity check type statistics in
  607. lzma_index_cat(). This bug made lzma_index_checks() return
  608. only the type of the integrity check of the last Stream
  609. when multiple lzma_indexes were concatenated. Most
  610. applications don't use these APIs but in xz it made
  611. xz --list not list all check types from concatenated .xz
  612. files. In xz --list --verbose only the per-file "Check:"
  613. lines were affected and in xz --robot --list only the "file"
  614. line was affected.
  615. - Added ABI compatibility with executables that were linked
  616. against liblzma in RHEL/CentOS 7 or other liblzma builds
  617. that had copied the problematic patch from RHEL/CentOS 7
  618. (xz-5.2.2-compat-libs.patch). For the details, see the
  619. comment at the top of src/liblzma/validate_map.sh.
  620. WARNING: This uses __symver__ attribute with GCC >= 10.
  621. In other cases the traditional __asm__(".symver ...")
  622. is used. Using link-time optimization (LTO, -flto) with
  623. GCC versions older than 10 can silently result in
  624. broken liblzma.so.5 (incorrect symbol versions)! If you
  625. want to use -flto with GCC, you must use GCC >= 10.
  626. LTO with Clang seems to work even with the traditional
  627. __asm__(".symver ...") method.
  628. * xzgrep: Fixed compatibility with old shells that break if
  629. comments inside command substitutions have apostrophes (').
  630. This problem was introduced in 5.2.6.
  631. * Build systems:
  632. - New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX
  633. - Windows: Fixed liblzma.dll build with Visual Studio project
  634. files. It broke in 5.2.6 due to a change that was made to
  635. improve CMake support.
  636. - Windows: Building liblzma with UNICODE defined should now
  637. work.
  638. - CMake files are now actually included in the release tarball.
  639. They should have been in 5.2.5 already.
  640. - Minor CMake fixes and improvements.
  641. * Added a new translation: Turkish
  642. 5.2.6 (2022-08-12)
  643. * xz:
  644. - The --keep option now accepts symlinks, hardlinks, and
  645. setuid, setgid, and sticky files. Previously this required
  646. using --force.
  647. - When copying metadata from the source file to the destination
  648. file, don't try to set the group (GID) if it is already set
  649. correctly. This avoids a failure on OpenBSD (and possibly on
  650. a few other OSes) where files may get created so that their
  651. group doesn't belong to the user, and fchown(2) can fail even
  652. if it needs to do nothing.
  653. - Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on
  654. MIPS32 because on MIPS32 userspace processes are limited
  655. to 2 GiB of address space.
  656. * liblzma:
  657. - Fixed a missing error-check in the threaded encoder. If a
  658. small memory allocation fails, a .xz file with an invalid
  659. Index field would be created. Decompressing such a file would
  660. produce the correct output but result in an error at the end.
  661. Thus this is a "mild" data corruption bug. Note that while
  662. a failed memory allocation can trigger the bug, it cannot
  663. cause invalid memory access.
  664. - The decoder for .lzma files now supports files that have
  665. uncompressed size stored in the header and still use the
  666. end of payload marker (end of stream marker) at the end
  667. of the LZMA stream. Such files are rare but, according to
  668. the documentation in LZMA SDK, they are valid.
  669. doc/lzma-file-format.txt was updated too.
  670. - Improved 32-bit x86 assembly files:
  671. * Support Intel Control-flow Enforcement Technology (CET)
  672. * Use non-executable stack on FreeBSD.
  673. - Visual Studio: Use non-standard _MSVC_LANG to detect C++
  674. standard version in the lzma.h API header. It's used to
  675. detect when "noexcept" can be used.
  676. * xzgrep:
  677. - Fixed arbitrary command injection via a malicious filename
  678. (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for
  679. this was released to the public on 2022-04-07. A slight
  680. robustness improvement has been made since then and, if
  681. using GNU or *BSD grep, a new faster method is now used
  682. that doesn't use the old sed-based construct at all. This
  683. also fixes bad output with GNU grep >= 3.5 (2020-09-27)
  684. when xzgrepping binary files.
  685. This vulnerability was discovered by:
  686. cleemy desu wayo working with Trend Micro Zero Day Initiative
  687. - Fixed detection of corrupt .bz2 files.
  688. - Improved error handling to fix exit status in some situations
  689. and to fix handling of signals: in some situations a signal
  690. didn't make xzgrep exit when it clearly should have. It's
  691. possible that the signal handling still isn't quite perfect
  692. but hopefully it's good enough.
  693. - Documented exit statuses on the man page.
  694. - xzegrep and xzfgrep now use "grep -E" and "grep -F" instead
  695. of the deprecated egrep and fgrep commands.
  696. - Fixed parsing of the options -E, -F, -G, -P, and -X. The
  697. problem occurred when multiple options were specied in
  698. a single argument, for example,
  699. echo foo | xzgrep -Fe foo
  700. treated foo as a filename because -Fe wasn't correctly
  701. split into -F -e.
  702. - Added zstd support.
  703. * xzdiff/xzcmp:
  704. - Fixed wrong exit status. Exit status could be 2 when the
  705. correct value is 1.
  706. - Documented on the man page that exit status of 2 is used
  707. for decompression errors.
  708. - Added zstd support.
  709. * xzless:
  710. - Fix less(1) version detection. It failed if the version number
  711. from "less -V" contained a dot.
  712. * Translations:
  713. - Added new translations: Catalan, Croatian, Esperanto,
  714. Korean, Portuguese, Romanian, Serbian, Spanish, Swedish,
  715. and Ukrainian
  716. - Updated the Brazilian Portuguese translation.
  717. - Added French man page translation. This and the existing
  718. German translation aren't complete anymore because the
  719. English man pages got a few updates and the translators
  720. weren't reached so that they could update their work.
  721. * Build systems:
  722. - Windows: Fix building of resource files when config.h isn't
  723. used. CMake + Visual Studio can now build liblzma.dll.
  724. - Various fixes to the CMake support. Building static or shared
  725. liblzma should work fine in most cases. In contrast, building
  726. the command line tools with CMake is still clearly incomplete
  727. and experimental and should be used for testing only.
  728. 5.2.5 (2020-03-17)
  729. * liblzma:
  730. - Fixed several C99/C11 conformance bugs. Now the code is clean
  731. under gcc/clang -fsanitize=undefined. Some of these changes
  732. might have a negative effect on performance with old GCC
  733. versions or compilers other than GCC and Clang. The configure
  734. option --enable-unsafe-type-punning can be used to (mostly)
  735. restore the old behavior but it shouldn't normally be used.
  736. - Improved API documentation of lzma_properties_decode().
  737. - Added a very minor encoder speed optimization.
  738. * xz:
  739. - Fixed a crash in "xz -dcfv not_an_xz_file". All four options
  740. were required to trigger it. The crash occurred in the
  741. progress indicator code when xz was in passthru mode where
  742. xz works like "cat".
  743. - Fixed an integer overflow with 32-bit off_t. It could happen
  744. when decompressing a file that has a long run of zero bytes
  745. which xz would try to write as a sparse file. Since the build
  746. system enables large file support by default, off_t is
  747. normally 64-bit even on 32-bit systems.
  748. - Fixes for --flush-timeout:
  749. * Fix semi-busy-waiting.
  750. * Avoid unneeded flushes when no new input has arrived
  751. since the previous flush was completed.
  752. - Added a special case for 32-bit xz: If --memlimit-compress is
  753. used to specify a limit that exceeds 4020 MiB, the limit will
  754. be set to 4020 MiB. The values "0" and "max" aren't affected
  755. by this and neither is decompression. This hack can be
  756. helpful when a 32-bit xz has access to 4 GiB address space
  757. but the specified memlimit exceeds 4 GiB. This can happen
  758. e.g. with some scripts.
  759. - Capsicum sandbox is now enabled by default where available
  760. (FreeBSD >= 10). The sandbox debug messages (xz -vv) were
  761. removed since they seemed to be more annoying than useful.
  762. - DOS build now requires DJGPP 2.05 instead of 2.04beta.
  763. A workaround for a locale problem with DJGPP 2.05 was added.
  764. * xzgrep and other scripts:
  765. - Added a configure option --enable-path-for-scripts=PREFIX.
  766. It is disabled by default except on Solaris where the default
  767. is /usr/xpg4/bin. See INSTALL for details.
  768. - Added a workaround for a POSIX shell detection problem on
  769. Solaris.
  770. * Build systems:
  771. - Added preliminary build instructions for z/OS. See INSTALL
  772. section 1.2.9.
  773. - Experimental CMake support was added. It should work to build
  774. static liblzma on a few operating systems. It may or may not
  775. work to build shared liblzma. On some platforms it can build
  776. xz and xzdec too but those are only for testing. See the
  777. comment in the beginning of CMakeLists.txt for details.
  778. - Visual Studio project files were updated.
  779. WindowsTargetPlatformVersion was removed from VS2017 files
  780. and set to "10.0" in the added VS2019 files. In the future
  781. the VS project files will be removed when CMake support is
  782. good enough.
  783. - New #defines in config.h: HAVE___BUILTIN_ASSUME_ALIGNED,
  784. HAVE___BUILTIN_BSWAPXX, and TUKLIB_USE_UNSAFE_TYPE_PUNNING.
  785. - autogen.sh has a new optional dependency on po4a and a new
  786. option --no-po4a to skip that step. This matters only if one
  787. wants to remake the build files. po4a is used to update the
  788. translated man pages but as long as the man pages haven't
  789. been modified, there's nothing to update and one can use
  790. --no-po4a to avoid the dependency on po4a.
  791. * Translations:
  792. - XZ Utils translations are now handled by the Translation
  793. Project: https://translationproject.org/domain/xz.html
  794. - All man pages are now included in German too.
  795. - New xz translations: Brazilian Portuguese, Finnish,
  796. Hungarian, Chinese (simplified), Chinese (traditional),
  797. and Danish (partial translation)
  798. - Updated xz translations: French, German, Italian, and Polish
  799. - Unfortunately a few new xz translations weren't included due
  800. to technical problems like too long lines in --help output or
  801. misaligned column headings in tables. In the future, many of
  802. these strings will be split and e.g. the table column
  803. alignment will be handled in software. This should make the
  804. strings easier to translate.
  805. 5.2.4 (2018-04-29)
  806. * liblzma:
  807. - Allow 0 as memory usage limit instead of returning
  808. LZMA_PROG_ERROR. Now 0 is treated as if 1 byte was specified,
  809. which effectively is the same as 0.
  810. - Use "noexcept" keyword instead of "throw()" in the public
  811. headers when a C++11 (or newer standard) compiler is used.
  812. - Added a portability fix for recent Intel C Compilers.
  813. - Microsoft Visual Studio build files have been moved under
  814. windows/vs2013 and windows/vs2017.
  815. * xz:
  816. - Fix "xz --list --robot missing_or_bad_file.xz" which would
  817. try to print an uninitialized string and thus produce garbage
  818. output. Since the exit status is non-zero, most uses of such
  819. a command won't try to interpret the garbage output.
  820. - "xz --list foo.xz" could print "Internal error (bug)" in a
  821. corner case where a specific memory usage limit had been set.
  822. 5.2.3 (2016-12-30)
  823. * xz:
  824. - Always close a file before trying to delete it to avoid
  825. problems on some operating system and file system combinations.
  826. - Fixed copying of file timestamps on Windows.
  827. - Added experimental (disabled by default) sandbox support using
  828. Capsicum (FreeBSD >= 10). See --enable-sandbox in INSTALL.
  829. * C99/C11 conformance fixes to liblzma. The issues affected at least
  830. some builds using link-time optimizations.
  831. * Fixed bugs in the rarely-used function lzma_index_dup().
  832. * Use of external SHA-256 code is now disabled by default.
  833. It can still be enabled by passing --enable-external-sha256
  834. to configure. The reasons to disable it by default (see INSTALL
  835. for more details):
  836. - Some OS-specific SHA-256 implementations conflict with
  837. OpenSSL and cause problems in programs that link against both
  838. liblzma and libcrypto. At least FreeBSD 10 and MINIX 3.3.0
  839. are affected.
  840. - The internal SHA-256 is faster than the SHA-256 code in
  841. some operating systems.
  842. * Changed CPU core count detection to use sched_getaffinity() on
  843. GNU/Linux and GNU/kFreeBSD.
  844. * Fixes to the build-system and xz to make xz buildable even when
  845. encoders, decoders, or threading have been disabled from libilzma
  846. using configure options. These fixes added two new #defines to
  847. config.h: HAVE_ENCODERS and HAVE_DECODERS.
  848. 5.2.2 (2015-09-29)
  849. * Fixed bugs in QNX-specific code.
  850. * Omitted the use of pipe2() even if it is available to avoid
  851. portability issues with some old Linux and glibc combinations.
  852. * Updated German translation.
  853. * Added project files to build static and shared liblzma (not the
  854. whole XZ Utils) with Visual Studio 2013 update 2 or later.
  855. * Documented that threaded decompression hasn't been implemented
  856. yet. A 5.2.0 NEWS entry describing multi-threading support had
  857. incorrectly said "decompression" when it should have said
  858. "compression".
  859. 5.2.1 (2015-02-26)
  860. * Fixed a compression-ratio regression in fast mode of LZMA1 and
  861. LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases.
  862. * Fixed a portability problem in xz that affected at least OpenBSD.
  863. * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs
  864. from most other mktemp implementations.
  865. * Changed CPU core count detection to use cpuset_getaffinity() on
  866. FreeBSD.
  867. 5.2.0 (2014-12-21)
  868. Since 5.1.4beta:
  869. * All fixes from 5.0.8
  870. * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset
  871. was used.
  872. * xzdiff: If mktemp isn't installed, mkdir will be used as
  873. a fallback to create a temporary directory. Installing mktemp
  874. is still recommended.
  875. * Updated French, German, Italian, Polish, and Vietnamese
  876. translations.
  877. Summary of fixes and new features added in the 5.1.x development
  878. releases:
  879. * liblzma:
  880. - Added support for multi-threaded compression. See the
  881. lzma_mt structure, lzma_stream_encoder_mt(), and
  882. lzma_stream_encoder_mt_memusage() in <lzma/container.h>,
  883. lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads()
  884. in <lzma/hardware.h> for details.
  885. - Made the uses of lzma_allocator const correct.
  886. - Added lzma_block_uncomp_encode() to create uncompressed
  887. .xz Blocks using LZMA2 uncompressed chunks.
  888. - Added support for LZMA_IGNORE_CHECK.
  889. - A few speed optimizations were made.
  890. - Added support for symbol versioning. It is enabled by default
  891. on GNU/Linux, other GNU-based systems, and FreeBSD.
  892. - liblzma (not the whole XZ Utils) should now be buildable
  893. with MSVC 2013 update 2 or later using windows/config.h.
  894. * xz:
  895. - Fixed a race condition in the signal handling. It was
  896. possible that e.g. the first SIGINT didn't make xz exit
  897. if reading or writing blocked and one had bad luck. The fix
  898. is non-trivial, so as of writing it is unknown if it will be
  899. backported to the v5.0 branch.
  900. - Multi-threaded compression can be enabled with the
  901. --threads (-T) option.
  902. [Fixed: This originally said "decompression".]
  903. - New command line options in xz: --single-stream,
  904. --block-size=SIZE, --block-list=SIZES,
  905. --flush-timeout=TIMEOUT, and --ignore-check.
  906. - xz -lvv now shows the minimum xz version that is required to
  907. decompress the file. Currently it is 5.0.0 for all supported
  908. .xz files except files with empty LZMA2 streams require 5.0.2.
  909. * xzdiff and xzgrep now support .lzo files if lzop is installed.
  910. The .tzo suffix is also recognized as a shorthand for .tar.lzo.
  911. 5.1.4beta (2014-09-14)
  912. * All fixes from 5.0.6
  913. * liblzma: Fixed the use of presets in threaded encoder
  914. initialization.
  915. * xz --block-list and --block-size can now be used together
  916. in single-threaded mode. Previously the combination only
  917. worked in multi-threaded mode.
  918. * Added support for LZMA_IGNORE_CHECK to liblzma and made it
  919. available in xz as --ignore-check.
  920. * liblzma speed optimizations:
  921. - Initialization of a new LZMA1 or LZMA2 encoder has been
  922. optimized. (The speed of reinitializing an already-allocated
  923. encoder isn't affected.) This helps when compressing many
  924. small buffers with lzma_stream_buffer_encode() and other
  925. similar situations where an already-allocated encoder state
  926. isn't reused. This speed-up is visible in xz too if one
  927. compresses many small files one at a time instead running xz
  928. once and giving all files as command-line arguments.
  929. - Buffer comparisons are now much faster when unaligned access
  930. is allowed (configured with --enable-unaligned-access). This
  931. speeds up encoding significantly. There is arch-specific code
  932. for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best
  933. results and there's no run-time CPU detection for now).
  934. For other archs there is only generic code which probably
  935. isn't as optimal as arch-specific solutions could be.
  936. - A few speed optimizations were made to the SHA-256 code.
  937. (Note that the builtin SHA-256 code isn't used on all
  938. operating systems.)
  939. * liblzma can now be built with MSVC 2013 update 2 or later
  940. using windows/config.h.
  941. * Vietnamese translation was added.
  942. 5.1.3alpha (2013-10-26)
  943. * All fixes from 5.0.5
  944. * liblzma:
  945. - Fixed a deadlock in the threaded encoder.
  946. - Made the uses of lzma_allocator const correct.
  947. - Added lzma_block_uncomp_encode() to create uncompressed
  948. .xz Blocks using LZMA2 uncompressed chunks.
  949. - Added support for native threads on Windows and the ability
  950. to detect the number of CPU cores.
  951. * xz:
  952. - Fixed a race condition in the signal handling. It was
  953. possible that e.g. the first SIGINT didn't make xz exit
  954. if reading or writing blocked and one had bad luck. The fix
  955. is non-trivial, so as of writing it is unknown if it will be
  956. backported to the v5.0 branch.
  957. - Made the progress indicator work correctly in threaded mode.
  958. - Threaded encoder now works together with --block-list=SIZES.
  959. - Added preliminary support for --flush-timeout=TIMEOUT.
  960. It can be useful for (somewhat) real-time streaming. For
  961. now the decompression side has to be done with something
  962. else than the xz tool due to how xz does buffering, but this
  963. should be fixed.
  964. 5.1.2alpha (2012-07-04)
  965. * All fixes from 5.0.3 and 5.0.4
  966. * liblzma:
  967. - Fixed a deadlock and an invalid free() in the threaded encoder.
  968. - Added support for symbol versioning. It is enabled by default
  969. on GNU/Linux, other GNU-based systems, and FreeBSD.
  970. - Use SHA-256 implementation from the operating system if one is
  971. available in libc, libmd, or libutil. liblzma won't use e.g.
  972. OpenSSL or libgcrypt to avoid introducing new dependencies.
  973. - Fixed liblzma.pc for static linking.
  974. - Fixed a few portability bugs.
  975. * xz --decompress --single-stream now fixes the input position after
  976. successful decompression. Now the following works:
  977. echo foo | xz > foo.xz
  978. echo bar | xz >> foo.xz
  979. ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz
  980. Note that it doesn't work if the input is not seekable
  981. or if there is Stream Padding between the concatenated
  982. .xz Streams.
  983. * xz -lvv now shows the minimum xz version that is required to
  984. decompress the file. Currently it is 5.0.0 for all supported .xz
  985. files except files with empty LZMA2 streams require 5.0.2.
  986. * Added an *incomplete* implementation of --block-list=SIZES to xz.
  987. It only works correctly in single-threaded mode and when
  988. --block-size isn't used at the same time. --block-list allows
  989. specifying the sizes of Blocks which can be useful e.g. when
  990. creating files for random-access reading.
  991. 5.1.1alpha (2011-04-12)
  992. * All fixes from 5.0.2
  993. * liblzma fixes that will also be included in 5.0.3:
  994. - A memory leak was fixed.
  995. - lzma_stream_buffer_encode() no longer creates an empty .xz
  996. Block if encoding an empty buffer. Such an empty Block with
  997. LZMA2 data would trigger a bug in 5.0.1 and older (see the
  998. first bullet point in 5.0.2 notes). When releasing 5.0.2,
  999. I thought that no encoder creates this kind of files but
  1000. I was wrong.
  1001. - Validate function arguments better in a few functions. Most
  1002. importantly, specifying an unsupported integrity check to
  1003. lzma_stream_buffer_encode() no longer creates a corrupt .xz
  1004. file. Probably no application tries to do that, so this
  1005. shouldn't be a big problem in practice.
  1006. - Document that lzma_block_buffer_encode(),
  1007. lzma_easy_buffer_encode(), lzma_stream_encoder(), and
  1008. lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
  1009. - The return values of the _memusage() functions are now
  1010. documented better.
  1011. * Support for multithreaded compression was added using the simplest
  1012. method, which splits the input data into blocks and compresses
  1013. them independently. Other methods will be added in the future.
  1014. The current method has room for improvement, e.g. it is possible
  1015. to reduce the memory usage.
  1016. * Added the options --single-stream and --block-size=SIZE to xz.
  1017. * xzdiff and xzgrep now support .lzo files if lzop is installed.
  1018. The .tzo suffix is also recognized as a shorthand for .tar.lzo.
  1019. * Support for short 8.3 filenames under DOS was added to xz. It is
  1020. experimental and may change before it gets into a stable release.
  1021. 5.0.8 (2014-12-21)
  1022. * Fixed an old bug in xzgrep that affected OpenBSD and probably
  1023. a few other operating systems too.
  1024. * Updated French and German translations.
  1025. * Added support for detecting the amount of RAM on AmigaOS/AROS.
  1026. * Minor build system updates.
  1027. 5.0.7 (2014-09-20)
  1028. * Fix regressions introduced in 5.0.6:
  1029. - Fix building with non-GNU make.
  1030. - Fix invalid Libs.private value in liblzma.pc which broke
  1031. static linking against liblzma if the linker flags were
  1032. taken from pkg-config.
  1033. 5.0.6 (2014-09-14)
  1034. * xzgrep now exits with status 0 if at least one file matched.
  1035. * A few minor portability and build system fixes
  1036. 5.0.5 (2013-06-30)
  1037. * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing
  1038. .lzma files that have less common settings in the headers
  1039. (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed
  1040. size greater than 256 GiB). The limitations existed to avoid false
  1041. positives when detecting .lzma files. The lc + lp <= 4 limitation
  1042. still remains since liblzma's LZMA decoder has that limitation.
  1043. NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT
  1044. affected by this change. They still consider uncommon .lzma headers
  1045. as not being in the .lzma format. Changing this would give way too
  1046. many false positives.
  1047. * xz:
  1048. - Interaction of preset and custom filter chain options was
  1049. made less illogical. This affects only certain less typical
  1050. uses cases so few people are expected to notice this change.
  1051. Now when a custom filter chain option (e.g. --lzma2) is
  1052. specified, all preset options (-0 ... -9, -e) earlier are on
  1053. the command line are completely forgotten. Similarly, when
  1054. a preset option is specified, all custom filter chain options
  1055. earlier on the command line are completely forgotten.
  1056. Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e"
  1057. which is equivalent to "xz -6e". Earlier -e didn't put xz back
  1058. into preset mode and thus the example command was equivalent
  1059. to "xz --lzma2=preset=5".
  1060. Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to
  1061. "xz -7". Earlier a custom filter chain option didn't make
  1062. xz forget the -e option so the example was equivalent to
  1063. "xz -7e".
  1064. - Fixes and improvements to error handling.
  1065. - Various fixes to the man page.
  1066. * xzless: Fixed to work with "less" versions 448 and later.
  1067. * xzgrep: Made -h an alias for --no-filename.
  1068. * Include the previously missing debug/translation.bash which can
  1069. be useful for translators.
  1070. * Include a build script for Mac OS X. This has been in the Git
  1071. repository since 2010 but due to a mistake in Makefile.am the
  1072. script hasn't been included in a release tarball before.
  1073. 5.0.4 (2012-06-22)
  1074. * liblzma:
  1075. - Fix lzma_index_init(). It could crash if memory allocation
  1076. failed.
  1077. - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ
  1078. filter is used and the application only provides exactly as
  1079. much output space as is the uncompressed size of the file.
  1080. - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't
  1081. check if the last call to lzma_code() really returned
  1082. LZMA_STREAM_END, which made the program think that truncated
  1083. files are valid.
  1084. - New example programs in doc/examples (old programs are now in
  1085. doc/examples_old). These have more comments and more detailed
  1086. error handling.
  1087. * Fix "xz -lvv foo.xz". It could crash on some corrupted files.
  1088. * Fix output of "xz --robot -lv" and "xz --robot -lvv" which
  1089. incorrectly printed the filename also in the "foo (x/x)" format.
  1090. * Fix exit status of "xzdiff foo.xz bar.xz".
  1091. * Fix exit status of "xzgrep foo binary_file".
  1092. * Fix portability to EBCDIC systems.
  1093. * Fix a configure issue on AIX with the XL C compiler. See INSTALL
  1094. for details.
  1095. * Update French, German, Italian, and Polish translations.
  1096. 5.0.3 (2011-05-21)
  1097. * liblzma fixes:
  1098. - A memory leak was fixed.
  1099. - lzma_stream_buffer_encode() no longer creates an empty .xz
  1100. Block if encoding an empty buffer. Such an empty Block with
  1101. LZMA2 data would trigger a bug in 5.0.1 and older (see the
  1102. first bullet point in 5.0.2 notes). When releasing 5.0.2,
  1103. I thought that no encoder creates this kind of files but
  1104. I was wrong.
  1105. - Validate function arguments better in a few functions. Most
  1106. importantly, specifying an unsupported integrity check to
  1107. lzma_stream_buffer_encode() no longer creates a corrupt .xz
  1108. file. Probably no application tries to do that, so this
  1109. shouldn't be a big problem in practice.
  1110. - Document that lzma_block_buffer_encode(),
  1111. lzma_easy_buffer_encode(), lzma_stream_encoder(), and
  1112. lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
  1113. - The return values of the _memusage() functions are now
  1114. documented better.
  1115. * Fix command name detection in xzgrep. xzegrep and xzfgrep now
  1116. correctly use egrep and fgrep instead of grep.
  1117. * French translation was added.
  1118. 5.0.2 (2011-04-01)
  1119. * LZMA2 decompressor now correctly accepts LZMA2 streams with no
  1120. uncompressed data. Previously it considered them corrupt. The
  1121. bug can affect applications that use raw LZMA2 streams. It is
  1122. very unlikely to affect .xz files because no compressor creates
  1123. .xz files with empty LZMA2 streams. (Empty .xz files are a
  1124. different thing than empty LZMA2 streams.)
  1125. * "xz --suffix=.foo filename.foo" now refuses to compress the
  1126. file due to it already having the suffix .foo. It was already
  1127. documented on the man page, but the code lacked the test.
  1128. * "xzgrep -l foo bar.xz" works now.
  1129. * Polish translation was added.
  1130. 5.0.1 (2011-01-29)
  1131. * xz --force now (de)compresses files that have setuid, setgid,
  1132. or sticky bit set and files that have multiple hard links.
  1133. The man page had it documented this way already, but the code
  1134. had a bug.
  1135. * gzip and bzip2 support in xzdiff was fixed.
  1136. * Portability fixes
  1137. * Minor fix to Czech translation
  1138. 5.0.0 (2010-10-23)
  1139. Only the most important changes compared to 4.999.9beta are listed
  1140. here. One change is especially important:
  1141. * The memory usage limit is now disabled by default. Some scripts
  1142. written before this change may have used --memory=max on xz command
  1143. line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED
  1144. NOW, because they interfere with user's ability to set the memory
  1145. usage limit himself. If user-specified limit causes problems to
  1146. your script, blame the user.
  1147. Other significant changes:
  1148. * Added support for XZ_DEFAULTS environment variable. This variable
  1149. allows users to set default options for xz, e.g. default memory
  1150. usage limit or default compression level. Scripts that use xz
  1151. must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT
  1152. instead if they need a way to pass options to xz via an
  1153. environment variable.
  1154. * The compression settings associated with the preset levels
  1155. -0 ... -9 have been changed. --extreme was changed a little too.
  1156. It is now less likely to make compression worse, but with some
  1157. files the new --extreme may compress slightly worse than the old
  1158. --extreme.
  1159. * If a preset level (-0 ... -9) is specified after a custom filter
  1160. chain options have been used (e.g. --lzma2), the custom filter
  1161. chain will be forgotten. Earlier the preset options were
  1162. completely ignored after custom filter chain options had been
  1163. seen.
  1164. * xz will create sparse files when decompressing if the uncompressed
  1165. data contains long sequences of binary zeros. This is done even
  1166. when writing to standard output that is connected to a regular
  1167. file and certain additional conditions are met to make it safe.
  1168. * Support for "xz --list" was added. Combine with --verbose or
  1169. --verbose --verbose (-vv) for detailed output.
  1170. * I had hoped that liblzma API would have been stable after
  1171. 4.999.9beta, but there have been a couple of changes in the
  1172. advanced features, which don't affect most applications:
  1173. - Index handling code was revised. If you were using the old
  1174. API, you will get a compiler error (so it's easy to notice).
  1175. - A subtle but important change was made to the Block handling
  1176. API. lzma_block.version has to be initialized even for
  1177. lzma_block_header_decode(). Code that doesn't do it will work
  1178. for now, but might break in the future, which makes this API
  1179. change easy to miss.
  1180. * The major soname has been bumped to 5.0.0. liblzma API and ABI
  1181. are now stable, so the need to recompile programs linking against
  1182. liblzma shouldn't arise soon.