Slax author's Blog (RSS)

XZ compression filters

XZ (LZMA) compression is the best available for our purposes, yet it can be even better, since LZMA supports BCJ filters. What's that? Simply said it is a filter, which takes the compressed block and thinks about it as if it was an executable code (in most cases it is). The filter normalizes jump targets before compression by changing relative position into absolute values.

This means for example that some of the jumps may be converted from the machine language "jump 1655 bytes backwards" style notation to normalized "jump to address 5554" style notation; all jumps to 5554, perhaps a common subroutine, are thus encoded identically, making them more compressible.

This normalization is of course machine dependent, thus the filter needs to be different for x86 code and also different for, say, sparc.

Using BCJ filters is mksquashfs is simple, just add -Xbcj xxx as a parameter after -comp xz, where xxx is a list of filters. Mksquashfs then tries to compress all blocks without any filter at first, and then it tries again for each given filter, and chooses the best compression.

For Slax, we simply add x86 filter since it will do the job for both 32bit and 64bit architectures. Slax core module is 2% smaller than before (52MB versus 51MB), so the difference for full Slax will be somewhere between 5-10 megabytes less of data at the cost of adding "-Xbcj x86" parameter to mksquashfs command. Cool!

User comments

Firefoxjet 2012-10-02 02:17

Cool!

:-)

fujitsusiemens 2012-10-11 00:30

ermmmm ... I'm running SLAX on a Transmeta TM5800 CPU
so "needs to be different for x86 code and also different for, say," Transmetas tingles.

Just mentioning. Seen from this perspective, 2% are not worth the difference between Working vs. NOT-working

Tomas M 2012-10-11 01:00

Why do you think it's not working? Adding the BCJ filter just applies different compression algorithm but everything WORKS as like without it.