In February 2019, the Consultative Committee for Space Data Systems (CCSDS) published the 123.0-B-2 standard titled “Low-Complexity Lossless and Near-Lossless Multispectral and Hyperspectral Image Compression”. It included a new state-of-the-art entropy encoder: the hybrid entropy encoder. This encoder, originally proposed by NASA’s Jet Propulsion Laboratory (JPL), was designed to efficiently code the low entropy data produced by the in-loop quantizer included in the standard. This document describes some of the insights that went into the design of that entropy encoder.

It is well known that in order to have an efficient entropy encoder at low entropies, the information of multiple symbols needs to necessarily be combined into a single codeword. Otherwise, compressed data rates cannot be lower than 1 bit per sample. The hybrid encoder in 123.0-B-2 uses a set of variable-to-variable (V2V) codes. These complement the Golomb-power-of-two (GPO2) codes, which are more efficient for for higher entropy data, and were already used in previous CCSDS compression standards.V2V codes are used for low entropies, while GPO2 codes can still be employed for higher entropies. This, however, presents some technical issues.

For example, the use of V2V codes in combination with an adaptive probability model, poses an interesting challenge: it is not possible for the probability model to work directly with the V2V codes. This is because a regular decoder needs to update the probability model using symbols that the V2V codes in the encoder are still packing together, creating a Catch-22 situation further described in the manuscript. This problem was finally solved by reconfiguring the compressed data so that it can be decoded in the reverse order. This design decision not only addresses the issue, but that at the same time moves the hardware cost of addressing this situation from the space-borne encoder to the ground decoder.

Another design decision described in this document is the use of a "breadcrumbing" strategy. The encoder leaves small pieces of information at critical points in the compressed file, allowing the decoder to retrace the encoder steps but in a reverse order. In 123.0-B-2, the final state of the encoder is transmitted at the end of a compressed image so that the decoding process can begin exactly where coding stopped. In addition, the adaptive probability model is kept in sync while decoding thanks to small pieces of information embedded in the compressed file that disambiguate rounding operations.

A few insights into the design of the V2V codes themselves are included as well. For instance, a data-driven model was involved in the optimization of the V2V codes that were finally standardized. This is in contrast to probability models based on geometric distributions, which are a sufficiently well fit for GPO2 codes. For V2V codes, the use of a data-driven model yielded gains worthwhile the effort.

In addition to providing some of the design insights that went into the design of the hybrid encoder in CCSDS 123.0-B-2, this paper aims to clearly describe the decoding process. This complements the CCSDS 123.0-B-2 standard, which only describes the operations from the point of view of a compliant compressor. Due to the notable differences in the coding and decoding processes of the hybrid entropy encoder, this manuscript provides as well a straightforward description of the steps involved in the decoding process.