Forward Modeling in Adaptive Compression: Bounds and Experimental Evaluation

Adaptive encoding plays a central role in data compression, allowing symbol probabilities to be estimated and updated dynamically. This paper presents a detailed comparison between the forward model, which assumes prior knowledge of the symbol frequencies, and the backward adaptive model, which increments the frequencies during encoding and decoding. We derive an exact expression for the bit-length difference between the two and show that for data distributed by Zipf's law, the forward encoding is beneficial even when we count the cost of information about symbols' frequencies. Extensive experiments conducted on real-world text corpora of varying sizes support the new theoretical results.