In place differential file compression

Dana Shapira, James A. Storer

Research output: Contribution to journalReview articlepeer-review

17 Scopus citations


We present algorithms for in-place differential file compression, where a target file T of size n is compressed with respect to a source file S of size m using no additional space in addition to the that used to replace S by T; that is, it is possible to encode using m + n + O(1) space and decode using MAX(m, n) + O(1) space (so that when decoding the source file is overwritten by the decompressed target file). From a theoretical point of view, an optimal solution (best possible compression) to this problem is known to be NP-hard, and in previous work we have presented a factor of 4 approximation (not in-place) algorithm based on a sliding window approach. Here we consider practical in-place algorithms based on sliding window compression where our focus is on decoding; that is, although in-place encoding is possible, we will allow O(m + n) space for the encoder so as to improve its speed and present very fast decoding with only MAX(m, n) + O(1) space. Although NP-hardness implies that these algorithms cannot always be optimal, the asymptotic optimality of sliding window methods along with their ability for constant-factor approximation is evidence that they should work well for this problem in practice. We introduce the IPSW algorithm (in-place sliding window) and present experiments that indicate that it compares favorably with traditional practical approaches, even those that do not decode in-place, while at the same time having low encoding complexity and extremely low decoding complexity. IPSW is most effective when S and T are reasonably well aligned (most large common substrings occur in approximately the same order). We also present a preprocessing step for string alignment that can be employed when the encoder determines significant gains will be achieved.

Original languageEnglish
Pages (from-to)677-691
Number of pages15
JournalComputer Journal
Issue number6
StatePublished - Nov 2005
Externally publishedYes


  • Algorithms
  • Approximation theory
  • Computational complexity
  • Decoding
  • Encoding (symbols)
  • Optimization
  • Constant factor approximation
  • Differential file compression
  • Sliding window compression
  • Data compression


Dive into the research topics of 'In place differential file compression'. Together they form a unique fingerprint.

Cite this