On the Randomness of Compressed Data

Shmuel T. Klein; Dana Shapira

doi:10.1109/DCC.2019.00093

On the Randomness of Compressed Data

Shmuel T. Klein, Dana Shapira

Ariel University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

It seems reasonable to expect from a good compression method that its output should not be further compressible, because it should behave essentially like random data. We investigate this premise for a variety of known compression techniques, and find that, surprisingly, there is much variability in the randomness, depending on the chosen method. Arithmetic coding seems to produce perfectly random output, whereas that of Huffman or Ziv-Lempel coding still contains many dependencies. In particular, the output of Huffman coding has already been proven to be random under certain conditions, and we show here that arithmetic coding may produce an output that is identical to that of Huffman.

Original language	English
Title of host publication	Proceedings - DCC 2019
Subtitle of host publication	2019 Data Compression Conference
Editors	James A. Storer, Joan Serra-Sagrista, Ali Bilgin, Michael W. Marcellin
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	581
Number of pages	1
ISBN (Electronic)	9781728106571
DOIs	https://doi.org/10.1109/DCC.2019.00093
State	Published - 10 May 2019
Event	2019 Data Compression Conference, DCC 2019 - Snowbird, United States Duration: 26 Mar 2019 → 29 Mar 2019

Publication series

Name	Data Compression Conference Proceedings
Volume	2019-March
ISSN (Print)	1068-0314

Conference

Conference	2019 Data Compression Conference, DCC 2019
Country/Territory	United States
City	Snowbird
Period	26/03/19 → 29/03/19

Keywords

Kullback Leibler
Lossless Compression
Randomness

Access to Document

10.1109/DCC.2019.00093

Cite this

Klein, S. T., & Shapira, D. (2019). On the Randomness of Compressed Data. In J. A. Storer, J. Serra-Sagrista, A. Bilgin, & M. W. Marcellin (Eds.), Proceedings - DCC 2019: 2019 Data Compression Conference (pp. 581). Article 8712629 (Data Compression Conference Proceedings; Vol. 2019-March). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DCC.2019.00093

@inproceedings{e06999a758e64af6941eba343570913f,

title = "On the Randomness of Compressed Data",

abstract = "It seems reasonable to expect from a good compression method that its output should not be further compressible, because it should behave essentially like random data. We investigate this premise for a variety of known compression techniques, and find that, surprisingly, there is much variability in the randomness, depending on the chosen method. Arithmetic coding seems to produce perfectly random output, whereas that of Huffman or Ziv-Lempel coding still contains many dependencies. In particular, the output of Huffman coding has already been proven to be random under certain conditions, and we show here that arithmetic coding may produce an output that is identical to that of Huffman.",

keywords = "Kullback Leibler, Lossless Compression, Randomness",

author = "Klein, {Shmuel T.} and Dana Shapira",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 Data Compression Conference, DCC 2019 ; Conference date: 26-03-2019 Through 29-03-2019",

year = "2019",

month = may,

day = "10",

doi = "10.1109/DCC.2019.00093",

language = "אנגלית",

series = "Data Compression Conference Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "581",

editor = "Storer, {James A.} and Joan Serra-Sagrista and Ali Bilgin and Marcellin, {Michael W.}",

booktitle = "Proceedings - DCC 2019",

address = "ארצות הברית",

}

Klein, ST & Shapira, D 2019, On the Randomness of Compressed Data. in JA Storer, J Serra-Sagrista, A Bilgin & MW Marcellin (eds), Proceedings - DCC 2019: 2019 Data Compression Conference., 8712629, Data Compression Conference Proceedings, vol. 2019-March, Institute of Electrical and Electronics Engineers Inc., pp. 581, 2019 Data Compression Conference, DCC 2019, Snowbird, United States, 26/03/19. https://doi.org/10.1109/DCC.2019.00093

On the Randomness of Compressed Data. / Klein, Shmuel T.; Shapira, Dana.
Proceedings - DCC 2019: 2019 Data Compression Conference. ed. / James A. Storer; Joan Serra-Sagrista; Ali Bilgin; Michael W. Marcellin. Institute of Electrical and Electronics Engineers Inc., 2019. p. 581 8712629 (Data Compression Conference Proceedings; Vol. 2019-March).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On the Randomness of Compressed Data

AU - Klein, Shmuel T.

AU - Shapira, Dana

PY - 2019/5/10

Y1 - 2019/5/10

N2 - It seems reasonable to expect from a good compression method that its output should not be further compressible, because it should behave essentially like random data. We investigate this premise for a variety of known compression techniques, and find that, surprisingly, there is much variability in the randomness, depending on the chosen method. Arithmetic coding seems to produce perfectly random output, whereas that of Huffman or Ziv-Lempel coding still contains many dependencies. In particular, the output of Huffman coding has already been proven to be random under certain conditions, and we show here that arithmetic coding may produce an output that is identical to that of Huffman.

AB - It seems reasonable to expect from a good compression method that its output should not be further compressible, because it should behave essentially like random data. We investigate this premise for a variety of known compression techniques, and find that, surprisingly, there is much variability in the randomness, depending on the chosen method. Arithmetic coding seems to produce perfectly random output, whereas that of Huffman or Ziv-Lempel coding still contains many dependencies. In particular, the output of Huffman coding has already been proven to be random under certain conditions, and we show here that arithmetic coding may produce an output that is identical to that of Huffman.

KW - Kullback Leibler

KW - Lossless Compression

KW - Randomness

UR - http://www.scopus.com/inward/record.url?scp=85066321472&partnerID=8YFLogxK

U2 - 10.1109/DCC.2019.00093

DO - 10.1109/DCC.2019.00093

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85066321472

T3 - Data Compression Conference Proceedings

SP - 581

BT - Proceedings - DCC 2019

A2 - Storer, James A.

A2 - Serra-Sagrista, Joan

A2 - Bilgin, Ali

A2 - Marcellin, Michael W.

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 Data Compression Conference, DCC 2019

Y2 - 26 March 2019 through 29 March 2019

ER -

On the Randomness of Compressed Data

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this