Abstract:
An underdetermined Bernoulli source generates symbols of a given underdetermined alphabet independently with some probabilities. To each underdetermined symbol there corresponds a set of basic (fully defined) symbols such that it can be substituted (specified) by any of them. An underdetermined source is characterized by its entropy, which is implicitly introduced as a minimum of a certain function and plays a role similar to the Shannon entropy for fully defined sources. Coding of an underdetermined source must ensure, for any sequence generated by the source, recovering some its specification. Coding is asymptotically optimal if the average code length is asymptotically equal to the source entropy. Coding is universal if it does not depend on the probabilities of source symbols. We describe an asymptotically optimal universal coding method for underdetermined Bernoulli sources for which the encoding and decoding procedures can be realized by RAM-programs of almost linear complexity.
Keywords:underdetermined source, specification, entropy of an underdetermined source, quasientropy of a word, combinatorial entropy of a class, underdetermined source coding, universal coding, polynomial algorithm.
Fulltext:
PDF file (239 kB)
