Bhattacharyya bound, cutoff rate, and constellation design for the companding channel

In modern voiceband data communication, the received signal is subject to nonlinear quantization noise due to companding. Under certain conditions, this quantization noise may become dominant and cause serious degradation in performance. In this paper, we first calculate the Bhattacharyya bound on the error probability between two signal points, which we then use to obtain some insight into the effects of the companding channel. Next, we compute the cutoff rate, and then use these results to design new "optimal" signal constellations for the companding channel. The bound, cutoff rate, and new constellations are computed assuming a signal-dependent noncircular Gaussian channel transition density. In addition, we obtain "optimal" a priori probabilities for this channel.