TH3.R1.2

An Achievable and Analytic Solution to Information Bottleneck for Gaussian Mixtures

Yi Song, Technische Universitat Berlin, Germany; Kai Wan, Zhenyu Liao, Huazhong University of Science and Technology, China; Hao Xu, University College London, United Kingdom; Giuseppe Caire, Technische Universitat Berlin, Germany; Shlomo Shamai, Technion-Israel Institute of Technology, Israel

Session:
Information Bottleneck

Track:
8: Machine Learning

Location:
Ballroom II & III

Presentation Time:
Thu, 11 Jul, 14:55 - 15:15

Session Chair:
Lampros Gavalakis, Gustave Eiffel University
Abstract
In this paper, we consider a remote source coding problem with binary phase shift keying (BPSK) modulation sources, where observations are corrupted by additive white Gaussian noise (AWGN). An intermediate node, such as a relay, receives these observations and performs further compression to find the optimal trade-off between complexity and relevance. This problem can be formulated as an information bottleneck (IB) problem with Bernoulli sources and Gaussian mixture observations, for which no closed-form solution is known. To address this challenge, we propose a unified achievable scheme that employs three different compression strategies for intermediate node processing, i.e., two-level quantization, multi-level deterministic quantization, and soft quantization with tanh function. Comparative analyses with existing methods, such as the Blahut-Arimoto (BA) algorithm and the Information Dropout approach, are performed through numerical evaluations. The proposed analytic scheme is observed to consistently approach the (numerically) optimal performance over a range of signal-to-noise ratios (SNRs), confirming its effectiveness in the considered setting.
Resources