TU4.R1.4

Dense KO Codes: Faster Convergence and Reduced Complexity through Dense Connectivity

Shubham Srivastava, Adrish Banerjee, Indian Institute of Technology Kanpur, India

Session:
Deep Learning in Coding

Track:
8: Deep Learning (such as understanding large language models)

Location:
Ballroom II & III

Presentation Time:
Tue, 9 Jul, 17:05 - 17:25

Session Chair:
Natasha Devroy, University of Illinois Chicago
Abstract
This paper proposes Dense KO (DKO) codes to enhance the recently introduced KO coding framework for faster convergence and reduced model complexity. The key idea is to replace the stacked fully-connected layers in the KO encoder and decoder with a DenseNet-inspired architecture to improve parameter efficiency. Additional modifications like Mish activations further aid representation. A cyclical learning rate policy accelerates training convergence within fewer epochs. DKO codes match the error resilience of KO codes for short blocklengths, while requiring less parameters. The reduction in training time and model size facilitates the adoptability of learned coding schemes for latency-sensitive short blocklength applications.
Resources