Channel Capacity

Channel Capacity

When to Use

Use this skill when working on channel-capacity problems in information theory.

Decision Tree

1. Mutual Information

   • I(X;Y) = H(X) + H(Y) - H(X,Y)
   • I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X)
   • Symmetric: I(X;Y) = I(Y;X)
   • scipy.stats.entropy(p) + scipy.stats.entropy(q) - joint_entropy

2. Channel Model

   • Input X, output Y, channel P(Y|X)
   • Channel matrix: rows = inputs, columns = outputs
   • Element (i,j) = P(Y=j | X=i)

3. Channel Capacity

   • C = max_{p(x)} I(X;Y)
   • Maximize over input distribution
   • Achieved by capacity-achieving distribution

4. Common Channels

   Channel	Capacity
   Binary Symmetric (BSC)	1 - H(p) where p = crossover prob
   Binary Erasure (BEC)	1 - epsilon where epsilon = erasure prob
   AWGN	0.5 * log2(1 + SNR)

5. Blahut-Arimoto Algorithm

   • Iterative algorithm to compute capacity
   • Alternates between optimizing p(x) and p(y|x)
   • Converges to capacity
   • z3_solve.py prove "capacity_upper_bound"

Tool Commands