Michael Gastpar

A detailed look at Sibson's -mutual information and its multifaceted applications.

This study explores how transformers learn from Markov processes through initialization and gradient flow.

Learn how prompt compression can enhance language model performance and reduce resource use.

Investigating transformers' interaction with Markov data reveals insights into model efficiency.