Substantial student participation required. The topics to be covered each semester that the course may be offered will be announced by the middle of the preceding semester; see departmental bulletins. Recent topics include: Bayesian statistics, statistics and finance, random matrix theory, high-dimensional statistics.

Information theory offers a fundamental alternative language for solving problems in probability, communication, and inference that complements traditional statistics. Classically, information theory quantifies the uncertainty in random variables and the mutual information shared between coupled variables, often in the context of communication channels and signal processing. We will discuss entropies as measures of uncertainty, information and mutual information, channel capacities, and error-correcting codes. In addition, information geometry is now widely used to define loss functions for machine learning models, entropy maximization provides axiomatic foundations for a wide variety of classical distribution families, and entropy production rates provide strong constraints on the behavior of stochastic processes, unifying the thermodynamic and information-theoretic notions of entropy. Prerequisites: Probability: Stat 134, Data 140, or equivalent, Multivariate Calculus: Math 53 or equivalent, Linear Algebra: Math 56 or equivalent. Knowledge of scientific computing in R, Python, or Matlab.

Repeat Rules