Multi-Armed Bandits

A bandit is online decision-making under uncertainty: you have several options (arms), each with an unknown payoff, and every choice both earns a reward and teaches you something. The tension is explore versus exploit: try uncertain arms to learn, or pull the current best to win. Epsilon-greedy, UCB, and Thompson sampling balance that tradeoff; contextual bandits pick per request using features. Unlike a fixed A/B test, a bandit shifts traffic toward winners as it learns, cutting the cost of running a loser.