Tuning Up: From A/B Testing to Bayesian Optimization

What This Book Covers

Modern production systems—from recommender engines to automated trading strategies—become measurably better through live experimentation. David Sweet draws on years of hands-on experience to show how proven experimental methods can tune any engineered system that interacts with real users or markets.

From Simple Tests to Automated Optimization 📊

The book builds methodically from A/B testing to Bayesian optimization, with each technique presented in prose, mathematics, and code. You will see the logic, the formulas, and practical implementation details that make abstract concepts concrete.

Key methods covered

• A/B testing
• Multi-armed bandits
• Response surface methodology
• Contextual bandits
• Bayesian optimization

Real-World Workflows for Three Engineering Roles

The book grounds each method in the daily work of machine learning engineers, quantitative analysts, and software engineers. You will follow concrete examples—ranking news headlines, tuning trading strategies, and reducing web server bounce rates—to understand how live measurements drive system improvement.

Why Experiments Beat Assumptions

Offline estimates and domain knowledge are useful, but they cannot replace live data. This book explains why real-world experimentation is uniquely valuable, how to design experiments that minimize risk, and how to extract maximum insight from every test you run.

Practical Guidance for Serious Tuning

Sweet includes realistic expectations about timescales, data sizes, and operational challenges, supported by interviews with practitioners and published results from large technology companies. If you need to optimize ad placement, refine a recommender, or improve a trading system, this MEAP edition gives you early access to a complete, battle-tested playbook.