Cambrian: AI-Enhanced ML Research Platform

Distributionally Robust Optimization: A Review: This paper provides a comprehensive review of distributionally robust optimization (DRO) and its relationships with other optimization concepts such as robust optimization, risk-aversion, chance-constrained optimization, and function regularization. It explores the main concepts and contributions to DRO, highlighting its growing importance in both the operations research and statistical learning communities.

Distributionally Robust Bayesian Optimization: This paper focuses on the problem of robustness to distributional shift in contemporary machine learning. It introduces a novel algorithm called distributionally robust Bayesian optimization (DRBO) that seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift. The algorithm uses the maximum mean discrepancy (MMD) to measure the distributional shift and achieves sub-linear robust regret in various settings. The effectiveness of the algorithm is demonstrated through experiments on synthetic and real-world benchmarks.

Distributionally Robust Optimization with Correlated Data from Vector Autoregressive Processes: This paper presents a distributionally robust formulation of a stochastic optimization problem for non-i.i.d vector autoregressive data. It defines robustness using the Wasserstein distance and shows that the problem is equivalent to a finite convex-concave saddle point problem. The paper demonstrates the performance of the method using both synthetic and real data.

Optimal algorithms for group distributionally robust optimization and beyond: This paper focuses on designing stochastic algorithms for a class of distributionally robust optimization (DRO) problems, including group DRO, subpopulation fairness, and empirical conditional value at risk (CVaR) optimization. The algorithms achieve faster convergence rates compared to existing methods for multiple DRO settings. The paper also provides a new information-theoretic lower bound and demonstrates the superior performance of the algorithms through empirical experiments.

Distributionally Robust Optimization with Markovian Data: This paper addresses a stochastic program where the probability distribution of uncertain problem parameters is unknown but can be indirectly observed through finitely many correlated samples generated by an unknown Markov chain. The paper proposes a data-driven distributionally robust optimization model that estimates the problem’s objective function and optimal solution. It leverages results from large deviations theory to derive statistical guarantees and introduces a customized Frank-Wolfe algorithm for solving the problem efficiently. Numerical experiments demonstrate the method’s computational and statistical advantages.

A Distributionally Robust Boosting Algorithm: This paper establishes a connection between distributionally robust optimization (DRO) and boosting algorithms. It introduces a boosting type algorithm named DRO-Boosting, which solves a DRO problem. The paper applies the algorithm to a financial dataset on credit card default payment prediction and compares its performance to alternative boosting methods.

Learning Distributionally Robust Models at Scale via Composite Optimization: This paper addresses the challenge of learning distributionally robust models from large datasets. It shows that different variants of distributionally robust optimization (DRO) can be formulated as instances of a finite-sum composite optimization problem. The paper proposes scalable methods for solving this problem and provides empirical results that demonstrate the effectiveness of the proposed algorithm in learning robust models from large datasets.

Distributionally Robust Learning: This monograph develops a comprehensive statistical learning framework that is robust to perturbations in the data using distributionally robust optimization (DRO) under the Wasserstein metric. It explores fundamental properties of the Wasserstein metric and DRO formulation, develops tractable formulations using duality, and provides finite-sample and asymptotic performance guarantees for various learning problems.

DORO: Distributional and Outlier Robust Optimization: This paper addresses the issue of sensitivity to outliers in distributionally robust optimization (DRO) and proposes a framework called Distributional and Outlier Robust Optimization (DORO). DORO prevents DRO from overfitting to outliers by using a refined risk function. The paper presents theoretical proofs of the proposed method’s effectiveness and demonstrates its improved performance and stability compared to DRO through experiments on large modern datasets.

Distributionally Robust Bayesian Quadrature Optimization: This paper studies Bayesian quadrature optimization (BQO) under distributional uncertainty, where the underlying probability distribution is unknown except for a limited set of samples. It proposes a method to maximize the expectation of an expensive black-box integrand given the fixed sample set. The paper compares the proposed approach to standard BQO and shows its effectiveness in handling small sample sizes and high variance.

Cambrian