Loading…
Saturday, July 19
 

8:30am PDT

Summer School - Check In
Saturday July 19, 2025 8:30am - 9:00am PDT
Saturday July 19, 2025 8:30am - 9:00am PDT
TBA

9:00am PDT

Summer School - Opening Remarks
Saturday July 19, 2025 9:00am - 9:30am PDT
Saturday July 19, 2025 9:00am - 9:30am PDT
Seeley G. Mudd Building (SGM) 123 3620 McClintock Ave, 123, Los Angeles, CA 90089

9:30am PDT

Summer School - Course 1: Distributed secure and privacy-aware optimization
Saturday July 19, 2025 9:30am - 12:30pm PDT
SUMMER SCHOOL COURSE  #1: Distributed secure and privacy-aware optimization
LECTURER: Dr. Meisam Razaviyayn
TIME: 9:30 am-12:30 pm
BUILDING/ROOM: SGM 123

Speakers
avatar for Meisam Razaviyayn

Meisam Razaviyayn

Associate Professor, University of Southern California
Bio: Meisam Razaviyayn (https://sites.usc.edu/razaviyayn) is an associate professor in the departments of Industrial and Systems Engineering, Computer Science, Quantitative and Computational Biology, and Electrical Engineering at the University of Southern California. He also serves as the associate director of the USC-Meta Center for... Read More →
Saturday July 19, 2025 9:30am - 12:30pm PDT
Seeley G. Mudd Building (SGM) 123 3620 McClintock Ave, 123, Los Angeles, CA 90089

12:30pm PDT

Summer School - Lunch 1
Saturday July 19, 2025 12:30pm - 1:30pm PDT
Chicken or Vegetarian CAVA bowls. Dessert and water will be available.
Saturday July 19, 2025 12:30pm - 1:30pm PDT
Olin Hall of Engineering (OHE) Patio 3650 McClintock Ave, Los Angeles, CA 90089

1:30pm PDT

Summer School - Course 2: Nonconvex stochastic optimization
Saturday July 19, 2025 1:30pm - 4:30pm PDT
SUMMER SCHOOL COURSE #2: Nonconvex stochastic optimization
LECTURER: Dr. Ying Cui
BUILDING/ROOM: SGM 123
TIME: 1:30-4:30 pm

Speakers
YC

Ying Cui

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Saturday July 19, 2025 1:30pm - 4:30pm PDT
Seeley G. Mudd Building (SGM) 123 3620 McClintock Ave, 123, Los Angeles, CA 90089
 
Sunday, July 20
 

9:30am PDT

Summer School - Course 3: Recent advances in continuous optimization for discrete problems
Sunday July 20, 2025 9:30am - 12:30pm PDT
SUMMER SCHOOL COURSE  #3: Recent advances in continuous optimization for discrete problems
LECTURER: Dr.Jon Lee
TIME: 9:30 am-12:30 pm
BUILDING/ROOM: SGM 123

Speakers
JL

Jon Lee

Jon Lee obtained his Ph.D. at Cornell University. He has held long-term positions at Yale University, University of Kentucky, IBM Research, and New York University. Now at the University of Michigan, Jon is the G. Lawton and Louise G. Johnson Professor of Engineering and Professor... Read More →
Sunday July 20, 2025 9:30am - 12:30pm PDT
Seeley G. Mudd Building (SGM) 123 3620 McClintock Ave, 123, Los Angeles, CA 90089

12:30pm PDT

Summer School - Lunch 2
Sunday July 20, 2025 12:30pm - 1:30pm PDT
Chicken or Vegetarian CAVA bowls. Dessert and water will be available.
Sunday July 20, 2025 12:30pm - 1:30pm PDT
Olin Hall of Engineering (OHE) Patio 3650 McClintock Ave, Los Angeles, CA 90089

1:30pm PDT

Summer School - Course 4: Variational analysis and computation
Sunday July 20, 2025 1:30pm - 4:30pm PDT
SUMMER SCHOOL COURSE #4: Variational analysis and computation
LECTURER: Dr. Johannes Royset
BUILDING/ROOM: SGM 123
TIME: 1:30-4:30 pm

Speakers
JR

Johannes Royset

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Sunday July 20, 2025 1:30pm - 4:30pm PDT
Seeley G. Mudd Building (SGM) 123 3620 McClintock Ave, 123, Los Angeles, CA 90089

3:00pm PDT

Early Check In
Sunday July 20, 2025 3:00pm - 5:00pm PDT
Sunday July 20, 2025 3:00pm - 5:00pm PDT
Taper Hall (THH) 3501 Trousdale Pkwy, Los Angeles, CA 90089

4:00pm PDT

Break
Sunday July 20, 2025 4:00pm - 5:00pm PDT
Sunday July 20, 2025 4:00pm - 5:00pm PDT
TBA

6:30pm PDT

ICCOPT Conference Welcome Mixer (Tickets Needed)
Sunday July 20, 2025 6:30pm - 8:30pm PDT
Tickets Needed
Sunday July 20, 2025 6:30pm - 8:30pm PDT
California Science Center
 
Monday, July 21
 

8:00am PDT

Auditorium opens for seating
Monday July 21, 2025 8:00am - 8:45am PDT
Monday July 21, 2025 8:00am - 8:45am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

8:00am PDT

Check In
Monday July 21, 2025 8:00am - 8:45am PDT
Monday July 21, 2025 8:00am - 8:45am PDT
Taper Hall (THH) 3501 Trousdale Pkwy, Los Angeles, CA 90089

8:45am PDT

Opening Remarks
Monday July 21, 2025 8:45am - 9:00am PDT
Monday July 21, 2025 8:45am - 9:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

9:00am PDT

Plenary 1
Monday July 21, 2025 9:00am - 10:00am PDT
Speakers
SJ

Stephen J. Wright

UW-Madison
Stephen J. Wright is the George B. Dantzig Professor of Computer Sciences, Sheldon Lubar Chair of Computer Sciences, and Hilldale Professor at the University of Wisconsin-Madison. He also serves as Chair of the Computer Sciences Department. His research is in computational optimization... Read More →
Monday July 21, 2025 9:00am - 10:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

10:00am PDT

Coffee & Snack Break (Provided)
Monday July 21, 2025 10:00am - 10:30am PDT
Monday July 21, 2025 10:00am - 10:30am PDT
TBA

10:30am PDT

Parallel Sessions 1A: Optimization and Resilience in the Power Grid
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Optimization and Resilience in the Power Grid
Chair: Madeleine Udell
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Leveraging predictions for optimal voltage control: an adaptive approach
Speaker: Wenqi Cui
Abstract: High variability of solar PV and sudden changes in load (e.g., electric vehicles and storage) can lead to large voltage fluctuations. In recent years, a number of advanced controllers have been designed to optimize voltage control. These controllers, however, almost always assume that the net load in the system remains constant over a sufficiently long time. Given the intermittent and uncertain nature of renewable resources, it is becoming important to explicitly consider net load that is time-varying. This talk will describe an adaptive approach to voltage control in power systems with a significant time-varying net load. We leverage the advances in short-term load forecasting, where the net load in the system can be predicted using local measurements. We integrate these predictions into the design of adaptive controllers, and prove that the overall control architecture achieves input-to-state stability in a decentralized manner. We optimize the control policy through a sample-efficient reinforcement learning framework, which update the control policy successively with online date collection . Case studies are conducted using time-varying load data from Caltech's campus grid.

Talk 2: Learning-enhanced Design and Optimization of Microgrids under Uncertainty
Speaker: Harsha Nagarajan
Abstract: To mitigate the vulnerability of distribution grids to severe weather events, some electric utilities use preemptive de-energization as a primary defense, leading to significant power outages. Networked microgrids can enhance resiliency and maximize load delivery, but challenges arise from modeling unbalanced three-phase networks and managing uncertainties in renewables and loads. We present a two-stage mixed-integer robust optimization approach to configure and operate networked microgrids, ensuring robustness to all net-load uncertainties while maximizing load delivery. To solve this problem, we propose an ML-accelerated cutting-plane algorithm with convergence guarantees, which approximates a recourse function with cuts predicted by an ML regressor for worst-case uncertain scenarios. A case study on the IEEE 37-bus system demonstrates the economic benefits of networking microgrids to maximize load delivery.

Talk 3: A Reliability Puzzle for Large Scale Batteries
Speaker: Steven Diamond
Abstract: Large scale batteries are playing an increasingly prominent role in electricity markets. Battery operators earn revenue through two main sources: energy arbitrage and ancillary services. Ancillary services are commitments to support the grid through actions outside of the standard market mechanism. For example, a battery might promise to provide extra power to the grid in an emergency. In this talk we explore the reliability opportunities and challenges posed by batteries selling ancillary services. We discuss the contrasting approaches taken by the California and Texas electricity markets and propose a new mechanism that better aligns the interests of market regulators and battery operators.

Speakers
avatar for Madeleine Udell

Madeleine Udell

Postdoctoral Fellow, Caltech Center for the Mathematics of Information
Madeleine Udell is a postdoctoral fellow at Caltech's Center for the Mathematics of Information, hosted by Joel Tropp. She will be joining Cornell as an Assistant Professor in the School of Operations Research and Information Engineering in July 2016. Her research focus is on modeling... Read More →
avatar for Harsha Nagarajan

Harsha Nagarajan

Los Alamos National Laboratory
Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SD

Steven Diamond

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1B: Regularization Methods for stochastic operator splitting dynamics
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Regularization Methods for stochastic operator splitting dynamics
Chair: Mathias Staudigl
Cluster: Nonsmooth Optimization

Talk 1: Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces
Speaker: Rodrigo Maulen
Abstract: To solve convex optimization problems with a noisy gradient input, we analyze the global behavior of subgradient-like flows under stochastic errors. The objective function is composite, being equal to the sum of two convex functions, one being differentiable and the other potentially non-smooth. We then use stochastic differential inclusions where the drift term is minus the subgradient of the objective function, and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz's continuity of the differentiable term and a growth condition of the non-smooth term, our first main result shows almost sure weak convergence of the trajectory process towards a minimizer of the objective function. Then, using Tikhonov regularization with a properly tuned vanishing parameter, we can obtain almost sure strong convergence of the trajectory towards the minimum norm solution. We find an explicit tuning of this parameter when our objective function satisfies a local error-bound inequality. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex and strongly convex case.

Talk 2: Accelerated Variance-Reduced Forward-Reflected Methods for Generalized Equations
Speaker: Quoc Tran-Dinh
Abstract: In this talk, we propose a novel class of Nesterov’s stochastic accelerated forward- reflected-based methods with variance reduction to solve [non]linear equations under a co-coerciveness assumption. Our algorithm is single-loop and leverages a new family of unbiased variance-reduced estimators specifically designed for a wider class of root-finding algorithms. It achieves both O(L^{2}/k^{2}) and o(1/k^2)-last-iterate convergence rates in terms of expected operator squared norm, where k denotes the iteration counter. We instantiate our framework for two prominent estimators: SVRG and SAGA. By an appropriate choice of parameters, both variants attain an oracle complexity of O(n+ Ln^{2/3}ϵ−1) to reach an ϵ-solution, where n represents the number of summands in the finite-sum operator. Furthermore, under µ-strong quasi-monotonicity, our method achieves a linear convergence rate and an oracle complexity of O(n+ max{n,n^{2/3}κ} log(ϵ−1)), where κ := L/µ. We extend our approach to solve a class of finite-sum monotone inclusions, demonstrating that our schemes retain the same theoretical guarantees as in the equation setting. Finally, we present some numerical experiments to validate our algorithms and demonstrate their promising performance compared to state-of-the-art methods.

Talk 3: Tikhonov regularized exterior penalty dynamics for constrained variational inequalities
Speaker: Mathias Staudigl
Abstract: A recent trend in optimization and variational analysis is the investigation of dynamical systems, either in discrete or continuous time, with explicit regularization of the trajectory using either Tikhonov of the Halpern method. These schemes exhibit a higher degree of stability and in some cases also exhibit interesting acceleration phenomena. Within an infinite-dimensional Hilbert space context, this paper is concerned with the study of a class of constrained variational inequalities in which the constrained domain can be represented as the zero set of a maximally monotone operator. The Tikhonov regularization parameter is assumed to tend to zero as time tends to infinity, which preserves equilibria and serves to enforce strong convergence of the trajectory. To steer the dynamical system towards the feasible set over which the original variational problem is to be solved, we append to the trajectory and exterior penalization term. The combination of the Tikhonov regularization and the penalization lead to a multiscale dynamical system, pioneered in the work of Attouch, Czarnecki, Peypouqet, and many others. In this paper we study the case of a cocoercive and non-cocoercive monotone operator separately. Full Splitting based dynamical systems, based on the Forward-Backward and Forward-Backward-Forward Splitting method are constructed. Proofs of existence and uniqueness of the solution trajectories of these new time-varying dynamical systems, as well as the strong convergence towards the least norm solution of the underlying variational inequality problem are derived. Numerical experiments conclude this work, showing the reconstruction power of the approach.

Speakers
RM

Rodrigo Maulen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
QT

Quoc Tran-Dinh

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MS

Mathias Staudigl

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1C: Methods for Large-Scale Nonlinear Optimization I
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Methods for Large-Scale Nonlinear Optimization I
Chair: Albert Berahas
Cluster: Nonlinear Optimization

Talk 1: Advanced, Adaptive and Flexible Algorithms for Decentralized Optimization
Speaker: Albert Berahas
Abstract: The problem of optimizing an objective function by employing a decentralized procedure using multiple agents in a connected network has gained significant attention over the last decades. This is due to the wide applicability of decentralized optimization to many important science and engineering applications such as, optimal control, machine learning, robotics, sensor networks, and smart grids. Decentralized optimization problems come in diverse shapes and forms, and could have very different characteristics. In this talk, we discuss novel flexible approaches for solving decentralized optimization problems that adapt to problem characteristics. We present two unifying algorithmic frameworks that recover popular algorithms as special cases. We discuss the rationale behind our proposed techniques, convergence in expectation and complexity guarantees for our algorithms, and present encouraging numerical results.

Talk 2: Nearly Optimal L_p Risk Minimization
Speaker: Zhichao Jia
Abstract: Convex risk measures play a foundational role in the area of stochastic optimization. However, in contrast to risk neutral models, their applications are still limited due to the lack of efficient solution methods. In particular, the mean L_p semi-deviation is a classic risk minimization model, but its solution is highly challenging due to the composition of concave-convex functions and the lack of uniform Lipschitz continuity. In this talk, we discuss some progresses on the design of efficient algorithms for L_p risk minimization, including a novel lifting reformulation to handle the concave-convex composition, and a new stochastic approximation method to handle the non-Lipschitz continuity. We establish an upper bound on the sample complexity associated with this approach and show that this bound is not improvable for L_p risk minimization in general.

Talk 3: Higher order polynomial model-based derivative-free methods
Speaker: Abraar Chaudhry
Abstract: Coming sTraditional model-based derivative free methods, such as those pioneered by Powell, iteratively construct second-order interpolation models of the objective function and optimize these models over trust-regions. Higher order polynomials may provide better models, however, they are hard to optimize. We propose a new approach of constructing and optimizing higher degree polynomial models for derivative free optimization. This is done using techniques from polynomial optimization and sum of squares. We will discuss the practical and theoretical properties of our method. oon

Speakers
AB

Albert Berahas

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZJ

Zhichao Jia

PhD Student, Georgia Institute of Technology
Name: Zhichao JiaTitle: PhD StudentAffiliation: Georgia Institute of Technology
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1D: GPU-Accelerated Mathematical Programming (Part 1)
Monday July 21, 2025 10:30am - 11:45am PDT
Session: GPU-Accelerated Mathematical Programming (Part 1)
Chair: Haihao Lu
Cluster: Computational Software

Talk 1: HPR-LP: An implementation of an HPR method for solving linear programming
Speaker: Defeng Sun
Abstract: In this talk, we aim to introduce an HPR-LP solver, an implementation of a Halpern Peaceman-Rachford (HPR) method with semi-proximal terms for solving linear programming (LP). We start with showing that the HPR method enjoys the highly desired iteration complexity of O(1/k) in terms of the Karush-Kuhn-Tucker residual and the objective error via the theory developed recently for accelerated degenerate proximal point methods. Based on the complexity results, we then design an adaptive strategy of restart and penalty parameter update to improve the efficiency and robustness of the HPR method. We conduct extensive numerical experiments on different LP benchmark datasets using NVIDIA A100-SXM4-80GB GPU in different stopping tolerances. Our solver's Julia version achieves a 2.39x to 5.70x speedup measured by SGM10 on benchmark datasets with presolve (2.03x to 4.06x without presolve) over the award-winning solver PDLP with the tolerance of 10^{-8}. Several practical techniques underlining the efficiency of solver will be highlighted.

Talk 2: Restarted Halpern PDHG for linear programming
Speaker: Jinwen Yang
Abstract: We propose and analyze a new matrix-free primal-dual algorithm, called restarted Halpern primal-dual hybrid gradient (rHPDHG), for solving linear programming (LP). We show that rHPDHG can achieve optimal accelerated linear convergence on feasible and bounded LP. Furthermore, we present a refined analysis that demonstrates an accelerated two-stage convergence of rHPDHG over the vanilla PDHG with an improved complexity for identification and an accelerated eventual linear convergence that does not depend on the conservative global Hoffman constant. Regarding infeasible LP, we show that rHPDHG can recover infeasibility certificates with an accelerated linear rate, improving the previous convergence rates. Furthermore, we discuss an extension of rHPDHG by adding reflection operation (which is dubbed as ), and demonstrate that it shares all theoretical guarantees of rHPDHG with an additional factor of 2 speedup in the complexity bound. Lastly, we build up a GPU-based LP solver, and the experiments showcase an improved numerical performance compared to cuPDLP.jl.

Talk 3: GPU-Accelerated Linear Programming and Beyond
Speaker: Haihao Lu
Abstract: In this talk, I will talk about the recent trend of research on new first-order methods for scaling up and speeding up linear programming (LP) and convex quadratic programming (QP) with GPUs. The state-of-the-art solvers for LP and QP are mature and reliable at delivering accurate solutions. However, these methods are not suitable for modern computational resources, particularly GPUs. The computational bottleneck of these methods is matrix factorization, which usually requires significant memory usage and is highly challenging to take advantage of the massive parallelization of GPUs. In contrast, first-order methods (FOMs) only require matrix-vector multiplications, which work well on GPUs and have already accelerated machine learning training during the last 15 years. This ongoing line of research aims to scale up and speed up LP and QP by using FOMs and GPUs. I’ll discuss how we can achieve this by explaining: (i) the behaviors of FOMs for LP; (ii) computational results on GPUs; (iii) theoretical results, including complexity theory and condition number theory, and how theory can lead to better computation and better understanding of the algorithm’s performance. If time permits, I’ll also talk about how to extend it to QP.

Speakers
DS

Defeng Sun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JY

Jinwen Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HL

Haihao Lu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1E: Large-scale Optimization Algorithms and Implementations
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Large-scale Optimization Algorithms and Implementations
Chair: Johannes Brust
Cluster: Computational Software

Talk 1: GPU Implementation of Algorithm NCL
Speaker: Michael Saunders
Abstract: For constrained optimization, LANCELOT solves about 10 subproblems that minimize an augmented Lagrangian subject to bounds and are immune to LICQ and MPEC difficulties. Algorithm NCL solves equivalent subproblems that are suited to nonlinear interior methods. We focus on reducing the associated KKT systems to smaller systems that can be solved by Cholesky-type factorizations. Our NCL implementation is based on MadNLP.jl (a nonlinear optimization solver working on GPU) and CUDSS.jl (a Julia interface to the NVIDIA library cuDSS). We present numerical results on large SCOPF problems (which are MPECS). (joint work with Alexis Montoison, François Pacaud, Sungho Shin, Dominique Orban)

Talk 2: A Two-Stage Optimization Based Algorithm for Tensor Decomposition
Speaker: Zequn Zheng
Abstract: Tensor canonical polyadic decomposition is important in exploring the multi-dimensional tensor structure. It has become an increasingly active area of research and has important applications in data science, statistics, and engineering. However, it is difficult to find a tensor decomposition when the tensor's rank is greater than the second largest dimension. In this case, traditional optimization methods like nonlinear least squares or alternative least squares methods usually fail to find the tensor decomposition. Direct methods typically suffer from high computation costs. We propose a novel two-stage optimization based algorithm for the general tensor decomposition problem when the rank is between the largest dimension and the second largest dimension. We will discuss the equivalence between tensor decompositions and the global minimizers of the two-stage optimization problems. We will also show promising numerical results of our algorithm compared with other state-of-the-art methods for tensor decomposition. (joint work with Hongchao Zhang)

Talk 3: A nonsmooth exact penalty method for equality-constrained optimization: Complexity and implementation
Speaker: Dominique Orban
Abstract: Penalty methods are a well known class of algorithms for constrained optimization. They transform a constrained problem into a sequence of unconstrained penalized problems in the hope that approximate solutions of the latter converge to a solution of the former. If Lagrange multipliers exist, exact penalty methods ensure that the penalty parameter only need increase a finite number of times, but are typically scorned in smooth optimization for the penalized problems are not smooth. This led researchers to consider the implementation of exact penalty methods inconvenient. Recently, advances in proximal methods have led to increasingly efficient solvers for nonsmooth optimization. We show that the exact ℓ2-penalty method for equality-constrained optimization can in fact be implemented efficiently by solving the penalized problem with a proximal-type algorithm. We study the convergence of our algorithm and establish a worst-case complexity bound of O(ϵ^{−2}) to bring a stationarity measure below ϵ > 0 under the Mangarasian-Fromowitz constraint qualification and Lipschitz continuity of the objective gradient and constraint Jacobian. In a degenerate scenario where the penalty parameter grows unbounded, the complexity becomes O(ϵ^{−8}), which is worse than another bound found in the literature. We justify the difference by arguing that our feasibility measure is properly scaled. Finally, we report numerical experience on small-scale problems from a standard collection and compare our solver with an augmented-Lagrangian and an SQP method. Our preliminary implementation is on par with the augmented Lagrangian in terms of robustness and efficiency. It is on par with the SQP method in terms of robustness, though the former remains ahead in terms of number of problem function evaluations.

Speakers
avatar for Michael Saunders

Michael Saunders

Professor (Research) Emeritus, Stanford University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DO

Dominique Orban

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1F: Optimization as the Engine of Generative AI - I
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Optimization as the Engine of Generative AI - I
Chair: Yinbin Han
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: InfAlign: Inference-aware language model alignment
Speaker: Theertha Suresh
Abstract: Language model alignment is a critical step in training modern generative language models. Alignment targets to improve the win rate of a sample from the aligned model against the base model. Today, we are increasingly using inference-time algorithms (e.g., Best-of-N, controlled decoding, tree search) to decode from language models rather than standard sampling. In this talk, we will first overview different inference-time algorithms and the standard RLHF procedure. We then show that this train/test mismatch makes the standard RLHF framework sub-optimal in view of such inference-time methods. To this end, we propose a framework for inference-aware alignment (InfAlign), which aims to optimize inference-time win rate of the aligned policy against the base model. We prove that for any inference-time decoding procedure, the optimal aligned policy is the solution to the standard RLHF problem with a transformation of the reward. This motivates us to provide the calibrate-and-transform RL (InfAlign-CTRL) algorithm to solve this problem, which involves a reward calibration step and a KL-regularized reward maximization step with a transformation of the calibrated reward. For best-of-N sampling and best-of-N jailbreaking, we propose specific transformations offering up to 3-8% improvement on inference-time win rates. Finally, we also show that our proposed reward calibration method is a strong baseline for optimizing standard win rate.

Talk 2: LLMs for MILP Solver Configuration
Speaker: Connor Lawless
Abstract: Mixed integer linear programming (MILP) solvers ship with a staggering number of parameters that are challenging to select a priori for all but expert optimization users, but can have an outsized impact on the performance of the MILP solver. We introduce a new LLM-based framework to configure which cutting plane separators to use for a given MILP problem with little to no training data based on characteristics of the instance, such as a natural language description of the problem and the associated LaTeX formulation. Our LLM-based methodology requires no custom solver interface, can find a high-performing configuration by solving only a small number of MILPs, and can generate the configuration with simple API calls that run in under a second.

Talk 3: Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence
Speaker: Yinbin Han
Abstract: Diffusion models have emerged as powerful tools for generative modeling, demonstrating exceptional capability in capturing target data distributions from large datasets. However, fine-tuning these massive models for specific downstream tasks, constraints, and human preferences remains a critical challenge. While recent advances have leveraged reinforcement learning algorithms to tackle this problem, much of the progress has been empirical, with limited theoretical understanding. To bridge this gap, we propose a stochastic control framework for fine-tuning diffusion models. Building on denoising diffusion probabilistic models as the pre-trained reference dynamics, our approach integrates linear dynamics control with Kullback-Leibler regularization. We establish the well-posedness and regularity of the stochastic control problem and develop a policy iteration algorithm (PI-FT) for numerical solution. We show that PI-FT achieves global convergence at a linear rate. Unlike existing work that assumes regularities throughout training, we prove that the control and value sequences generated by the algorithm maintain the regularity. Additionally, we explore extensions of our framework to parametric settings and continuous-time formulations.

Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1G: Robust decision making in dynamic environments
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Robust decision making in dynamic environments
Chair: Tobias Sutter
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Towards Optimal Offline Reinforcement Learning
Speaker: Mengmeng Li
Abstract: We study offline reinforcement learning problems with a long-run average reward objective. The state-action pairs generated by any fixed behavioral policy thus follow a Markov chain, and the empirical state-action-next-state distribution satisfies a large deviations principle. We use the rate function of this large deviations principle to construct an uncertainty set for the unknown true state-action-next-state distribution. We also construct a distribution shift transformation that maps any distribution in this uncertainty set to a state-action-next-state distribution of the Markov chain generated by a fixed evaluation policy, which may differ from the unknown behavioral policy. We prove that the worst-case average reward of the evaluation policy with respect to all distributions in the shifted uncertainty set provides, in a rigorous statistical sense, the least conservative estimator for the average reward under the unknown true distribution. This guarantee is available even if one has only access to one single trajectory of serially correlated state-action pairs. The emerging robust optimization problem can be viewed as a robust Markov decision process with a non-rectangular uncertainty set. We adapt an efficient policy gradient algorithm to solve this problem. Numerical experiments show that our methods compare favorably against state-of-the-art methods.

Talk 2: Optimal Decision Making in Abstract Stochastic Environments
Speaker: Radek Salac
Abstract: Given data from an abstract stochastic process, we study how to construct an optimal decision for a general stochastic optimization problem in a statistically optimal manner. Our approach seeks to identify decisions, where the corresponding risk of the shifted regret, evaluated on the underlying stochastic data process, converges to zero at a specified exponential rate under a minimal shift. This optimal decision emerges as a solution to a specific multi-objective optimization problem driven by the properties of the data-generating process, particularly by a corresponding rate function—a generalization of the well-known concept from large deviation theory. Moreover, the regret of such decision itself is proven to converge to zero, providing a notion of consistency. Our findings are established within a highly abstract framework, of which the above interpretation is a mere instance. The characterization of risk, crucial to our main results, is grounded in the concept of a so-called maxitive integral and its properties, which resemble the less universal Varadhan Lemma in the context of asymptotic relative entropy. Several well-known results from the literature on data-driven decision-making under uncertainty can be recovered as special cases within our general framework.

Talk 3: Revisiting Model Selection for Math Programming-Based Approximate Dynamic Programming
Speaker: Negar Soheili
Abstract: Approximations of Markov decision processes are widely used to develop policies for large-scale sequential decision-making problems. Math-programming-based approximate dynamic programming (ADP) methods, such as approximate linear programming (ALP) and pathwise optimization (PO), are notable for providing both policies and upper bounds on policy performance. ALP and PO typically solve large-scale linear or convex programming models, with recent advances in first-order methods improving solvability. Traditionally, ADP models are compared, assuming fixed features and that models can be solved optimally, in which case PO outperforms ALP. However, with machine learning techniques like random features, both ALP and PO become asymptotically optimal as more features are added. We revisit model selection between ALP and PO under random features and introduce a new ALP relaxation that improves both quality and solvability compared to the original ALP for fixed features. When no clear dominance exists between the ALP relaxation and PO models, solution methods and model structure become critical. Our ALP relaxation has a separability structure, making it preferable to PO both theoretically and numerically when using block-coordinate descent, the state-of-the-art method for solving PO. These findings offer new insights into model selection for math-programming-based ADP, where feature architecture, model, and solution method must be considered collectively, which is potentially relevant beyond these specific approaches.

Speakers
TS

Tobias Sutter

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ML

Mengmeng Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RS

Radek Salac

Doctoral Student, University of Konstanz
Name: Radek SalacTitle: Doctoral Student at Chair of Machine Learning and OptimizationAffiliation: University of Konstanz
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1H: Frontiers of Optimization for Machine Learning - Part I
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Frontiers of Optimization for Machine Learning - Part I
Chair: Fred Roosta
Cluster: Nonlinear Optimization

Talk 1: A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods
Speaker: Andre Milzarek
Abstract: In this talk, we discuss a novel analysis framework for non-descent-type optimization methodologies in nonconvex scenarios based on the Kurdyka-Lojasiewicz property. The proposed framework allows covering a broad class of algorithms, including those commonly employed in stochastic and distributed optimization. Specifically, it enables the asymptotic analysis of first-order methods that lack a sufficient descent property and do not require access to full (deterministic) gradient information. We leverage this framework to establish, for the first time, iterate convergence and corresponding rates of convergence for the decentralized gradient method and federated averaging under mild assumptions. Additional applications of the developed analysis techniques and new convergence results will be shown (e.g., for the random reshuffling and the stochastic gradient descent method) to illustrate generality of the approach.

Talk 2: Rockafellian Relaxation and Stochastic Optimization Under Perturbations
Speaker: Johannes Royset
Abstract: In practice, optimization models are often prone to unavoidable inaccuracies because of dubious assumptions and corrupted data. Traditionally, this placed special emphasis on risk-based and robust formulations, and their focus on “conservative” decisions. We develop, in contrast, an “optimistic” framework based on Rockafellian relaxations in which optimization is conducted not only over the original decision space but also jointly with a choice of model perturbation. The framework enables us to address challenging problems with ambiguous probability distributions from the areas of two-stage stochastic optimization without relatively complete recourse, probability functions lacking continuity properties, expectation constraints, and outlier analysis. We are also able to circumvent the fundamental difficulty in stochastic optimization that convergence of distributions fails to guarantee convergence of expectations. The framework centers on the novel concepts of exact and limit-exact Rockafellians, with interpretations of “negative” regularization emerging in certain settings. We illustrate the role of Phi-divergence, examine rates of convergence under changing distributions, and explore extensions to first-order optimality conditions. The main development is free of assumptions about convexity, smoothness, and even continuity of objective functions. Numerical results in the setting of computer vision and text analytics with label noise illustrate the framework.

Talk 3: Convergence analyses under computable models of nonsmoothness
Speaker: Vivak Patel
Abstract: In this talk, we discuss limitations of popular approaches to automatic differentiation for nonsmooth functions, and the issues that they can create for the reliability of gradient-based methods. We then discuss correct ways of computing differentials for nonsmooth functions, and we analyze the behavior of gradient-based methods under these corrected differentials.

Speakers
FR

Fred Roosta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AM

Andre Milzarek

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JR

Johannes Royset

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
VP

Vivak Patel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1I: Nonconvex Optimization and Applications
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Nonconvex Optimization and Applications
Chair: Bissan Ghaddar
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Solution framework for the facility location problem under endogenous road congestion
Speaker: Marten Soer
Abstract: We consider a bi-level optimization problem, where the upper-level decision maker chooses locations of facilities, by minimizing the total system travel time of the lower-level decision makers. While the lower-level decision makers, in turn, choose their fastest routes to the facilities. When the number of lowerlevel decision makers is large, their route choices affect the road congestion and thus the travel times. This setup is inspired by locating remote offices within the highly congested road network of Mexico City, but it is also applicable to other problems, such as the placement of electric vehicle charging stations and modeling electricity networks. To the best of our knowledge, the current literature focuses on finding a heuristic solution approach due to the complexity of the problem, and no approximation or exact solution framework for large instances exist. We study the properties of the problem and introduce an approximation algorithm that allows evaluating the quality of existing solutions and obtaining new close-to-optimal solutions. Moreover, we conduct numerical experiments in real-life instances to illustrate the applicability of the proposed endogenous road congestion solution framework and the effect when opening facility locations.

Talk 2: Finding long TSP tours in the unit square using the FICO Xpress global optimization solver
Speaker: Imre Polik
Abstract: We will investigate the problem of finding a set of points in the unit square such that the length of the optimal TSP tour over them is maximal. We will look at some interesting duality relations and provide nonconvex quadratic computational formulations. Additionally, we will look at related problems such as matching and the minimum pairwise distance. The theory will enable us to conduct computational experiments with the FICO Xpress global optimization MINLP solver and to find previously unknown optimal configurations. We will present some theoretical results and formulate several conjectures.

Talk 3: Facial Reduction for Semidefinite Relaxations of Combinatorial Optimization Problems
Speaker: Hao Hu
Abstract: In this talk, we present new findings on facial reduction for semidefinite relaxations of combinatorial optimization problems. In semidefinite programming (SDP), Slater’s condition is crucial for both theoretical convergence guarantees and the practical performance of optimization algorithms. When Slater’s condition fails, facial reduction can restore it through a finite sequence of reformulations. However, these reformulations often involve solving auxiliary optimization problems that can be as challenging as the original. Recent research has therefore focused on developing more efficient strategies for performing facial reduction. In our work, we specifically consider SDP problems that arise as relaxations of combinatorial optimization problems. This perspective enables us to exploit the underlying combinatorial structure, allowing the development of novel and highly efficient facial reduction techniques. We also establish theoretical results demonstrating the effectiveness of our approach. Numerical experiments further show that applying our specialized facial reduction method significantly improves both the speed and accuracy of solving SDP problems.

Speakers
MS

Marten Soer

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
IP

Imre Polik

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1J: Distributed Optimization and Learning
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Distributed Optimization and Learning
Chair: Shi Pu
Cluster: Multi-agent Optimization and Games

Talk 1: Variance-reduced accelerated methods for decentralized stochastic double-regularized nonconvex strongly-concave minimax problems
Speaker: Yangyang Xu
Abstract: In this talk, I will present an algorithmic framework for solving the decentralized, stochastic nonconvex strongly-concave (NCSC) minimax problem with nonsmooth regularization terms on both primal and dual variables, wherein a network of $m$ computing agents collaborate via peer-to-peer communications. We consider when the coupling function is in expectation or finite-sum form and the double regularizers are convex functions, applied separately to the primal and dual variables. Our algorithmic framework introduces a Lagrangian multiplier to eliminate the consensus constraint on the dual variable. Coupling this with variance-reduction (VR) techniques, our proposed method, entitled VRLM, by a single neighbor communication per iteration, is able to achieve an $\mathcal{O}(\kappa^3\varepsilon^{-3})$ sample complexity under the general stochastic setting, with either a big-batch or small-batch VR option, where $\kappa$ is the condition number of the problem and $\varepsilon$ is the desired solution accuracy. With a big-batch VR, we can additionally achieve $\mathcal{O}(\kappa^2\varepsilon^{-2})$ communication complexity. Under the special finite-sum setting, our method with a big-batch VR can achieve an $\mathcal{O}(n + \sqrt{n} \kappa^2\varepsilon^{-2})$ sample complexity and $\mathcal{O}(\kappa^2\varepsilon^{-2})$ communication complexity, where $n$ is the number of components in the finite sum. All complexity results match the best-known results achieved by a few existing methods for solving special cases of the problem we consider. To the best of our knowledge, this is the first work which provides convergence guarantees for NCSC minimax problems with general convex nonsmooth regularizers applied to both the primal and dual variables in the decentralized stochastic setting.

Talk 2: A Moreau Envelope Approach for LQR Meta-Policy Estimation
Speaker: César Uribe
Abstract: We study the problem of policy estimation for the Linear Quadratic Regulator (LQR) in discrete-time linear time-invariant uncertain dynamical systems. We propose a Moreau Envelope-based surrogate LQR cost, built from a finite set of realizations of the uncertain system, to define a meta-policy efficiently adjustable to new realizations. Moreover, we design an algorithm to find an approximate first-order stationary point of the meta-LQR cost function. Numerical results show that the proposed approach outperforms naive averaging of controllers on new realizations of the linear system. We also provide empirical evidence that our method has better sample complexity than Model-Agnostic Meta-Learning (MAML) approaches.

Talk 3: An Online Optimization Perspective on First-Order and Zero-Order Decentralized Nonsmooth Nonconvex Stochastic Optimization
Speaker: Emre Sahinoglu
Abstract: We investigate the finite-time analysis of finding \goldstat points for nonsmooth nonconvex objectives in decentralized stochastic optimization. A set of agents aim at minimizing a global function using only their local information by interacting over a network. We present a novel algorithm, called Multi Epoch Decentralized Online Learning (ME-DOL), for which we establish the sample complexity in various settings. First, using a recently proposed online-to-nonconvex technique, we show that our algorithm recovers the optimal convergence rate of smooth nonconvex objectives. We then extend our analysis to the nonsmooth setting, building on properties of randomized smoothing and Goldstein-subdifferential sets. We establish the sample complexity of $O(\delta^{-1}\epsilon^{-3})$, which to the best of our knowledge is the first finite-time guarantee for decentralized nonsmooth nonconvex stochastic optimization in the first-order setting (without weak-convexity), matching its optimal centralized counterpart. We further prove the same rate for the zero-order oracle setting without using variance reduction.

Speakers
YX

Yangyang Xu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for César Uribe

César Uribe

Assistant Professor, Electrical and Computer Engineering, Rice University
César A. Uribe received his BSc. in Electronic Engineering from Universidad de Antioquia in 2010. He then received an MSc. in Systems and Control from Delft University of Technology in the Netherlands in 2013. In 2016, be received an MSc. in Applied Mathematics from the University... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1K: Optimization for GenAI -- diffusion models and LLMs
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Optimization for GenAI -- diffusion models and LLMs
Chair: Wenpin Tang
Cluster: Optimization For Data Science

Talk 1: Gradient Guidance for Diffusion Models: An Optimization Perspective
Speaker: Minshuo Chen
Abstract: Diffusion models have demonstrated empirical successes in various applications and can be adapted to task-specific needs via guidance. This talk introduces a form of gradient guidance for adapting or fine-tuning diffusion models towards user-specified optimization objectives. We study the theoretic aspects of a guided score-based sampling process, linking the gradient-guided diffusion model to first-order optimization. We show that adding gradient guidance to the sampling process of a pre-trained diffusion model is essentially equivalent to solving a regularized optimization problem, where the regularization term acts as a prior determined by the pre-training data. We further consider an iteratively fine-tuned version of gradient-guided diffusion where one can query gradients at newly generated data points and update the score network using new samples. This process mimics a first-order optimization iteration in expectation, for which we prove O(1/K) convergence rate to the global optimum when the objective function is concave.

Talk 2: RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization
Speaker: Hanyang Zhao
Abstract: Recently, numerous preference optimization algorithms have been introduced as extensions to the Direct Preference Optimization (DPO) family. While these methods have successfully aligned models with human preferences, there is a lack of understanding regarding the contributions of their additional components. Moreover, fair and consistent comparisons are scarce, making it difficult to discern which components genuinely enhance downstream performance. In this work, we propose RainbowPO, a unified framework that demystifies the effectiveness of existing DPO methods by categorizing their key components into seven broad directions. We integrate these components into a single cohesive objective, enhancing the performance of each individual element. Through extensive experiments, we demonstrate that RainbowPO outperforms existing DPO variants. Additionally, we provide insights to guide researchers in developing new DPO methods and assist practitioners in their implementations.

Talk 3: A preliminary study on the generation process of diffusion models with different noise distributions
Speaker: Nanshan Jia
Abstract: We propose a class of structured diffusion models, in which the prior distribution is chosen as a mixture of Gaussians, rather than a standard Gaussian distribution. The specific mixed Gaussian distribution, as prior, can be chosen to incorporate certain structured information of the data. We develop a simple-to-implement training procedure that smoothly accommodates the use of mixed Gaussian as prior. Theory is provided to quantify the benefits of our proposed models, compared to the classical diffusion models. Numerical experiments with synthetic, image and operational data are conducted to show comparative advantages of our model. Our method is shown to be robust to mis-specifications and in particular suits situations where training resources are limited or faster training in real time is desired.

Speakers
WT

Wenpin Tang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MC

Minshuo Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HZ

Hanyang Zhao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
NJ

Nanshan Jia

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1L: Applications of polynomial optimization to data analysis I
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Applications of polynomial optimization to data analysis I
Chair: Victor Magron
Cluster: Conic and Semidefinite Optimization

Talk 1: Learning polynomial Lyapunov functions from fixed data
Speaker: Oumayma Khattabi
Abstract: Stability analysis, a cornerstone of control theory, usually relies on an explicit system model. Recently, a surge of data-driven control methods occurred, due to data abundance, high compu- tational power and model shortcomings (inaccuracies, model complexity...). This has given cause to new frameworks relying less on modelling and more on data. In particular, optimization-based frameworks, that minimize generalization errors, play a key role in this area of research, see e.g. Kernel Predictive Control (KPC) and Data-enabled Predictive Control (DeePC). More specifically, polynomial optimization and sum-of-squares (SoS) programming found an important application in this field, in a recent contribution by T. Martin and F. Allg¨ower, relying on polynomial repre- sentation of model-free nonlinear systems, through a set-membership characterization for Taylor polynomials derived from a noisy dataset. For better accuracy, multiple Taylor polynomials are combined into a piecewise polynomial representation, which enhances system property inference and allows for the verification of dissipativity properties. In this context, we propose a novel data- driven methodology to compute certified inner approximations of a region of attraction, for an equilibrium point associated to a dynamical system with unknown model. The approach requires an input-output dataset and information on the variations of the dynamics (e.g. Lipschitz bounds), and returns a Lyapunov function, valid for any dynamical system that matches the dataset and bounds given. To perform this task, the (compact) admissible state space is first partitioned into an appropriate tessellation, after which a polynomial Lyapunov candidate is assigned to each of the resulting cells. The Lyapunov condition is enforced on each cell, and complemented with boundary conditions enforcing continuity of the resulting global, piecewise polynomial Lyapunov candidate. A key contribution is that the Lyapunov condition is split in learning subproblems, following the observation that the more datapoints, the more difficult to analyze the uncertainty set for the ground truth. The whole learning problem can be recast under the form of an SoS programming problem, resulting in semidefinite programming problems (SDP) of increasing size. Interestingly, thanks to our data-wise splitting, the special case of degree one, i.e. piecewise-affine Lyapunov candidates, can be relaxed into a second order cone programming problem (SOCP) while main- taining convergence guarantees, resulting in much faster computations than the higher degree SDP formulations. Another key contribution of this work, for higher degrees of the polynomial, is the inclusion of Lagrangian duality, which hasn’t figured in previous works in data-driven SoS program- ming for dynamical systems. This approach opens the door to a probabilistic interpretation of the methodology.

Talk 2: A sparsified Christoffel function for high-dimensional inference
Speaker: Lucas Slot
Abstract: Christoffel polynomials are classical tools from approximation theory. They can be used to estimate the (compact) support of a measure based on its low-degree moments. Recently, they have been applied to problems in data science, including outlier detection and support inference. A major downside of Christoffel polynomials in such applications is the fact that, in order to compute their coefficients, one must invert a matrix whose size grows rapidly with the dimension. In this talk, we propose a modification of the Christoffel polynomial which is significantly cheaper to compute, but retains many of its desirable properties. Our approach relies on sparsity of the underlying measure, described by a graphical model. The complexity of our modification depends on the treewidth of this model. Based on joint work with Jean-Bernard Lasserre.

Talk 3: Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization
Speaker: Srećko Ðurašinović
Abstract: In this talk, we explore methods for verifying the properties of Binary Neural Networks (BNNs), focusing on robustness against adversarial attacks. Despite their lower computational and memory needs, BNNs, like their full-precision counterparts, are also sensitive to input perturbations. Established methods for solving this problem are predominantly based on Satisfiability Modulo Theories and Mixed-Integer Linear Programming techniques, which often face scalability issues. We introduce an alternative approach using Semidefinite Programming relaxations derived from sparse Polynomial Optimization. Our approach, compatible with continuous input space, not only mitigates numerical issues associated with floating-point calculations but also enhances verification scalability through the strategic use of tighter first-order semidefinite relaxations. We demonstrate the effectiveness of our method in verifying robustness against both infinity-norm and L2-norm based adversarial attacks.

Speakers
avatar for Oumayma Khattabi

Oumayma Khattabi

PhD student, Paris-Saclay University
I am currently working on stability analysis of dynamic systems using data.
LS

Lucas Slot

Postdoc, ETH Zurich
avatar for Srećko Ðurašinović

Srećko Ðurašinović

PhD Student, Nanyang Technological University, Singapore
Areas of interest:- Sparse Polynomial Optimization- Neural Network Verification- Christoffel-Darboux Kernels
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1M: Optimization Method Generation with Large Language Models
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Optimization Method Generation with Large Language Models
Chair: Xiangfeng Wang
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Generative Models in Reinforcement Learning
Speaker: Wenhao Li
Abstract: ~

Talk 2: LLM-based Simulation Optimization
Speaker: Jun Luo
Abstract: ~

Talk 3: LLM-based Optimization Method for Scheduling
Speaker: Xiangfeng Wang
Abstract: ~

Speakers
WL

Wenhao Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jun Luo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XW

Xiangfeng Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1N: Optimization for Robotics I
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Optimization for Robotics I
Chair: Panos Patrinos
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Robotics Applications of the Direct Optimal Control of Nonsmooth Systems
Speaker: Anton Pozharskiy
Abstract: When developing control algorithms for robotic systems, practitioners must contend with modeling complex environmental interactions, including contact and friction, which are often modeled as nonsmooth dynamical systems. We discuss several alternate models of varying fidelity that can be applied to robotic manipulation problems, particularly comparing those coming from complementarity-lagrangian models and those coming from simpler projected dynamical systems. In order to efficiently be used in an optimal control context, these systems must be accurately discretized, as naive discretizations result in low accuracy and incorrect sensitivities. To this end, the Finite Elements with Switch Detection (FESD) discretization can be applied, which results in nonsmooth optimization problems called Mathematical Programs with Complementarity Constraints (MPCCs). The theoretical and practical difficulties of solving MPCCs coming from optimal control and several solution methods are then described. Finally, we present the open source package nosnoc, in which both the discretization and MPCC solution methods are implemented.

Talk 2: Real-time constrained nonlinear MPC in robotics: augmented Lagrangians and fast block-sparse matrix factorizations
Speaker: Wilson Jallet
Abstract: In high-dimensional robotic platforms, such as legged robots and humanoids, achieving real-time control is a critical challenge, particularly when managing complex dynamics and constraints in nonlinear model predictive control (MPC). This talk presents recent advances in constrained nonlinear MPC, focusing on augmented Lagrangian methods and fast block-sparse matrix factorizations. By exploiting the block-banded structure arising from the time dependency in MPC, we extend the Riccati recursion to efficiently handle constraints. Additionally, a Schur complement-like approach enables parallelization, significantly accelerating computation. We also discuss ongoing developments in a flexible C++ library, open-sourced last year, designed for real-time robotic applications. Current work emphasizes performance optimization, including updating OCPs and warm-starting MPC, improvements to cache-friendliness and future work on a computation graph. Flexibility remains a key focus, enabling users to define dynamics from their own ODE or DAE models (such as those provided by the Pinocchio rigid-body dynamics library), with support for a variety of time integrators, such as Euler and Runge-Kutta (with potential support for more advanced, energy-conserving integrators in the future). Additionally, we explore the use of generalized augmented Lagrangian methods, which allow geometric handling of more complex constraint sets, further enhancing the library's capabilities for constrained optimization. These advancements aim to make real-time control in complex robotic systems, particularly humanoids, more efficient and adaptable.

Talk 3: High-performance linear algebra in quadratic programming solvers for real-time optimal control
Speaker: Pieter Pas
Abstract: Model predictive control (MPC) is a powerful control strategy that is widely used in robotics due to its excellent performance and the ability to handle constraints. However, the real-time implementation of MPC presents significant computational challenges, especially in high-speed or large-scale control applications. Efficient numerical optimization solvers are therefore essential, and remain an active area of research. Solvers based on quadratic programming and interior point methods both rely on the fast solution of linear systems with a particular KKT structure. In this talk, we explore how the specific block-wise structure of KKT systems that arise in optimal control problems can be exploited in specialized batched linear algebra routines. By employing tailored storage schemes and highly optimized micro-kernels, combined with advanced vectorization and parallelization techniques, these routines leverage the full power of modern hardware, even for small to moderately sized models. We conclude by demonstrating that the practical performance of the quadratic programming solver QPALM can be substantially improved by replacing its general-purpose linear solver with optimal-control-specific variants based on the aforementioned batched linear algebra routines. The resulting QPALM-OCP solver is released as an open-source software library.

Speakers
AP

Anton Pozharskiy

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WJ

Wilson Jallet

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PP

Pieter Pas

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1O: Optimization For Data Science
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Optimization For Data Science
Chair: Niao He
Cluster: Optimization For Data Science

Talk 1: Momentum & stochasticity — some insights from continuous time models
Speaker: Stephan Wojtowytsch
Abstract: Gradient descent and its variations (stochastic, with or without momentum) are the workhorse of machine learning. We give some examples where we gain insight into high-dimensional optimization problems in a machine learning context from continuous time models and possible effects of large step sizes compared to the continuous dynamics.

Talk 2: On a continuous time model of gradient descent dynamics and instability in deep learning
Speaker: Mihaela Rosca
Abstract: The recipe behind the success of deep learning has been the combination of neural networks and gradient-based optimization. Understanding the behavior of gradient descent however, and particularly its instability, has lagged behind its empirical success. To add to the theoretical tools available to study gradient descent we propose the principal flow (PF), a continuous time flow that approximates gradient descent dynamics. To our knowledge, the PF is the only continuous flow that captures the divergent and oscillatory behaviors of gradient descent, including escaping local minima and saddle points. Through its dependence on the eigendecomposition of the Hessian the PF sheds light on the recently observed edge of stability phenomena in deep learning. Using our new understanding of instability we propose a learning rate adaptation method which enables us to control the trade-off between training stability and test set evaluation performance.

Talk 3: A Hessian-Aware Stochastic Differential Equation for Modelling SGD
Speaker: Zebang Shen
Abstract: Continuous-time approximation of Stochastic Gradient Descent (SGD) is a crucial tool to study its escaping behaviors from stationary points. However, existing stochastic differential equation (SDE) models fail to fully capture these behaviors, even for simple quadratic objectives. Built on a novel stochastic backward error analysis framework, we derive the Hessian-Aware Stochastic Modified Equation (HA-SME), an SDE that incorporates Hessian information of the objective function into both its drift and diffusion terms. Our analysis shows that HA-SME matches the order-best approximation error guarantee among existing SDE models in the literature, while achieving a significantly reduced dependence on the smoothness parameter of the objective. Further, for quadratic objectives, under mild conditions, HA-SME is proved to be the first SDE model that recovers exactly the SGD dynamics in the distributional sense. Consequently, when the local landscape near a stationary point can be approximated by quadratics, HA-SME is expected to accurately predict the local escaping behaviors of SGD.

Speakers
avatar for Stephan Wojtowytsch

Stephan Wojtowytsch

Name: Stephan WojtowytschAffiliation: University of PittsburghBio: I am an assistant professor in the Department of Mathematics at the University of Pittsburgh. My research interests lie in the mathematics of machine learning and data science. Previously, I was an assistant profe... Read More →
MR

Mihaela Rosca

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZS

Zebang Shen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1P: Advances in large-scale nonlinear optimization for data science
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Advances in large-scale nonlinear optimization for data science
Chair: Jiawei Zhang
Cluster: Nonlinear Optimization

Talk 1: On Squared-Variable Formulations for Nonlinear Semidefinite Programming
Speaker: Lijun Ding
Abstract: We study squared-variable formulations for nonlinear semidefinite programming. We show an equivalence result of second-order stationary points of the nonsymmetric-squared-variable formulations and the nonlinear semidefinite programs. We also show that such an equivalence fails for the local minimizers and second-order stationary points of the symmetric-squared-variable formulations and the nonlinear semidefinite programs, correcting a false understanding in the literature and providing sufficient conditions for such a correspondence to hold.

Talk 2: High-probability complexity guarantees for nonconvex minimax problems
Speaker: Yasa Syed
Abstract: Stochastic smooth nonconvex minimax problems are prevalent in machine learning, e.g., GAN training, fair classification, and distributionally robust learning. Stochastic gradient descent ascent (GDA)-type methods are popular in practice due to their simplicity and single-loop nature. However, there is a significant gap between the theory and practice regarding high-probability complexity guarantees for these methods on stochastic nonconvex minimax problems. Existing high-probability bounds for GDA-type single-loop methods only apply to convex/concave minimax problems and to particular non-monotone variational inequality problems under some restrictive assumptions. In this work, we address this gap by providing the first high-probability complexity guarantees for nonconvex/PL minimax problems corresponding to a smooth function that satisfies the PL-condition in the dual variable. Specifically, we show that when the stochastic gradients are light-tailed, the smoothed alternating GDA method can compute an $\varepsilon$-stationary point within $\mathcal{O}(\frac{\ell \kappa^2 \delta^2}{\varepsilon^4} + \frac{\kappa}{\varepsilon^2}(\ell+\delta^2\log({1}/{\bq})))$ stochastic gradient calls with probability at least $1-\bq$ for any $\bq\in(0,1)$, where $\mu$ is the PL constant, $\ell$ is the Lipschitz constant of the gradient, $\kappa=\ell/\mu$ is the condition number, and $\delta^2$ denotes a bound on the variance of stochastic gradients. We also present numerical results on a nonconvex/PL problem with synthetic data and on distributionally robust optimization problems with real data, illustrating our theoretical findings.

Talk 3: Sparse Solutions to Linear Systems via Polyak’s Stepsize
Speaker: Yura Malitsky
Abstract: This talk explores the implicit bias of entropic mirror descent in finding sparse solutions to linear systems, emphasizing the importance of appropriate initialization. We present an adaptive approach to improving the algorithm, using Polyak's stepsizes as a key tool.

Speakers
JZ

Jiawei Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LD

Lijun Ding

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YS

Yasa Syed

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YM

Yura Malitsky

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1Q: Optimization with Approximate or Uncertain Models
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Optimization with Approximate or Uncertain Models
Chair: Matthias Heinkenschloss
Cluster: PDE-constrained Optimization

Talk 1: Preconditioned Pseudo-time Continuation for Parameterized Inverse Problems
Speaker: Bart van Bloemen Waanders
Abstract: In this presentation we discuss a continuation approach to overcome linearization limitations associated with evaluating the post-optimality sensitivity of uncertainties with respect to optimization solutions. The post-optimality sensitivities (sensitivities of the first order optimality condition)  arise from the Implicit Function Theorem and depend on second-order derivatives. If the magnitude of uncertainty is large, the post-optimality sensitivities are insufficient to predict the effects of perturbed uncertainty parameters on the optimization solution. To address this issue, we introduce a continuation process that uses a pseudo time-stepping scheme to evolve the sensitivities.  A combination of specialized time-discretization and preconditioning helps to accelerate convergence.  A key computational challenge is the calculation of the inverse Hessian as part of the post-optimality sensitivity evaluation at each iteration of the continuation process.  To that end, we use a preconditioned Conjugate Gradient (PCG) solution strategy in which two novel Quasi-Newton update schemes are implemented that exploit the pseudo-time continuation structure. Our first update scheme introduces a secant equation to captures the uncertainty variations.  The second is an adaption of the block BFGS methods that leverages the PCG iteration history.  We demonstrate our approach on an insightful yet simple Poisson PDE with nonlinear boundary conditions and a nonlinear forcing term that in turn embeds uncertainty.   We invert for a spatially distributed diffusion coefficient and demonstrate the efficacy of our time-stepping and preconditioning algorithms.

Talk 2: Shape and topology optimization under uncertainty by robust approaches with application to electric machines
Speaker: Stefan Ulbrich
Abstract: We consider shape and topology optimization for PDE-constrained problems, where parameters in the PDE (e.g. coefficients) as well as the design itself (e.g. manufacturing tolerances) are uncertain. We propose a robust optimization approach, where the usually nonsmooth maximum value functions of constraints and objective function on the uncertainty sets are used in the robust counterpart. We discuss the efficient calculation of generalized derivatives of the robustified objective function and constraints. In particular, we introduce a novel robust topological derivative that can be used for robust topology optimization. We apply the methodology to shape and topology optimization of electric machines.

Talk 3: Adaptive Surrogate Modeling for Trajectory Optimization with Model Inexactness
Speaker: Matthias Heinkenschloss
Abstract: In many applications, one must compute optimal trajectories from imperfect knowledge of the dynamics. For example, solving trajectory optimization problems for hypersonic vehicles requires computing lift and drag coefficients at many flight configurations. Determining these coefficients over the entire state space would require expensive high-fidelity computations using detailed representations of the hypersonic vehicle at prohibitively many samples. This talk proposes using computationally inexpensive adaptive kernel regression models constructed from high-fidelity samples to approximate the components of the dynamics that are expensive to evaluate. To reduce the effect of model errors on the optimal trajectory, the current kernel regression model is updated as needed at the cost of evaluating the components of the dynamics at a small number of additional sample points. First, the optimal control problem is solved using the current kernel model to represent the dynamics. Next, a new optimality sensitivity analysis is combined with error estimates of the kernel model to determine whether the kernel regression model needs to be updated and, if so, at which samples the dynamics should be evaluated to update it. This talk outlines our current model refinement procedure and demonstrates its performance on a trajectory optimization problem for a hypersonic vehicle with lift and drag models that are known but expensive to evaluate.

Speakers
BV

Bart van Bloemen Waanders

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SU

Stefan Ulbrich

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MH

Matthias Heinkenschloss

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1R: Advances in Riemannian Optimization: Algorithms and Applications
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Advances in Riemannian Optimization: Algorithms and Applications
Chair: Andi Han
Cluster: Optimization on Manifolds

Talk 1: Riemannian Accelerated Zeroth-order Algorithm: Improved Robustness and Lower Query Complexity
Speaker: Chang He
Abstract: Optimization problems with access to only zeroth-order information of the objective function on Riemannian manifolds arise in various applications, spanning from statistical learning to robot learning. While various zeroth-order algorithms have been proposed in Euclidean space, they are not inherently designed to handle the challenging constraints imposed by Riemannian manifolds. The proper adaptation of zeroth-order techniques to Riemannian manifolds remained unknown until the pioneering work of (Li et al., 2023a). However, zeroth-order algorithms are widely observed to converge slowly and be unstable in practice. To alleviate these issues, we propose a Riemannian accelerated zeroth-order algorithm with improved robustness. Regarding efficiency, our accelerated algorithm has the function query complexity of $\mathcal{O}(\epsilon^{-7/4}d)$ for finding an $\epsilon$-approximate first-order stationary point. By introducing a small perturbation, it exhibits a function query complexity of $\tilde{\mathcal{O}}(\epsilon^{-7/4}d)$ for seeking a second-order stationary point with a high probability, matching state-of-the-art result in Euclidean space. Moreover, we further establish the almost sure convergence in the asymptotic sense through the Stable Manifold Theorem. Regarding robustness, our algorithm requires larger smoothing parameters in the order of $\tilde{\mathcal{O}}(\epsilon^{7/8}d^{-1/2})$, improving the existing result by a factor of $\tilde{\mathcal{O}}(\epsilon^{3/4})$.

Talk 2: Extragradient Type Methods for Riemannian Variational Inequality Problems
Speaker: Zihao Hu
Abstract: In this work, we consider monotone Riemannian Variational Inequality Problems (RVIPs), which encompass both Riemannian convex optimization and minimax optimization as particular cases. In Euclidean space, the last-iterates of both the extragradient (EG) and past extragradient (PEG) methods converge to the solution of monotone variational inequality problems at a rate of $O\left(\frac{1}{\sqrt{T}}\right)$ \citep{cai2022finite}. However, analogous behavior on Riemannian manifolds remains open. To bridge this gap, we introduce the Riemannian extragradient (REG) and Riemannian past extragradient (RPEG) methods. We show that both exhibit $O\left(\frac{1}{\sqrt{T}}\right)$ last-iterate convergence and $O\left(\frac{1}{{T}}\right)$ average-iterate convergence, aligning with observations in the Euclidean case. These results are enabled by judiciously addressing the holonomy effect so that additional complications in Riemannian cases can be reduced and the Euclidean proof inspired by the performance estimation problem (PEP) technique can be applied again.

Talk 3: Riemannian ADMM
Speaker: Jiaxiang Li
Abstract: We consider a class of Riemannian optimization problems where the objective is the sum of a smooth function and a nonsmooth function, considered in the ambient space. This class of problems finds important applications in machine learning and statistics such as the sparse principal component analysis, sparse spectral clustering, and orthogonal dictionary learning. We propose a Riemannian alternating direction method of multipliers (ADMM) to solve this class of problems. Our algorithm adopts easily computable steps in each iteration. The iteration complexity of the proposed algorithm for obtaining an $\epsilon$-stationary point is analyzed under mild assumptions. Existing ADMM for solving nonconvex problems either does not allow nonconvex constraint set, or does not allow nonsmooth objective function. In contrast, our complexity result is established for problems with simultaneous nonsmooth objective and manifold constraint. Numerical experiments are conducted to demonstrate the advantage of the proposed method.

Speakers
AH

Andi Han

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CH

Chang He

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZH

Zihao Hu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jiaxiang Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1S: Advances in Semidefinite Programming: Relaxations, Hierarchies, and Optimization Techniques
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Advances in Semidefinite Programming: Relaxations, Hierarchies, and Optimization Techniques
Chair: Makoto Yamashita & Sunyoung Kim
Cluster: Conic and Semidefinite Optimization

Talk 1: Well-conditioned primal-dual interior-point method for accurate low-rank semidefinite programming
Speaker: Hong-Ming Chiu
Abstract: We describe how the low-rank structure in an SDP can be exploited to reduce the per-iteration cost of a convex primal-dual interior-point method down to cubic time and quadratic memory, even at very high accuracies. A traditional difficulty is the dense Newton subproblem at each iteration, which becomes progressively ill-conditioned as progress is made towards the solution. Preconditioners have previously been proposed to improve conditioning, but these can be expensive to set up, and become ineffective as the preconditioner itself becomes increasingly ill-conditioned at high accuracies. Instead, we present a well-conditioned reformulation of the Newton subproblem that is cheap to set up, and whose condition number is guaranteed to remain bounded over all iterations. In theory, applying an inner iterative method to the reformulation reduces the per-iteration cost of the outer interior-point method to cubic time and quadratic memory. We also present a well-conditioned preconditioner that greatly improves the convergence of the inner iterations.

Talk 2: Exact SDP relaxations for a class of quadratic programs with finite and infinite quadratic constraints
Speaker: Sunyoung Kim
Abstract: We study exact semidefinite programming (SDP) relaxations for the problem of minimizing a nonconvex quadratic objective function over a feasible region defined by both finitely and infinitely many nonconvex quadratic inequality constraints (semi- infinite QCQPs). Specifically, we present two sufficient conditions on the feasible region under which the QCQP, with any quadratic objective function over the feasible region, is equivalent to its SDP relaxation. The first condition is an extension of a result recently proposed by the authors (arXiv:2308.05922, to appear in SIAM J. Optim.) from finitely constrained quadratic programs to semi-infinite QCQPs. The newly introduced second condition offers a clear geometric characterization of the feasible region for a broad class of QCQPs that are equivalent to their SDP relaxations. Several illustrative examples, including quadratic programs with ball-, parabola-, and hyperbola-based constraints, are also provided.

Talk 3: Semidefinite Programming Relaxation Hierarchy Using Third-Order Tensors for Constrained Polynomial Optimization
Speaker: Makoto Yamashita
Abstract: Exploiting the computational structure of third-order tensors, Zheng et al. (2022) proposed a semidefinite programming (SDP) hierarchy of relaxations for unconstrained polynomial optimization problems (POPs). We extend this by employing the Lagrange function to propose a hierarchy of SDP relaxation for constrained polynomial optimization problems involving third-order tensors. This relaxation can be computationally efficient, as it can be transformed into an SDP problem with a block diagonal matrix structure via the discrete Fourier transformation. Additionally, we show under a mild assumption, the objective value of the hierarchy converges to the optimal value of the POP as the degree of relaxation increases.

Speakers
HC

Hong-Ming Chiu

PhD student, University of Illinois Urbana Champaign
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SK

Sunyoung Kim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Makoto Yamashita

Makoto Yamashita

Professor, Institute of Science Tokyo
Name: Dr. Makoto YamashitaTitle: ProfessorAffiliation: Institute of Science TokyoBio:Dr. Makoto Yamashita is a professor of Department of Mathematical and Computing Science of the Institute of Science Tokyo.His recent research interests includes conic optimization and its applications... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1T: Recent Advances in Stochastic Optimization: Complexity, Adaptivity, and Nonsmooth Extensions (I)
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Recent Advances in Stochastic Optimization: Complexity, Adaptivity, and Nonsmooth Extensions (I)
Chair: Sen Na & Zhaosong Lu
Cluster: Nonlinear Optimization

Talk 1: Adaptive Optimization with Highly Corrupted Inputs: A Unified Framework for High-Probability Iteration Complexity Analysis
Speaker: Miaolan Xie
Abstract: We consider an unconstrained continuous optimization problem in which gradient estimates may be arbitrarily corrupted in each iteration with a probability greater than $\frac 1 2$. Additionally, function value estimates may be noisy or adversarially corrupted throughout the algorithm’s execution. This framework is applicable to many real-world problems and is particularly relevant to stochastic and derivative-free optimization settings. We introduce an algorithmic and analytical framework that provides high probability bounds on iteration complexity for this highly corrupted setting. The analysis offers a unified approach, accommodating noisy or corrupted inputs and encompassing methods such as line search and trust region.

Talk 2: Adaptive Stochastic Algorithms for Nonconvex Constrained Optimization
Speaker: Baoyu Zhou
Abstract: In this talk, we will discuss some recent works on the design, analysis, and implementation of a class of efficient algorithms for solving stochastic optimization problems with deterministic nonlinear nonconvex constraints. Those optimization problems arise in a plethora of science and engineering applications including physics-informed learning, PDE-constrained optimization, machine learning fairness, and optimal power flow. We are especially interested in the case where the problem's feasible region is difficult to detect and projection-type methods are intractable. The theoretical results and numerical performance demonstrate the efficiency and efficacy of our proposed algorithms.

Talk 3: Variance-Reduced First-Order Methods for Constrained Stochastic Optimization
Speaker: Zhaosong Lu
Abstract: We study a class of deterministically constrained stochastic optimization problems. Existing methods typically aim to find an approximate stochastic stationary point, where the expected violations of both the constraints and first-order stationarity are nearly satisfied. However, such approximate solutions can lead to significant constraint violations. To address this issue, we propose single-loop variance-reduced stochastic first-order methods. In our approach, the stochastic gradient of the stochastic component is computed using either a truncated recursive scheme or a truncated Polyak momentum scheme for variance reduction, while the gradient of the deterministic component is computed exactly. Under suitable assumptions, our proposed methods not only achieve new sample and first-order operation complexity but also produce stronger approximate stochastic stationary points that more reliably satisfy the constraints compared to existing methods.

Speakers
avatar for Miaolan Xie

Miaolan Xie

Assistant professor, Purdue University
Name: Miaolan XieTitle: Assistant Professor of Stochastic Optimization and Continuous OptimizationAffiliation: Purdue University
BZ

Baoyu Zhou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZL

Zhaosong Lu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1U: Recent Advances in Fixed-Point Methods and Stability in Optimization Problems
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Recent Advances in Fixed-Point Methods and Stability in Optimization Problems
Chair: Chao Ding
Cluster: Fixed Points and Variational Inequalities

Talk 1: HPR-QP: An implementation of an HPR method for solving quadratic programming
Speaker: Kaihuang Chen
Abstract: This talk introduces HPR-QP, a solver based on a Halpern Peaceman-Rachford (HPR) method for solving quadratic programming (QP). The iteration complexity of the algorithm is $O(1/k)$ in terms of the Karush-Kuhn-Tucker residual and the objective error. We compare the performance of HPR-QP and other solvers on extensive QP datasets.

Talk 2: Characterizations of Tilt-Stable Local Minimizers of a Class of Matrix Optimization Problems
Speaker: Shiwei Wang
Abstract: As a fundamental perturbation property, tilt stability has been widely studied as it can deeply characterize the difficulty of a problem and reveal the good behavior of multiplier methods for certain problem. In this talk, we mainly focus on establishing a new characterization of tilt stability via the newly proposed quadratic bundle. By calculating the explicit form of the minimal quadratic bundle of the polyhedral spectral function, we can further obtain the equivalence between tilt stability and strong second order sufficient condition under nondegeneracy for general composite optimization problem.

Talk 3: On the ergodic convergence properties of the Peaceman-Rachford method and their applications in solving linear programming
Speaker: Guojun Zhang
Abstract: In this talk, we study the ergodic convergence properties of the Peaceman-Rachford (PR) method with semi-proximal terms for solving convex optimization problems (COPs). For the first time, we establish the global convergence of the ergodic sequence of the PR method with semi-proximal terms by leveraging the theory of the degenerate proximal point method. This result represents a significant departure from previous studies on the non-ergodic convergence of the PR method, which typically require strong convexity or monotonicity conditions that are not generally satisfied in COPs. Moreover, we demonstrate an ergodic iteration complexity of $O(1/k)$ of the PR method with semi-proximal terms, measured by the objective error and the Karush–Kuhn–Tucker residual using the $\varepsilon$-subdifferential. Based on these convergence properties, we introduce EPR-LP, using the ergodic sequence of the PR method with semi-proximal terms for solving linear programming (LP) problems. EPR-LP incorporates an adaptive restart strategy and dynamic penalty parameter updates for efficiency and robustness. Extensive numerical experiments on LP benchmark datasets, executed on a high-performance GPU, show that our Julia-based solver outperforms the award-winning solver PDLP at a tolerance level of $10^{-8}$.

Speakers
CD

Chao Ding

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KC

Kaihuang Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SW

Shiwei Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GZ

Guojun Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1V: Stochastic Gradient Descent (SGD) and Variants
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Stochastic Gradient Descent (SGD) and Variants
Chair: Melinda Hagedorn
Cluster: nan

Talk 1: Optimized convergence of stochastic gradient descent by weighted averaging
Speaker: Melinda Hagedorn
Abstract: Under mild assumptions stochastic gradient methods asymptotically achieve an optimal rate of convergence if the arithmetic mean of all iterates is returned as an approximate optimal solution. However, in the absence of stochastic noise, the arithmetic mean of all iterates converges considerably slower to the optimal solution than the iterates themselves. And also in the presence of noise, when a termination of the stochastic gradient method after a finite number of steps is considered, the arithmetic mean is not necessarily the best possible approximation to the unknown optimal solution. This paper aims at identifying optimal strategies in a particularly simple case, the minimization of a strongly convex function with i.i.d. noise terms and termination after a finite number of steps. Explicit formulas for the stochastic error and the optimality error are derived in dependence of certain parameters of the SGD method. The aim was to choose parameters such that both stochastic error and optimality error are reduced compared to arithmetic averaging. This aim could not be achieved; however, by allowing a slight increase of the stochastic error it was possible to select the parameters such that a significant reduction of the optimality error could be achieved. This reduction of the optimality error has a strong effect on the approximate solution generated by the stochastic gradient method in case that only a moderate number of iterations is used or when the initial error is large. The numerical examples confirm the theoretical results and suggest that a generalization to non-quadratic objective functions may be possible. This paper was written together with Professor Florian Jarre and is already published in Optimization Methods and Software 39 (2024), Nr. 4, S. 699–724

Talk 2: Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under (L0, L1)-Smoothness
Speaker: Aleksandr Lobanov
Abstract: The gradient descent (GD) method -- is a fundamental and likely the most popular optimization algorithm in machine learning (ML), with a history traced back to a paper in 1847 \cite{Cauchy_1847}. In this paper, we provide an improved convergence analysis of gradient descent and its variants, assuming generalized smoothness $(L_0,L_1)$. In particular, we show that GD has the following behavior of convergence in the \textit{convex setup}: as long as $\norms{\nabla f(x^k)} \geq \frac{L_0}{L_1}$ the algorithm has \textit{linear convergence}, and approaching the solution $x^*$ such that $\norms{\nabla f(x^k)} < \frac{L_0}{L_1}$, has standard sublinear rate. Moreover, we show that this behavior of convergence is also common for its variants using different types of oracle: \textit{Normalized Gradient Descent} as well as \textit{Clipped Gradient Descent} (the case when the oracle has access to the full gradient $\nabla f(x)$); \textit{Random Coordinate Descent} (when the oracle has access only to the gradient component $\nabla_{i} f(x)$); \textit{Random Coordinate Descent with Order Oracle} (when the oracle has access only to the comparison value of the objective function $\text{sign} [f(y) - f(x)]$). In addition, we also analyze the behavior of convergence rate of GD algorithm in a strongly convex setup. Finally, we validate our theoretical results via numerical experiment. https://arxiv.org/pdf/2412.17050

Talk 3: High Probability Guarantees for Random Reshuffling
Speaker: Hengxu Yu
Abstract: We consider the stochastic gradient method with random reshuffling (RR) for tackling smooth nonconvex optimization problems. RR finds broad applications in practice, notably in training neural networks. In this work, we provide high probability first-order and second-order complexity guarantees for this method. First, we establish a high probability first-order sample complexity result for driving the Euclidean norm of the gradient (without taking expectation) below a required accuracy. Our derived complexity matches the best existing in-expectation one up to a logarithmic term while imposing no additional assumptions nor changing RR's updating rule. We then propose a simple and computable stopping criterion for RR (denoted as RR-sc). This criterion is guaranteed to be triggered after a finite number of iterations, enabling us to prove a high probability first-order complexity guarantee for the last iterate. Second, building on the proposed stopping criterion, we design a perturbed random reshuffling method (p-RR) that involves an additional randomized perturbation procedure near stationary points. We derive that p-RR provably escapes strict saddle points and establish a high probability second-order complexity result, without requiring any sub-Gaussian tail-type assumptions on the stochastic gradient errors. The fundamental ingredient in deriving the aforementioned results is a new concentration property for sampling without replacement in RR, which could be of independent interest. Finally, we conduct numerical experiments on neural network training to support our theoretical findings. The full preprint paper, which is under revision for SIOPT, can be found at https://arxiv.org/abs/2311.11841

Speakers
avatar for Melinda Hagedorn

Melinda Hagedorn

PhD student, Heinrich Heine University Düsseldorf
Name: Melinda HagedornDegrees: Master's degrees in Mathematics and PhysicsAffiliation: Heinrich Heine University Düsseldorf, GermanyMelinda Hagedorn is a PhD student in Mathematical Optimization under the supervision of Prof. Florian Jarre, research associate and teaching assistant... Read More →
AL

Aleksandr Lobanov

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HY

Hengxu Yu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1W: Adaptive and Accelerated First-Order Methods
Monday July 21, 2025 10:30am - 11:45am PDT
Session: Adaptive and Accelerated First-Order Methods
Chair: Wenzhi Gao
Cluster: nan

Talk 1: Gradient Descent as a Collaborative Game
Speaker: Wenzhi Gao
Abstract: We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to update the stepsize in gradient descent with online learning and provably accelerates gradient-based methods. A key insight is to view gradient descent as a collaborative game between the stepsize scheduler and the optimization landscape -- both players working together for faster convergence. We also discuss implications of the framework, including global and local convergence properties and several extensions. Numerical experiments on deterministic convex and nonconvex problems demonstrate the promising performance of our method. Reference: https://arxiv.org/pdf/2411.01803

Talk 2: An Adaptive and Parameter-Free Nesterov's Accelerated Gradient Method
Speaker: Jaewook J. Suh
Abstract: In this talk, we introduce AdaNAG, an adaptive accelerated gradient method based on Nesterov's accelerated gradient (NAG). The algorithm is line-search-free, parameter-free, and achieves the accelerated convergence rates $f(x_k) - f_\star = O(1/k^2)$ and $\min_{i\in\{1, ... ,k\}} \|\nabla f(x_i)\|^2 = O(1/k^3)$ for an $L$-smooth convex function $f$. We provide a Lyapunov analysis for the convergence proof of AdaNAG, which additionally enables us to propose a novel adaptive gradient descent (GD) method, AdaGD. AdaGD achieves the non-ergodic convergence rate $f(x_k) - f_\star = O(1/k)$, like the original GD. Motivated by the relationship between the parameter choice and the convergence guarantee of AdaGD, we obtain a generalized AdaNAG that provides a practically useful variant of AdaNAG. We provide numerical results showing that our method outperforms other recently proposed adaptive methods in certain scenarios.

Talk 3: Stochastic gradient methodswithBlock Coordinate Optimistic Stepsizes
Speaker: Tao Jiang
Abstract: Ill-conditioning is a major challenge for optimization with first-order methods. This is especially the case for stochastic optimization, where preconditioners in the classical sense are hard to construct due to the nature of stochastic gradients. We propose a block-coordinate stepsize rule that can effectively combat ill-conditioning as well as inhomogeneous noise in the stochastic setting. Our method is motivated by minimizing the expected distance to an optimal point during each iteration. Specifically, we use the optimistic stepsizes as if the expected search directions (e.g., stochastic gradients with or without momentum) along each coordinate always point to the optimal point. These stepsizes rely on online estimates of the second-moments of the coordinate-wise search directions. The popular Adam algorithm can be interpreted as a heuristic for such an estimation. Compared with Adam, our method requires fewer hyperparameters, obtains similar or better performance, and is numerically more stable.

Speakers
WG

Wenzhi Gao

Ph.D. student, Stanford University
Name: Wenzhi GaoSecond year Ph.D. student at Stanford ICME, working on large-scale numerical optimization, first-order methods, and online decision-making problems.
avatar for Jaewook J. Suh

Jaewook J. Suh

Name: Jaewook J. SuhAffiliation: Rice University
TJ

Tao Jiang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1X
Monday July 21, 2025 10:30am - 11:45am PDT
Monday July 21, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 1Y
Monday July 21, 2025 10:30am - 11:45am PDT
Monday July 21, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

11:45am PDT

Lunch 1 (provided)
Monday July 21, 2025 11:45am - 1:15pm PDT
Mediterranean Buffet
Monday July 21, 2025 11:45am - 1:15pm PDT
USC Founder's / Hutton Park 3551 Trousdale Pkwy, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2A: Advances in Solving Large-Scale Problems: Accelerated Methods and Sharp Analyses (II)
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Advances in Solving Large-Scale Problems: Accelerated Methods and Sharp Analyses (II)
Chair: Liwei Jiang Sen Na
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Online Learning Guided Quasi-Newton Methods: Improved Global Non-Asymptotic Guarantees
Speaker: Ruichen Jiang
Abstract: Quasi-Newton methods are popular iterative algorithms known for their superior practical performance over gradient-descent-type methods. However, existing theoretical results for this class of algorithms fail to fully justify their observed advantages. In this talk, we discuss our recent efforts to address this issue. Specifically, in the strongly convex setting, we propose the first “globally” convergent quasi-Newton method that achieves an explicit “non-asymptotic superlinear” rate. We show that the rate presented for our method is provably faster than gradient descent after at most $O(d)$ iterations, where $d$ is the problem dimension. Additionally, in the convex setting, we present an accelerated variant of our proposed method that provably outperforms the accelerated gradient method and converges at a rate of $O(\min\{1/k^2, \sqrt{d \log k }/ k^{2.5}\})$, where $k$ is the number of iterations. To attain these results, we diverge from conventional approaches and construct our quasi-Newton methods based on the Hybrid Proximal Extragradient framework and its accelerated variants. Furthermore, a key algorithmic concept in our methods is an online learning framework for updating the Hessian approximation matrices. Specifically, we relate our method's convergence rate to the regret of a specific online convex optimization problem in the matrix space and choose the sequence of Hessian approximation matrices to minimize its overall regret.

Talk 2: Accelerated stochastic approximation with state-dependent noise
Speaker: Tianjiao Li
Abstract: We consider a class of stochastic smooth convex optimization problems under rather general assumptions on the noise in the stochastic gradient observation. As opposed to the classical problem setting in which the variance of noise is assumed to be uniformly bounded, herein we assume that the variance of stochastic gradients is related to the “sub-optimality” of the approximate solutions delivered by the algorithm. Such problems naturally arise in a variety of applications, in particular, in the well-known generalized linear regression problem in statistics. However, to the best of our knowledge, none of the existing stochastic approximation algorithms for solving this class of problems attain optimality in terms of the dependence on accuracy, problem parameters, and mini-batch size. We discuss two non-Euclidean accelerated stochastic approximation routines—stochastic accelerated gradient descent (SAGD) and stochastic gradient extrapolation (SGE)—which carry a particular duality relationship. We show that both SAGD and SGE, under appropriate conditions, achieve the optimal convergence rate, attaining the optimal iteration and sample complexities simultaneously. However, corresponding assumptions for the SGE algorithm are more general; they allow, for instance, for efficient application of the SGE to statistical estimation problems under heavy tail noises and discontinuous score functions. We also discuss the application of the SGE to problems satisfying quadratic growth conditions, and show how it can be used to recover sparse solutions. Finally, we report on some simulation experiments to illustrate numerical performance of our proposed algorithms in high-dimensional settings.

Talk 3: SQP for physics-informed machine learning
Speaker: Qi Wang
Abstract: A methodology for physics-informed machine learning is presented, which incorporates prior information in the training problem through hard constraints, rather than the typical modern practice of using soft constraints (i.e., regularization terms or penalty methods). The methodology is based on a recently proposed stochastic-gradient-based SQP algorithm and is extended to use Adam-type step computation in the presence of hard constraints. The effectiveness of the method is demonstrated through numerical experiments on physics-informed learning problems.

Speakers
avatar for Ruichen Jiang

Ruichen Jiang

PhD student, UT Austin
Hi, I am a PhD student in the Department of ECE at UT Austin, advised by Aryan Mokhtari. My current research interests focus on convex and non-convex optimization, particularly in using online learning techniques to design optimization methods. I am always happy to chat about min-max... Read More →
avatar for Tianjiao Li

Tianjiao Li

PhD student, Georgia Institute of Technology
Name: Tianjiao LiTitle: PhD studentAffiliation: Georgia Institute of TechnologyBio:Tianjiao Li is a fifth-year PhD candidate in the School of Industrial and Systems Engineering at Georgia Institute of Technology, advised by Prof. George Lan and Prof. Ashwin Pananjady. His research... Read More →
QW

Qi Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2B: Deterministic and Stochastic Methods for Optimization and Games- Part I
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Deterministic and Stochastic Methods for Optimization and Games- Part I
Chair: Gesualdo Scutari
Cluster: Multi-agent Optimization and Games

Talk 1: On the computation of quasi-Nash equilibria under uncertainty
Speaker: Zhuoyu Xiao
Abstract: Motivated by applications in network congestion games and Cournot games, we consider the computation of either Nash or quasi-Nash equilibria in static stochastic noncooperative games afflicted by either non convexity or non-monotonicity. We consider sampled variants of gradient and best-response and show that under specified conditions, both schemes generate sequences which converge to quasi-Nash equilibria almost surely. We also provide non-asymptotic rate statements in some cases. Time permitting, we briefly discuss distributed extensions in networked settings of both schemes. Numerical experiments are also provided to support our theoretical results.

Talk 2: Iteratively Regularized Gradient Tracking Methods for Distributed Optimal Equilibrium Seeking
Speaker: Farzad Yousefian
Abstract: We consider a class of distributed constrained optimization problems where the constraint set is characterized by the solution set of a distributed monotone variational inequality problem. This problem is motivated by the need for estimation of the efficiency of equilibria in Nash games. First, we consider solving this problem over directed networks. We develop an iteratively regularized distributed gradient tracking method where the agents employ a push-pull protocol to communicate over the network. Second, we consider a stochastic variant of this problem over undirected networks and develop an iteratively regularized distributed stochastic gradient tracking method. For both algorithms, we establish the convergence of the generated iterates by the agents to the optimal equilibrium and derive new convergence rate statements. We validate the two proposed methods and present preliminary numerical results for computing the optimal equilibrium in a Cournot competition.

Talk 3: Clipped-Stochastic Methods for Generalized Smooth Stochastic Variational Inequalities
Speaker: Angelia Nedich
Abstract: We focus on solving a stochastic variational inequality (SVI) problem under relaxed smoothness assumption for a class of structured non-monotone operators. The SVI problem has attracted significant interest in the machine learning community due to its immediate application to adversarial training and multi-agent reinforcement learning. In many such applications, the resulting operators do not satisfy the smoothness assumption. To address this issue, we focus on the generalized smoothness assumption and consider two well-known stochastic methods with clipping, namely, projection and Korpelevich. For these clipped methods, we provide the first almost-sure convergence results without making any assumptions on the boundedness of either the stochastic operator or the stochastic samples. Furthermore, we provide the first almost-sure convergence results and in-expectation convergence rate results for these methods under a relaxed smoothness assumption.

Speakers
GS

Gesualdo Scutari

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FY

Farzad Yousefian

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AN

Angelia Nedich

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2C: Recent Advances in Theory and Algorithms for Multiagent Systems
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Recent Advances in Theory and Algorithms for Multiagent Systems
Chair: Andrew Liu
Cluster: Multi-agent Optimization and Games

Talk 1: Approximate Global Convergence of Independent Learning in Multi-Agent Systems
Speaker: Zaiwei Chen
Abstract: Independent learning (IL), despite being a popular approach in practice to achieve scalability in large-scale multi-agent systems, usually lacks global convergence guarantees. In this paper, we study two representative algorithms, independent Q-learning and independent natural actor-critic, within value-based and policy-based frameworks and provide the first finite-sample analysis for approximate global convergence. Our results indicate that IL can achieve global convergence up to a fixed error, which arises from the dependence among agents and characterizes the fundamental limit of IL in attaining global convergence. To establish the result, we develop a novel approach for analyzing IL by constructing a separable Markov decision process (MDP) for convergence analysis and then bounding the gap due to the model difference between the separable MDP and the original one. Moreover, we conduct numerical experiments using a synthetic MDP and an electric vehicle charging example to demonstrate our results and the practical applicability of IL.

Talk 2: Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependencies
Speaker: Alex Deweese
Abstract: Many multi-agent systems in practice are decentralized and have dynamically varying dependencies. There has been a lack of attempts in the literature to analyze these systems theoretically. In this paper, we propose and theoretically analyze a decentralized model with dynamically varying dependencies called the Locally Interdependent Multi-Agent MDP. This model can represent problems in many disparate domains such as cooperative navigation, obstacle avoidance, and formation control. Despite the intractability that general partially observable multi-agent systems suffer from, we propose three closed-form policies that are theoretically near-optimal in this setting and can be scalable to compute and store. Consequentially, we reveal a fundamental property of Locally Interdependent Multi-Agent MDP's that the partially observable decentralized solution is exponentially close to the fully observable solution with respect to the visibility radius. We then discuss extensions of our closed-form policies to further improve tractability. We also provide simulations to investigate some long horizon behaviors of our closed-form policies.

Talk 3: Hybrid Mean-Field Control and Mean-Field Equilibrium: Theories, Algorithms and Applications
Speaker: Andrew Liu
Abstract: In this talk, we introduce a hybrid multiagent modeling framework that combines Mean Field Control (MFC) and Mean Field Equilibrium (MFE). A perfect example of this framework is the operation of multiple virtual power plants (VPPs) or aggregators, each applying an MFC algorithm to manage the distributed energy resources (DERs) within their portfolios. These aggregators participate in the wholesale energy market by bidding on behalf of the DERs they represent, navigating the dynamic and uncertain market environment. Traditional game-theoretic approaches fall short in capturing the complexity of repeated and dynamic interactions under such uncertainties. Hence, we leverage the MFG approach to study these agent interactions and the resulting market dynamics. The MFC framework empowers each aggregator to determine optimal control policies despite uncertainties in solar output, demand fluctuations, and price volatility. Simultaneously, the MFE framework models strategic interactions between aggregators and other market participants, enabling a scalable approach for large systems. We establish the existence of a strong Nash equilibrium within this hybrid structure and propose a reinforcement learning-based algorithm to help aggregators learn and optimize their strategies over time. Crucially, this prescriptive approach facilitates control automation, enabling the integration of advanced AI and machine learning techniques at the grid edge, to optimize resource management and achieve system-wide benefits. We validate this framework through simulations of the Oahu Island electricity grid, showing that the combination of energy storage and mean-field learning significantly reduces price volatility and yields stable market outcomes. This work demonstrates the power and flexibility of the hybrid MFC-MFE approach, offering a robust foundation for scalable, automated decision-making in energy markets and beyond.

Speakers
ZC

Zaiwei Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AD

Alex Deweese

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AL

Andrew Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2D: Methods for Large-Scale Nonlinear Optimization II
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Methods for Large-Scale Nonlinear Optimization II
Chair: Mike O'Neill
Cluster: Nonlinear Optimization

Talk 1: Stochastic-Gradient-based Interior-Point Methods
Speaker: Frank E. Curtis
Abstract: I will discuss some of our recent work on stochastic-gradient-based interior-point algorithms for solving constrained optimization problems, such as those arising in informed machine learning. The algorithms are single-loop in the sense that they do not require an inner criterion for updating the barrier parameter; rather, the barrier parameter is decreased according to a prescribed sequence. Convergence guarantees are attained in both deterministic and stochastic settings. The algorithms exhibit good practical performance in comparison to projected-gradient-based methods.

Talk 2: Fast unconstrained optimization via Hessian Averaging and Adaptive Gradient Sampling Methods
Speaker: Raghu Bollapragada
Abstract: In this talk, we discuss minimizing finite-sum and expectation objective functions using Hessian-averaging-based subsampled Newton methods. These methods accommodate gradient inexactness and maintain fixed per-iteration Hessian approximation costs. Recent work (Na et al. 2023) showed that Hessian averaging achieves fast $\mathcal{O}\left(\sqrt{\tfrac{\log k}{k}}\right)$ local superlinear convergence for strongly convex functions, but requires gradient exactness and strong convexity, limiting practical use. To address this, we propose Hessian-averaged methods with adaptive-sampling strategies allowing gradient inexactness. For finite-sum problems, we use deterministic sampling, yielding global linear and sublinear convergence for strongly convex and nonconvex functions. We derive an improved local superlinear rate of $\mathcal{O}\left(\tfrac{1}{k}\right)$. For expectation problems, we use stochastic sampling and derive global linear/sublinear rates and a local superlinear rate of $\mathcal{O}\left(\tfrac{1}{\sqrt{k}}\right)$. Additionally, we introduce scalable methods like the diagonally-averaged Newton (Dan) method for large-scale problems. Numerical results show that Hessian averaging enhances convergence and achieves state-of-the-art performance on challenging tasks like CIFAR100 classification with ResNets.

Talk 3: A Stochastic Objective-Function-Free Adaptive Regularization Method with Optimal Complexity
Speaker: Sadok Jerad
Abstract: A fully stochastic second-order adaptive-regularization method for unconstrained non-convex optimization is presented which never computes the objective-function value, but yet achieves the optimal complexity bound for finding first-order critical points. The method is fully adaptive and the inexactness conditions required for convergence depend on the history of past steps. Numerical experiments on large binary classification problems illustrate the potential of the new method.

Speakers
avatar for Frank E. Curtis

Frank E. Curtis

Professor, Lehigh University
Name: Frank E. Curtis, Ph.D.Title: ProfessorAffiliation: Lehigh UniversityBio: Please see my website.Fun Fact: My wife and I have been together for 20 years and she's never seen me without a beard.
RB

Raghu Bollapragada

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SJ

Sadok Jerad

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2E: Efficient Optimization Methods for LLMs (Part I)
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Efficient Optimization Methods for LLMs (Part I)
Chair: Ruoyu Sun
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Adam-mini: Use Fewer Learning Rates To Gain More
Speaker: Ruoyu Sun
Abstract: TBD

Talk 2: GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Speaker: Zhangyang Wang
Abstract: TBD

Talk 3: LoRA-GA: Low-Rank Adaptation with Gradient Approximation
Speaker: Jian Li
Abstract: TBD

Speakers
RS

Ruoyu Sun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZW

Zhangyang Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jian Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2F: Quantum Linear Algebra and Optimization (Part 1))
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Quantum Linear Algebra and Optimization (Part 1))
Chair: Mohammadhossein Mohammadisiahroudi
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Quantum Computing-based Sensitivity Analysis for PDE-constrained Optimization
Speaker: Sicheng He
Abstract: Quantum computing is an emerging paradigm offering significant speed-ups for solving specific mathematical problems. In recent years, optimization and scientific computing researchers have developed quantum algorithms that demonstrate complexity advantage for large-scale problems. A key area of focus has been to leverage quantum linear algebra techniques to solve linear systems that arise in optimization and scientific computing applications. We propose quantum computing-based direct and adjoint methods for implicit sensitivity analysis in PDE-constrained optimization. The proposed quantum approaches achieve exponential speed-up in complexity with respect to the problem dimension, i.e., the number of state variables, compared to classical methods. Notably, in the quantum computing framework, both the direct and adjoint methods exhibit similar computational complexity, a departure from their classical counterparts. We demonstrate the proposed method using a simple heat transfer problem implemented with the IBM Qiskit simulators

Talk 2: Recent Advances in Quantum Interior Point Methods
Speaker: Tamas Terlaky
Abstract: Quantum Interior Point Methods (QIPMs) have recently emerged as a potential approach to accelerating the solution of large-scale conic optimization problems by leveraging quantum linear system algorithms for solving the Newton systems in IPMs. However, the one of significant challenges of QIPMs is the inexact and noisy nature of quantum solvers. In this talk, we discuss recent advancements in the design of efficient QIPMs that effectively manage errors. We introduce novel reformulations of the Newton system that enable maintaining feasibility despite inexact Newton directions. Additionally, we employ iterative refinement techniques to enhance solution accuracy while operating under limited precision. Our proposed QIPMs achieve the best-known iteration complexity, offering a significant step forward in the practical realization of quantum-accelerated optimization.

Talk 3: Quantum Approaches to Mixed Integer PDE-Constrained Optimization
Speaker: Adrian Harkness
Abstract: Mixed-integer PDE-constrained optimization (MIPDECO) problems arise in applications like gas and power networks or turbine placement. These problems combine the combinatorial complexity of integer programming with the large-scale linear systems of PDE-constrained optimization. This work investigates quantum computing methods for solving MIPDECO problems, specifically with binary control variables and a knapsack constraint. By using a first-discretize-then-optimize approach, we derive a binary quadratic optimization (BQO) formulation. We then explore two quantum algorithmic strategies based on the Quantum Approximate Optimization Algorithm (QAOA): a constraint-enforcing approach using custom mixer Hamiltonians, and a penalty-based method embedding constraints into the objective function. We discuss penalty parameter selection for feasibility and compare simulations of both quantum approaches. Our results illustrate the potential advantages of quantum methods for solving PDE-constrained optimization problems

Speakers
MM

Mohammadhossein Mohammadisiahroudi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SH

Sicheng He

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TT

Tamas Terlaky

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2G: Recent progresses in derivative-free optimization I
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Recent progresses in derivative-free optimization I
Chair: Giampaolo Liuzzi
Cluster: Derivative-free Optimization

Talk 1: Worst-case complexity analysis of derivative-free methods for multi-objective optimization
Speaker: Giampaolo Liuzzi
Abstract: In this work, we consider unconstrained multiobjective optimization problems where objective function values can only be obtained by querying a black box. The main aim of the paper is to give worst-case complexity bounds for derivative-free multi-objective optimization methods which adopt a linesearch expansion technique. We show that the considered methods enjoy the same worst-case complexity bounds recently proved for a directional multisearch method. Furthermore, exploiting the expansion technique, we are also able to give a further complexity results concerning the number of iterations with a measure of stationarity above a prefixed tolerance.

Talk 2: Exploring complexity bounds of model based trust region derivative free methods
Speaker: Katya Scheinberg
Abstract: Model-based trust region derivative free methods pioneered by Powell rely on interpolation models to approximate objective function in a trust region. The quality of this approximation dictates algorithmic progress and is, in turn, dictated by the geometry of the sample set. The methods are designed to trade-off carefully between the "exploration" and "exploitation", i.e. between seeking progress an improving sample set geometry. While these methods have been very successful in practice and have been show to converge, their complexity analysis has been incomplete, especially in terms of dependence on the dimension. We will present an improved complexity analysis for different variants of these methods.

Talk 3: Revisiting the convergence analysis of derivative-free trust region and direct search
Speaker: Cunxin Huang
Abstract: Derivative-Free trust region and direct search are two popular classes of derivative-free optimization methods. In this paper, we propose a unified new perspective for the convergence analysis of these two classes of methods. Specifically, we find that the behavior of an algorithm-determined series will decide the asymptotic convergence, which is a generalization of the existing results under both deterministic and randomized settings.

Speakers
GL

Giampaolo Liuzzi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KS

Katya Scheinberg

Professor, Georgia Institute of Technology
Name:Katya Scheinberg Title: Coca-Cola Foundation Chair and ProfessorAffiliation: H. Milton Stewart School of Industrial and Systems Engineering  Georgia Institute of Technology, Atlanta, GABio:Katya Scheinberg is a Coca-Cola Foundation Chair and Professor in the H. Milton Stewart... Read More →
CH

Cunxin Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2H: Optimization as the Engine of Generative AI - II
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Optimization as the Engine of Generative AI - II
Chair: Renyuan Xu
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Designing and Optimizing Biological Molecules with Multimodal Stochastic Interpolant Generative Models
Speaker: Ge Liu
Abstract: The design and optimization of biological molecules, such as proteins and peptides, offers transformative potential in medicine, materials science, and synthetic biology. Traditional methods such as directed evolution often struggle to explore the vast and complex molecular landscape efficiently. In addition, molecule design problems inherently involve both discrete and continuous variables (e.g., protein sequences and 3D structures) and operate on non-Euclidean manifolds to model geometric information (e.g., rotational group SO(3)). Generative modeling has emerged as a powerful framework for biological molecule design. In this talk, I will present recent advances in SDE/ODE-based stochastic interpolant generative models, such as diffusion and flow-matching, that enabled precise and controllable generation of biological molecules across multiple modalities. First, I will introduce Statistical Flow Matching (SFM), a novel generative framework leveraging the Riemannian geometry of statistical manifolds that enables efficient generation of discrete data. SFM has demonstrated strong performance on biological sequence design (DNA, protein) and generalizable to text and image domains. Next, I will introduce OC-Flow, a theoretically grounded training-free optimal control framework for guiding flow-matching generative models on both Euclidean and non-Euclidean manifolds. By formulating generative sampling as an optimal control problem, OC-Flow enables effective guided sampling for solving a diverse set of inverse problem across computer vision, chemical molecule, and peptide design tasks, achieving controlled generation of molecules with optimized properties and energies. This talk will provide new perspectives on how stochastic interpolant generative models can bridge optimization, machine learning, and biomolecular engineering, paving the way for next-generation protein design.

Talk 2: Panel
Speaker: Ahmad Beirami
Abstract: Ahmad Beirami (Google DeepMind) and Renyuan Xu (Stanford University) will host a panel on the interactions between Optimization and Generative AI.

Talk 3: Panel
Speaker: Renyuan Xu
Abstract: Ahmad Beirami (Google DeepMind) and Renyuan Xu (Stanford University) will host a panel on the interactions between Optimization and Generative AI.

Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2I: Frontiers of Optimization for Machine Learning - Part II
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Frontiers of Optimization for Machine Learning - Part II
Chair: Vivak Patel
Cluster: Nonlinear Optimization

Talk 1: Online Statistics Inference via Matrix-Free Newton Methods
Speaker: Sen Na
Abstract: Given the ubiquity of streaming data, online algorithms have been widely used for parameter estimation, with second-order methods particularly standing out for their efficiency and robustness. In this talk, we introduce an online sketched Newton method that leverages a randomized sketching technique to perform an approximate Newton step in each iteration, thereby eliminating the computational bottleneck of second-order methods. While existing studies have established the asymptotic normality of sketched Newton methods, a consistent estimator of the limiting covariance matrix remains an open problem. We propose a fully online covariance matrix estimator that is constructed entirely from the Newton iterates and requires no matrix factorization. Compared to covariance estimators for first-order online methods, our estimator for second-order methods is batch-free. We establish the consistency and convergence rate of our estimator, and coupled with asymptotic normality results, we can then perform online statistical inference for the model parameters based on sketched Newton methods. We also discuss the extension of our estimator to constrained problems, and demonstrate its superior performance on regression problems as well as benchmark problems in the CUTEst set.

Talk 2: Optimization in Combinatorial and Non-Convex ML: Positive and Negative Results
Speaker: Jean Honorio
Abstract: Several modern machine learning (ML) problems are combinatorial and non-convex, for which theoretical guarantees are quite limited. My long-term research goal is to uncover the general foundations of ML and optimization that drives empirical success. I aim to develop a set of optimization-theoretic frameworks and tools to bridge the aforementioned gaps, to further our understanding of continuous (possibly non-convex) relaxations of combinatorial problems, as well as our knowledge of non-convexity. In this talk, I first focus on invex (non-convex) optimization problems, with some motivation from exploratory research on fairness and mixed linear regression. I present a generalization of gradient descent for the family of invex (non-convex) problems, which provably converges to the global minimum in polynomial time. Finally, for general non-convex problems, I show that any gradient-based algorithm, requires an exponential number of gradients to find the global minimum. Second, when stochastic gradients are biased, how can we obtain convergence to the global minima of a complex convex function? I propose a provably convergent algorithm that requires more computational effort as the algorithm progresses through several gradient descent iterations. Interestingly, more complex algorithms do not converge in this regime. Third, I discuss a difficult combinatorial problem over directed graphs with acyclicity constraints. Interestingly, using the statistical principle of identifiability, one can reduce the search space, and propose provably correct sequential optimization algorithms. Finally, I focus on problems with high-order relationships, usually formulated as tensor optimization problems. I propose a convex conic form relaxation. To this end, I carefully define the Caratheodory symmetric tensor cone, and discuss its benefits in optimization

Talk 3: Hessian-aware scaling of the gradient directions
Speaker: Fred Roosta
Abstract: Gradient descent is the primary workhorse for the optimisation of large-scale, nonconvex problems in machine learning. However, its performance is heavily dependent on step size selection. Due to a lack of natural scaling, this necessitates costly line searches or heuristic guess-and-check methods for step size selection. We propose an efficient scaling of gradient descent using a scalar informed by Hessian information. By carefully considering the curvature along the gradient direction, we demonstrate that Hessian-aware scaled gradient directions provide a local unit step size guarantee, even in the nonconvex setting. We extend this result to scenarios where the Hessian and gradient are stochastic. Additionally, we show global convergence of Hessian-aware scaled gradient descent under a significant weakening of the typical Lipschitz gradient smoothness assumption. We validate our results on large-scale machine learning problems and demonstrate that, through alternating scalings, we obtain an algorithm that rapidly converges across various problems.

Speakers
VP

Vivak Patel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SN

Sen Na

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JH

Jean Honorio

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FR

Fred Roosta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2J: Optimization in Control - Algorithms and Applications
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Optimization in Control - Algorithms and Applications
Chair: Arvind Raghunathan
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Optimization based control of networked systems
Speaker: Sridharakumar Narasimhan
Abstract: Process industries often use multiple units in series parallel combination, e.g., heat exchangers, pumps, compressors, fuel cells/solar cells etc. Optimal operation of such a network involves determining the appropriate loads of each equipment optimally by minimizing a cost function (e.g., power consumed, current drawn, heat loss) while satisfying safety constraints and meeting overall constraints on flow, power, etc. This results in a constrained optimization problem. Real time optimization or optimal control requires such a problem to solved in real time using measurements with uncertain plant models. In this work, we present a methodology for optimal control of a network of equipment using the structure of the optimal solution, e.g., . In many networked systems, optimality requires that the gradients of the cost function with respect to manipulated variables are equal. Hence, the optimal loads in the different branches of the network are manipulated such that the optimality conditions are satisfied. This is demonstrated using numerical simulations of a fuel cell stack.

Talk 2: Strong Disjunctive Cuts for MIP formulations of Optimal Control of Piecewise Affine Systems
Speaker: Prachi Shah
Abstract: Hybrid systems governed by piecewise affine dynamics are widely used in modern control applications, yet their optimal control remains a computational challenge due to weak relaxations of traditional mixed-integer programming (MIP) formulations. In this work, we present a novel methodology that introduces strong disjunctive cuts derived from the convex hull of a subproblem restricted to consecutive time intervals. Our approach tightens the linear relaxation of any choice of MIP formulation while keeping the cut-generation complexity independent of the overall time horizon. Comprehensive computational experiments on benchmark problems demonstrate that this strategy not only yields tighter bounds at the root node but also reduces the solution time of the MIP.

Talk 3: On global convergence of MPEC methods for Optimal Control of Hybrid Dynamical Systems
Speaker: Saif Kazi
Abstract: Optimal control of a Hybrid Dynamical System is a difficult problem because of unknown non differentiable points or switches in the solution of discontinuous ODEs. The optimal control problem for such hybrid dynamical system can be reformulated into a dynamic complementarity system (DCS) problem. Subsequently, the differential equation system is further reformulated using numerical discretization schemes such as Implicit Runge Kutta (IRK) or Orthogonal Collocation method for higher order accurate numerical solutions. Since the solutions are non-differentiable, a moving finite element with switch detection method is implemented to ensure higher order accuracy along with different reformulations for the non-smooth complementarity constraints such as relaxation based formulations or the l1-penalty term formulation. In this paper, we analyze the global convergence properties of such reformulations and introduce a mixed active-set based strategy to converge to real optimal solutions and escape spurious stationary points.

Speakers
SN

Sridharakumar Narasimhan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PS

Prachi Shah

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SK

Saif Kazi

Research Scientist, Los Alamos National Laboratory
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2K: Recent advances in matrix optimization
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Recent advances in matrix optimization
Chair: Chao Ding
Cluster: Conic and Semidefinite Optimization

Talk 1: A quadratically convergent semismooth Newton method for nonlinear semidefinite programming without subdifferential regularity
Speaker: Fuxiaoyue Feng
Abstract: The non-singularity of generalized Jacobians of the Karush-Kuhn-Tucker (KKT) system is crucial for local convergence analysis of semismooth Newton methods. In this talk, we present a new approach that challenges this conventional requirement. Our discussion revolves around a methodology that leverages some newly developed variational properties, effectively bypassing the necessity for non-singularity of all elements in the generalized Jacobian. Quadratic convergence results of our Newton methods are established without relying on commonly assumed subdifferential regularity conditions. This discussion may offer fresh insights into semismooth Newton methods, potentially paving the way for designing robust and efficient second-order algorithms for general nonsmooth composite optimizations.

Talk 2: On efficient and scalable computation of the nonparametric maximum likelihood estimator in mixture models
Speaker: Yangjing Zhang
Abstract: The nonparametric maximum likelihood estimation (NPMLE) is a classic and important method to estimate the mixture models from finite observations. In this talk, we propose an efficient and scalable semismooth Newton based augmented Lagrangian method (ALM). By carefully exploring the structure of the ALM subproblem, we show that the computational cost of the generalized Hessian (second order information) is independent of the number of grid points. Extensive numerical experiments are conducted to show the effectiveness of our approach. 

Talk 3: An Accelerated Proximal ADMM for ODC of Uncertain Systems
Speaker: Xinyuan Zhao
Abstract: To ensure the system stability of the H2-guaranteed cost optimal decentralized control (ODC) problem, we formulate an approximate semidefinite programming (SDP) problem that leverages the block diagonal structure of the decentralized controller's gain matrix. To minimize data storage requirements and enhance computational efficiency, we employ the Kronecker product to vectorize the SDP problem into a conic programming (CP) problem. We then propose a proximal alternating direction method of multipliers (PADMM) to solve the dual of the resulting CP problem. By using the equivalence between the semi-proximal ADMM and the (partial) proximal point algorithm, we identify the non-expansive operator of PADMM, enabling the use of Halpern fixed-point iteration to accelerate the algorithm. Finally, we demonstrate that the sequence generated by the proposed accelerated PADMM exhibits a fast convergence rate for the Karush-Kuhn-Tucker residual. Numerical experiments confirm that the accelerated algorithm outperforms the well-known COSMO, MOSEK, and SCS solvers in efficiently solving large-scale CP problems, particularly those arising from H2-guaranteed cost ODC problems.

Speakers
CD

Chao Ding

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FF

Fuxiaoyue Feng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Yangjing Zhang

Yangjing Zhang

assistant professor, Chinese Academy of Sciences
I am currently an assistant professor in Institute of Applied MathematicsAcademy of Mathematics and Systems Science, Chinese Academy of Sciences. My current research is focused on large scale sparse optimization problems, the design of efficient algorithms for statistical models and graphical models... Read More →
XZ

Xinyuan Zhao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2L: Decision-Aware Optimization
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Decision-Aware Optimization
Chair: Vishal Gupta
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Contextual Linear Optimization with Bandit Feedback
Speaker: Yichun Hu
Abstract: Contextual linear optimization (CLO) uses predictive contextual features to reduce uncertainty in random cost coefficients and thereby improve average-cost performance. An example is stochastic shortest path with random edge costs (e.g., traffic) and contextual features (e.g., lagged traffic, weather). Existing work on CLO assumes the data has fully observed cost coefficient vectors, but in many applications we can only see the realized cost of a historical decision, that is, just one projection of the random cost coefficient vector, to which we refer as bandit feedback. We study a class of offline learning algorithms for CLO with bandit feedback, which we term induced empirical risk minimization (IERM), where we fit a predictive model to directly optimize downstream performance of the policy it induces. We show a fast-rate regret bound for IERM that allows for misspecified model classes and flexible choices of the optimization estimate, and we develop computationally tractable surrogate losses. A byproduct of our theory of independent interest is fast-rate regret bound for IERM with full feedback and misspecified policy class. We compare the performance of different modeling choices numerically using a stochastic shortest path example and provide practical insights from the empirical results.

Talk 2: Learning Uncertainty Sets in Dynamic Robust Optimization
Speaker: Irina Wang
Abstract: We present a data-driven technique to automatically learn uncertainty sets in dynamic decision making under uncertainty. We formulate the learning problem as a control design problem where the control policy involves solving a robust optimization problem parametrized by the past disturbances, as well as the parameters of the uncertainty set. We propose a learning procedure to dynamically predict the parameters of the uncertainty set to minimize a closed-loop performance metric while satisfying probabilistic guarantees of constraint satisfaction. Our approach allows for uncertain data that is correlated across time periods, and can learn a wide range of commonly used uncertainty sets. By modeling our training problem objective and constraints using coherent risk metrics, we derive finite sample probabilistic guarantees of constraint satisfaction in multi-stage settings.

Talk 3: Surrogates for Decision-Aware Learning: Beyond the Linear Setting
Speaker: Vishal Gupta
Abstract: Designing surrogates that exploit downstream optimization structures is one of the key approaches to decision-aware learning. However, most work to date is either restricted to the linear contextual optimization setting or is largely heuristic with few theoretical performance guarantees. By extending recent work on using directional gradients to approximate decision loss, we show how to design surrogates with provable performance guarantees for nonlinear settings. This approach provides a natural recipe for attacking non-parametric and high-dimensional settings.

Speakers
YH

Yichun Hu

Assistant Professor, Cornell University
IW

Irina Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
VG

Vishal Gupta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2M: Applications of polynomial optimization to data analysis II
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Applications of polynomial optimization to data analysis II
Chair: Yong Sheng Soh
Cluster: Conic and Semidefinite Optimization

Talk 1: Semidefinite Network Games
Speaker: Antonios Varvitsiotis
Abstract: Network games are an important class of games that model agent interactions in networked systems, where players are situated at the nodes of a graph and their payoffs depend on the actions taken by their neighbors. We extend the classical framework by considering a game model where the strategies are positive semidefinite matrices having trace one. These (continuous) games can serve as a simple model of quantum strategic interactions. We focus on the zero-sum case, where the sum of all players’ payoffs is equal to zero. We establish that in this class of games, Nash equilibria can be characterized as the projection of a spectrahedron, that is, the feasible region of a semidefinite program. Furthermore, we demonstrate that determining whether a game is a semidefinite network game is equivalent to deciding if the value of a semidefinite program is zero. Beyond the zero-sum case, we characterize Nash equilibria as the solutions of a semidefinite linear complementarity problem.

Talk 2: Sum of squares hierarchy for the Gromov-Wasserstein Problem
Speaker: Yong Sheng Soh
Abstract: The Gromov-Wasserstein (GW) Problem is an extension of the classical optimal transport problem that allows one to compute distances between probability distributions specified over incomparable metric spaces. Broadly speaking, to get around the lack of a natural notion of distance between objects residing in different metric spaces, the GW computes the minimum of a suitably defined objective taken over all possible embeddings of the input metric spaces to a common space. This process leaves us with solving a non-convex quadratic programming instance. In this talk, we discuss the ideas of the sum-of-squares hierarchy applied to solving the GW problem. As a note, the central object of interest in the GW problem is a probability distribution, and we describe the necessary language in which ideas of polynomial optimization carry through to distributions.

Talk 3: On the geometric and computational complexity of polynomial bilevel optimization
Speaker: Quoc-Tung Le
Abstract: Bilevel optimization is an important mathematical tool to model phenomena in many domains, such as economic game theory, decision science and machine learning, to name but a few. Despite its importance, efficient and scalable algorithms for bilevel optimization are mostly developed for the (strong) convexity of the lower-level problem case, which is unrealistic for many practical tasks. In the quest to understand more general bilevel problems, we relax the lower level strong convexity and consider polynomial bilevel optimization, i.e., polynomial objective functions and constraints. We focus on the worst-case analysis of this class of problems, from both geometric and computational viewpoints. Our analysis suggests that even the algebraic rigidity of polynomials does not exclude extreme pathologies induced by the bilevel optimization. More specifically, we demonstrate that any semi-algebraic function can be represented as the objective of a polynomial bilevel problem. This discovery implies that solving polynomial bilevel optimization is equivalent to optimizing general semi-algebraic functions. We obtain other sharp variations of this result by considering relevant properties of the lower problem, such as convexity or feasible set compacity. In addition, we show the Σ2p-hardness of polynomial bilevel optimization, characterizing polynomial bilevel problems as vastly more challenging than NP-complete problems (under reasonable hardness assumptions).

Speakers
AV

Antonios Varvitsiotis

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YS

Yong Sheng Soh

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
QL

Quoc-Tung Le

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2N: Optimization for Robotics II
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Optimization for Robotics II
Chair: Panos Patrinos
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Graphs of Convex Sets with Applications to Robot Motion Planning
Speaker: Tobia Marcucci
Abstract: In this talk we introduce a novel modeling and computational framework for joint discrete and continuous decision making. We consider graphs where each vertex is associated with a convex optimization problem, and each edge couples two problems through additional convex costs and constraints. We call these Graphs of Convex Sets (GCS). Many classical problems in graph theory are naturally generalized to GCS, yielding a new class of problems at the interface of combinatorial and convex optimization with a wide variety of applications. For the solution of these problems, we present a unified technique that leverages perspective operators to formulate tight convex relaxations and strong mixed-integer formulations. In the second part of the presentation, we focus on the shortest-path problem in GCS and its application to robot motion planning. We present early experiments from Amazon Robotics, where our framework enables warehouse robots to move packages between bins nearly twice as fast as the current motion-planning solution.

Talk 2: Deep-Learning Aided Optimization for Decision-Making
Speaker: Evangelos Theodorou
Abstract: Optimization problems in robotics are typically nonlinear and nonconvex, while their scalability can range from few to millions of states and controls variables depending on the use case, the robotic system and the task in consideration. In this talk I will present a new class of algorithms for distributed optimization of multi-agent systems with emphasis on robotics applications. The primary goal is to out-perform existing algorithms in terms of convergence speed, optimality and scaling. To do so we will draw connections between iterative optimization algorithms and model-based deep learning approaches. These connections will allow us to develop neural networks architectures for learning to optimize that are interpretable, scalable and come with generalization guarantees. The interpretability arises from treating each iteration of an optimization method as layer with the corresponding tuning parameters treated as learnable parameters. Training of such architectures takes place in a supervised as well as semi-supervised learning fashion. We will show a range of applications of such neural network architectures including large-scale distributed optimal control, model predictive control, and network flow problems. The proposed architectures do not only improve performance, they also address a long-standing problem in industry and academia related to interpretability of neural network architectures when deployed to the real world.

Talk 3: Safe treatment of infeasible convex optimization problems via the augmented Lagrangian
Speaker: Roland Andrews
Abstract: This work focuses on constrained convex optimization problems. The augmented Lagrangian method is a popular algorithm designed to tackle such problems by solving sequences of unconstrained optimization problems. It is practically efficient and offers strong theoretical guarantees under minimal assumptions, provided that the feasible set associated with the constraints is non-empty. However, the infeasible setting for constrained optimization problems has only recently started to attract attention. This issue is particularly relevant in areas such as optimal control (e.g., Model Predictive Control) and machine learning (e.g., neural networks using convex optimization layers), where infeasibility frequently arises. Recent studies have approached this problem under various assumptions. In this work, we analyze the general case, relying solely on convexity as the key assumption. Our approach leverages the classical relationship between the augmented Lagrangian algorithm and the dual proximal point algorithm.

Speakers
avatar for Tobia Marcucci

Tobia Marcucci

Assistant Professor, University of California, Santa Barbara
Name: Tobia MarcucciTitle: Assistant Professor of Electrical and Computer EngineeringAffiliation: University of California, Santa BarbaraBio:Tobia Marcucci is an Assistant Professor in the department of Electrical and Computer Engineering at the University of California, Santa Barbara... Read More →
ET

Evangelos Theodorou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RA

Roland Andrews

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2O: Optimization in Global Health
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Optimization in Global Health
Chair: Aleksandr Aravkin
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Large-scale Kernel Regression in Complex Global Health Estimation
Speaker: Peng Zheng
Abstract: We present an efficient approach to large-scale kernel regression with applications to estimating global mortality and causes of death. We highlight computational elements, particularly how computational elements in kernel regression interact with the problem scale and technical requirements, including constraints and aggregated observations.

Talk 2: Joint estimation of Prevalence, Sensitivity, and Specificity
Speaker: Nora Gilbertson
Abstract: Lack of perfect tests is a classic problem in epidemiology, and must be overcome to understand prevalence and burden of disease. Multiple imperfect tests are typically available, with partial information on their diagnostic properties (such as sensitivity and specificity). We present a joint inversion approach that allows us to obtain improved results for location-specific prevalence and diagnostic properties of multiple tests jointly, using all available information multiple locations and multiple imperfect tests. We explain the approach and show results on both simulated cases and schistosomiasis data.

Talk 3: Fast optimization approaches for raking
Speaker: Ariane Ducellier
Abstract: Raking is a classic problem in survey science, where available granular estimates are updated so that their aggregations across particular dimensions match available constraints from independent sources. We formulate raking as an optimization problem, and show how to efficiently solve complex raking examples in multiple dimensions with both direct and aggregated observations. The approach leverages duality theory, and intermediate results together with the implicit function theorem allow us to efficiently estimate asymptotic uncertainty of the raked estimates.

Speakers
PZ

Peng Zheng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
NG

Nora Gilbertson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AD

Ariane Ducellier

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2P: Techniques of PDE Optimization in Machine Learning
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Techniques of PDE Optimization in Machine Learning
Chair: Anton Schiela
Cluster: PDE-constrained Optimization

Talk 1: SA-NODEs and the Universal Approximation of Dynamical Systems"
Speaker: Lorenzo Liverani
Abstract: In this talk, I will introduce the framework of semi-autonomous neural ordinary differential equations (SA-NODEs), a variation of vanilla NODEs employing fewer parameters. This is achieved by making the coefficients of the SA-NODEs independent of time. Despite this apparent simplification, I will demonstrate that SA-NODEs retain all the strong approximation properties of Vanilla NODEs, both from a theoretical and a numerical perspective. Specifically, SA-NODEs are able to learn the global flow of a dynamical system and track the entire trajectory over a finite (but arbitrary) time horizon. I will conclude the talk by presenting several numerical experiments, showing that SA-NODEs perform well for various systems and significantly outperform vanilla NODEs. This is joint work with Z. Li, K. Liu, and E. Zuazua.

Talk 2: Preconditioned Gradient Methods for Optimizing Neural Networks with Hilbert Space Layers
Speaker: Frederik Koehne
Abstract: Optimization problems in the context of machine learning typically involve optimization variables that are operators between Hilbert Spaces. In gradient based methods, selecting an appropriate inner product on this space of linear operators is fundamental to obtain meaningful search directions. We review the natural inner product on the space of Hilbert Schmidt operators and demonstrate its efficient application in computing gradients for the transition matrices in artificial neural networks. This approach ensures that structural information from network layers is incorporated into optimization updates. We present the theoretical foundations, discretization details, and numerical results, confirming that the solutions obtained retain the expected structural properties.

Talk 3: ProxSTORM: A Stochastic Trust Region Algorithm for Nonsmooth Optimization
Speaker: Aurya Javeed
Abstract: This talk is about minimizing a smooth term plus a convex nonsmooth term. We present a stochastic proximal Newton trust region algorithm that assumes models and estimates of the objective are sufficiently accurate, sufficiently often. Like STORM (work on stochastic optimization with random models), we use facts about martingales to prove our algorithm is globally convergent with probability one.

Speakers
AS

Anton Schiela

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LL

Lorenzo Liverani

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FK

Frederik Koehne

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AJ

Aurya Javeed

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2Q: New Riemannian Optimization Applications
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: New Riemannian Optimization Applications
Chair: Jiang Hu
Cluster: Optimization on Manifolds

Talk 1: Retraction-free optimization over the Stiefel manifold with application to the LoRA fine-tuning
Speaker: Jiang Hu
Abstract: Optimization over the Stiefel manifold has played a significant role in various machine learning tasks. Many existing algorithms either use the retraction operator to keep each iterate staying on the manifold, or solve an unconstrained quadratic penalized problem. The retraction operator in the former corresponds to orthonormalization of matrices and can be computationally costly for large-scale matrices. The latter approach usually equips with an unknown large penalty parameter. To address the above issues, we propose a retraction-free and penalty parameter-free algorithm, which lands on the manifold. Moreover, our convergence theory allows using constant step size, which improve the result of converging to a neighborhood in \citep{ablin2022fast}. A key component of the analysis is the convex-like property of the quadratic penalty of the Stiefel manifold, which enables us to explicitly characterize the constant penalty parameter. As an application, we introduce a new algorithm, Manifold-LoRA, which employs the landing technique and a carefully designed step size strategy to accelerate low-rank adaptation (LoRA) in fine-tuning large language models. Numerical experiments on the benchmark datasets demonstrate the efficiency of our proposed method.

Talk 2: Optimal Tensor Network Disentanglement via Manifold Optimization
Speaker: Chao Yang
Abstract: A tensor network can be disentangled by performing a unitary gauge transformation within the network to allow the transformed network to be approximated by a low rank decomposition. Seeking an unitary transformation to minimize the truncation error is equivalent to solving a constrained optimization problem in which the optimal solution of the problem lies on a Stiefel manifold. We describe the objective function for achieving disentanglement and show how the problem can be solved by a Riemannian Newton's method. We also discuss practical issues such as the choice of a starting guess, the stopping criterion and how the gradient and Hessian can be computed efficiently.

Talk 3: A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity
Speaker: Jiayuan Wu
Abstract: This paper aims to develop a Newton-type method to solve a class of nonconvex composite programs. In particular, the nonsmooth part is possibly nonconvex. To tackle the nonconvexity, we develop a notion of strong prox-regularity which is related to the singleton property and Lipschitz continuity of the associated proximal operator, and we verify it in various classes of functions, including weakly convex functions, indicator functions of proximally smooth sets, and two specific sphere-related nonconvex nonsmooth functions. In this case, the problem class we are concerned with covers smooth optimization problems on manifold and certain composite optimization problems on manifold. For the latter, the proposed algorithm is the first second-order type method. Combining with the semismoothness of the proximal operator, we design a projected semismooth Newton method to find a root of the natural residual induced by the proximal gradient method. Due to the possible nonconvexity of the feasible domain, an extra projection is added to the usual semismooth Newton step and new criteria are proposed for the switching between the projected semismooth Newton step and the proximal step. The global convergence is then established under the strong prox-regularity. Based on the BD regularity condition, we establish local superlinear convergence. Numerical experiments demonstrate the effectiveness of our proposed method compared with state-of-the-art ones.

Speakers
JH

Jiang Hu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CY

Chao Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JW

Jiayuan Wu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2R: Theoretical and computational advances in nonconvex optimization via convex relaxations and distributionally robust optimization
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Theoretical and computational advances in nonconvex optimization via convex relaxations and distributionally robust optimization
Chair: E. Alper Yildirim
Cluster: Global Optimization

Talk 1: Semidefinite liftings for the complex cut polytope
Speaker: Lennart Sinjorgo
Abstract: We consider the complex cut polytope: the convex hull of Hermitian rank-one matrices xx*, where the elements of the n-dimensional vector x are complex m-th unit roots. These polytopes find applications in MAX-3-CUT, digital communication, and more generally, complex quadratic programming. For m = 2, the complex cut polytope corresponds to the well-known real cut polytope. We provide an exact description of the complex cut polytope for m = n = 3 and investigate second order semidefinite liftings of the complex cut polytope. For such second order liftings, we show a method for reducing the size of the matrix, without weakening the approximation. We support our theoretical findings with numerical experiments.

Talk 2: Novel and Tractable Convex Relaxations of Standard Quadratic Optimization Problems under Cardinality Constraints
Speaker: E. Alper Yildirim
Abstract: Standard quadratic optimization problems (StQPs) provide a versatile modelling tool in a multitude of applications such as mathematical finance, machine learning (clustering) and modelling in biosciences (e.g. selection and ecology). In this talk, we consider StQPs under an additional cardinality (sparsity) constraint which, even for convex objectives, renders NP-hard problems. One motivation to study StQPs under such sparsity restrictions is the high-dimensional portfolio selection problem with too many assets to handle, in particular in the presence of transaction costs. We present novel computational approaches to this relevant but difficult problem, involving modern conic optimization techniques, along with significant dimensional reduction, which is essential for tractability of these methods when problem size grows. In addition, we propose a particular generation procedure that systematically avoids too easy instances. We present extensive computational results demonstrating the versatility and strength of the proposed relaxations.

Talk 3: Distributionally robust standard quadratic optimization with Wasserstein ambiguity
Speaker: Daniel de Vicente
Abstract: The standard quadratic optimization problem (StQP) consists of minimizing a quadratic form over the standard simplex. If the quadratic form is neither convex nor concave, the StQP is NP-hard. This problem has many interesting applications ranging from portfolio optimization to machine learning. Sometimes, the data matrix is uncertain but some information about its distribution can be inferred, e.g. the first two moments or else a reference distribution (typically, the empirical distribution after sampling). In distributionally robust optimization, the goal is to minimize over all possible distributions in an ambiguity set defined based upon above mentioned characteristics. We will explore two versions: the distributionally robust stochastic StQP focussing on expectations, and the distributionally robust chance constrained StQP, both with an ambiguity set based upon maximal Wasserstein distance to the sampled distribution.

Speakers
avatar for Lennart Sinjorgo

Lennart Sinjorgo

Phd Student, Tilburg University
Lennart SinjorgoPhD Student in Operations Research at Tilburg UniversityInterested in semidefinite programming
EA

E. Alper Yildirim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DD

Daniel de Vicente

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2S: Robust Machine Learning
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Robust Machine Learning
Chair: Zhengyuan Zhou
Cluster: Optimization For Data Science

Talk 1: Robust Reinforcement Learning from Corrupted Human Feedback
Speaker: Zixuan Zhang
Abstract: Reinforcement learning from human feedback (RLHF) provides a principled framework for aligning AI systems with human preference data. For various reasons, e.g., personal bias, context ambiguity, lack of training, etc, human annotators may give incorrect or inconsistent preference labels. To tackle this challenge, we propose a robust RLHF approach $R^3M$, which models the potentially corrupted preference label as sparse outliers. Accordingly, we formulate the robust reward learning as an l1-regularized maximum likelihood estimation problem. Computationally, we develop an efficient alternating optimization algorithm, which only incurs negligible computational overhead compared with the standard RLHF approach. Theoretically, we prove that under proper regularity conditions, $R^3M$ can consistently learn the underlying reward and identify outliers, provided that the number of outlier labels scales sublinearly with the preference sample size. Furthermore, we remark that $R^3M$ is versatile and can be extended to various preference optimization methods, including direct preference optimization (DPO). Our experiments on robotic control and natural language generation with large language models (LLMs) show that $R^3M$ improves robustness of the reward against several types of perturbations to the preference data.

Talk 2: Robust Online Control with Model Misspecification
Speaker: Xinyi Chen
Abstract: We study online control of an unknown nonlinear dynamical system that is approximated by a time-invariant linear system with model misspecification. Our study focuses on robustness, a measure of how much deviation from the assumed linear approximation can be tolerated by a controller while maintaining finite ℓ2-gain. A basic methodology to analyze robustness is via the small gain theorem. However, as an implication of recent lower bounds on adaptive control, this method can only yield robustness that is exponentially small in the dimension of the system and its parametric uncertainty. The work of Cusumano and Poolla shows that much better robustness can be obtained, but the control algorithm is inefficient, taking exponential time in the worst case. In this paper we investigate whether there exists an efficient algorithm with provable robustness beyond the small gain theorem. We demonstrate that for a fully actuated system, this is indeed attainable. We give an efficient controller that can tolerate robustness that is polynomial in the dimension and independent of the parametric uncertainty; furthermore, the controller obtains an ℓ2-gain whose dimension dependence is near optimal.

Talk 3: Approximations to worst-case data dropping: unmasking failure modes
Speaker: Jenny Huang
Abstract: A data analyst might worry about generalization if dropping a very small fraction of data points from a study could change its substantive conclusions. Finding the worst-case data subset to drop poses a combinatorial optimization problem. To overcome this intractability, recent works propose using additive approximations, which treat the contribution of a collection of data points as the sum of their individual contributions, and greedy approximations, which iteratively select the point with the highest impact to drop and re-run the data analysis without that point [Broderick et al., 2020, Kuschnig et al., 2021]. We identify that, even in a setting as simple as OLS linear regression, many of these approximations can break down in realistic data arrangements. Several of our examples reflect masking, where one outlier may hide or conceal the effect of another outlier. Based on the failures we identify, we provide recommendations for users and suggest directions for future improvements.

Speakers
ZZ

Zhengyuan Zhou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XC

Xinyi Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JH

Jenny Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2T: Recent Advances in Stochastic Optimization: Complexity, Adaptivity, and Nonsmooth Extensions (II)
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Recent Advances in Stochastic Optimization: Complexity, Adaptivity, and Nonsmooth Extensions (II)
Chair: Sen Na & Zhaosong Lu
Cluster: Nonlinear Optimization

Talk 1: On the convergence of policy gradient methods for stochastic nonlinear dynamical systems
Speaker: Sungho Shin
Abstract: We analyze the local convergence of policy gradient methods for stochastic nonlinear dynamical systems. Under several technical assumptions, we show that the policy gradient algorithm converges to the optimal policy.

Talk 2: Improved complexity of proximal bundle methods and new insights on bundle management
Speaker: Andy Sun
Abstract: Proximal bundle method (PBA) is a fundamental and computationally effective algorithm for solving optimization problems with nonsmooth components. In this talk, we will first investigate a composite setting where one function is smooth and the other is piecewise linear. We present a novel complexity analysis of PBA and derive a O(\epsilon^{-0.8}) iteration complexity, improving upon the known O(\epsilon^{-2}) guarantee. Our analysis also sheds new light on bundle management strategies. Computation experiments on two-stage robust optimization and support vector machine demonstrate the effectiveness of the new insights.

Talk 3: An Optimal Single-Loop Algorithm for Convex Finite-Sum Coupled Compositional Stochastic Optimization
Speaker: Tianbao Yang
Abstract: We will talk about a class of convex Finite-Sum Coupled Compositional Stochastic Optimization (cFCCO) problems with many applications, including group distributionally robust optimization (GDRO), learning with imbalanced data, reinforcement learning, and learning to rank. We will present an efficient single-loop primal-dual block-coordinate proximal algorithm, dubbed ALEXR. This algorithm leverages block-coordinate stochastic mirror ascent updates for the dual variable and stochastic proximal gradient descent updates for the primal variable. We establish the convergence rates of ALEXR in both convex and strongly convex cases under smoothness and non-smoothness conditions of involved functions, which not only improve the best rates in previous works on smooth cFCCO problems but also expand the realm of cFCCO for solving more challenging non-smooth problems such as the dual form of GDRO. Finally, we present lower complexity bounds to demonstrate that the convergence rates of ALEXR are optimal among first-order block-coordinate stochastic algorithms for the considered class of cFCCO problems.

Speakers
SS

Sungho Shin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AS

Andy Sun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TY

Tianbao Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2U: Advances in Domain-Specific Languages for Optimization
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Advances in Domain-Specific Languages for Optimization
Chair: Steven Diamond
Cluster: Computational Software

Talk 1: Ten Years of CVXPY
Speaker: William Zhang
Abstract: The CVXPY project has been under development for over ten years. The initial motivation was to reproduce in Python the functionality of the CVX MATLAB package. However, early CVXPY design decisions diverged from CVX, with a greater emphasis on modular problem construction, expression trees, and parameterized problems. We explore how these initial design decisions led to later research on object oriented optimization, matrix-free optimization, and differentiable optimization. We conclude with an overview of upcoming CVXPY features.

Talk 2: CvxLean, a convex optimization modeling framework based on the Lean 4 proof assistant
Speaker: Paul Jackson
Abstract: A proof assistant is a computer environment in which mathematical definitions, theorems and proofs can be expressed, developed and checked in a completely formal way. CvxLean at heart is a realization of the disciplined convex programming paradigm in the proof assistant Lean 4.  As with other DCP frameworks, optimization problems can be initially expressed in a language of atomic functions.  Rules are implemented for automatically inferring problems are convex, and automatic transformations are supported for reducing problems to conic form.  Currently CvxLean uses Mosek to solve these reduced problems. Unlike other frameworks, with CvxLean, the convexity rules and transformation rewrites all must be formally proven to be correct, giving the user extra confidence in the reduction process.  Also, initial problems do not have to be directly expressed in terms of atomic functions and they do not have to satisfy the rules for inferring convexity.  Rather, users can draw on a range of standard definitions available in Lean's mathematical library.  If this is done, standard Lean machinery can be used to formally manipulate problems into recognizably-convex forms using the atomic functions. These manipulations can be tedious to guide, and recent work has explored using e-graph rewriting to discover them automatically.

Talk 3: Disciplined Saddle Programming
Speaker: Philipp Schiele
Abstract: We consider convex-concave saddle point problems, and more generally convex optimization problems we refer to as saddle problems, which include the partial supremum or infimum of convex-concave saddle functions. Saddle problems arise in a wide range of applications, including game theory, machine learning, and finance. It is well known that a saddle problem can be reduced to a single convex optimization problem by dualizing either the convex (min) or concave (max) objectives, reducing a min-max problem into a min-min (or max-max) problem. Carrying out this conversion by hand can be tedious and error prone. In this paper we introduce disciplined saddle programming (DSP), a domain specific language (DSL) for specifying saddle problems, for which the dualizing trick can be automated. The language and methods are based on recent work by Juditsky and Nemirovski, who developed the idea of conic-representable saddle point programs, and showed how to carry out the required dualization automatically using conic duality. Juditsky and Nemirovski's conic representation of saddle problems extends Nesterov and Nemirovski's earlier development of conic representable convex problems; DSP can be thought of as extending disciplined convex programming (DCP) to saddle problems. Just as DCP makes it easy for users to formulate and solve complex convex problems, DSP allows users to easily formulate and solve saddle problems. Our method is implemented in an open-source package, also called DSP.

Speakers
SD

Steven Diamond

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PJ

Paul Jackson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PS

Philipp Schiele

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2V: Nonsmooth stochastic optimization and variational inequalities
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Nonsmooth stochastic optimization and variational inequalities
Chair: Wei Bian
Cluster: Fixed Points and Variational Inequalities

Talk 1: Dynamic Stochastic Approximation Jacobi-Type ADMM Method for Two-stage Stochastic Generalized Nash Equilibrium Problems
Speaker: Hailin Sun
Abstract: This paper studies a specific class of two-stage stochastic generalized Nash equilibrium problems (SGNEPs), where each player engages in a two-stage sequential decision-making process in a random environment: first, they make a decision in the current stage and compete with one another, followed by making a decision in the future stage. This type of two-stage SGNEPs is widely found in fields such as production and manufacturing, transportation logistics, and portfolio management. From the perspective of solving the problem, the main difference between two-stage SGNEPs and single-stage SGNEPs is the need to handle the optimal value function of the second-stage problem, which does not have an explicit expression. At present, there is no effective algorithm to address these challenges. To overcome this difficulty, an accelerated primal-dual method (APDM) is proposed in the paper to obtain an approximate $\epsilon$-subgradient of the second-stage optimal value function, achieving a convergence rate of $O\left(\frac{1}{\sqrt{N}}\right)$. Using this approximate $\epsilon$-subgradient along with a variance reduction technique, a dynamic stochastic approximation Jacobi-type Alternating Direction Method of Multipliers (DSA-JADMM) method is proposed and applied to solve two-stage SGNEPs. This algorithm represents an inexact stochastic version of the Jacobi-type ADMM, as it computes an approximate $\epsilon$-subgradient for the second stage randomly at each iteration using APDM. It is also demonstrated that the algorithm can converge to a weak $\epsilon$-variational equilibrium point of two-stage SGNEPs with a convergence rate of $O\left(\frac{1}{\sqrt{K}}\right)$, which is a special type of Nash equilibrium point. To validate the effectiveness of the DSA-JADMM method, preliminary numerical experiments are conducted. These experiments demonstrate the advantages and superior performance of the proposed method.

Talk 2: Nonsmooth convex-concave saddle point problems with cardinality penalties
Speaker: Wei Bian
Abstract: we focus on a class of convexly constrained nonsmooth convex-concave saddle point problems with cardinality penalties. Although such nonsmooth nonconvex-nonconcave and discontinuous min-max problems may not have a saddle point, we show that they have a local saddle point and a global minimax point, and some local saddle points have the lower bound properties. We define a class of strong local saddle points based on the lower bound properties for stability of variable selection. Moreover we give a framework to construct continuous relaxations of the discontinuous min-max problems based on convolution, such that they have the same saddle points with the original problem. We also establish the relations between the continuous relaxation problems and the original problems regarding local saddle points, global minimax points, local minimax points and stationary points.

Talk 3: AN AUGMENTED LAGRANGIAN METHOD FOR TRAINING RECURRENT NEURAL NETWORKS
Speaker: Chao Zhang
Abstract: Recurrent Neural Networks (RNNs) are widely used to model sequential data in a wide range of areas, such as natural language processing, speech recognition, machine translation, and time series analysis. In this paper, we model the training process of RNNs with the ReLU activation function as a constrained optimization problem with a smooth nonconvex objective function and piecewise smooth nonconvex constraints. We prove that any feasible point of the optimization problem satisfies the no nonzero abnormal multiplier constraint qualification (NNAMCQ), and any local minimizer is a Karush-Kuhn-Tucker (KKT) point of the problem. Moreover, we propose an augmented Lagrangian method (ALM) and design an efficient block coordinate descent (BCD) method to solve the subproblems of the ALM. The update of each block of the BCD method has a closed-form solution. The stop criterion for the inner loop is easy to check and can be stopped in finite steps. Moreover, we show that the BCD method can generate a directional stationary point of the subproblem. Furthermore, we establish the global convergence of the ALM to a KKT point of the constrained optimization problem. Compared with the state-of-the-art algorithms, numerical results demonstrate the efficiency and effectiveness of the ALM for training RNNs.

Speakers
HS

Hailin Sun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WB

Wei Bian

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CZ

Chao Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2W: Variance-Related Issues in Stochastic Optimization Methods
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Variance-Related Issues in Stochastic Optimization Methods
Chair: Yue Wu
Cluster: nan

Talk 1: Some Unified Theory for Variance Reduced Prox-Linear Methods
Speaker: Yue Wu
Abstract: This work considers the nonconvex, nonsmooth problem of minimizing a composite objective of the form f(g(x))+h(x) where the inner mapping g is a smooth finite summation or expectation amenable to variance reduction. In such settings, prox-linear methods can enjoy variance-reduced speed-ups despite the existence of nonsmoothness. We provide a unified convergence theory applicable to a wide range of common variance-reduced vector and Jacobian constructions. Our theory (i) only requires operator norm bounds on Jacobians (whereas prior works used potentially much larger Frobenius norms), (ii) provides state-of-the-art high probability guarantees, and (iii) allows inexactness in proximal computations.

Talk 2: An Accelerated Variance Reduced Extra-Point Approach to Finite-Sum Hemivariational Inequality Problem
Speaker: Kevin Huang
Abstract: In this paper, we develop stochastic variance reduced algorithms for solving a class of finite-sum hemivariational inequality (HVI) problem. In this HVI problem, the associated function is assumed to be differentiable, and both the vector mapping and the function are of finite-sum structure. We propose two algorithms to solve the cases when the vector mapping is either merely monotone or strongly monotone, while the function is assumed to be convex. We show how to apply variance reduction in the proposed algorithms when such an HVI problem has a finite-sum structure, and the resulting accelerated gradient complexities can match the best bound established for finite-sum VI problem, as well as the bound given by the direct Katyusha for finite-sum optimization respectively, in terms of the corresponding parameters such as (gradient) Lipschitz constants and the sizes of the finite-sums. We demonstrate the application of our algorithms through solving a finite-sum constrained finite-sum optimization problem and provide preliminary numerical results. Archival version: https://doi.org/10.48550/arXiv.2211.03269

Talk 3: Adaptive stochastic optimization algorithms for problems with biased oracles
Speaker: Yin Liu
Abstract: Motivated by multiple emerging applications, e.g., stochastic composition optimization, we consider a general optimization problem where the gradient of the objective is only available through a biased stochastic oracle where the bias magnitude can be controlled by a parameter; however, lower bias requires higher computation. Without exploiting a specific bias decay structure, we propose a couple of adaptive and nonadaptive stochastic algorithms to solve the underlying problem. We analyze the nonasymptotic performance of the proposed algorithms in the nonconvex regimes. The numerical performance of the proposed methods over three applications on composition optimization, policy optimization for infinite-horizon Markov decision processes, and distributionally robust optimization will be presented.

Speakers
YW

Yue Wu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KH

Kevin Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Yin Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2X: Proximal Methods and Splitting Algorithms
Monday July 21, 2025 1:15pm - 2:30pm PDT
Session: Proximal Methods and Splitting Algorithms
Chair: Mikhail Solodov
Cluster: nan

Talk 1: Asymmetric Variable Metric Proximal Point Methods, with Relations Between Relative-Errors and Summable-Errors
Speaker: Mikhail Solodov
Abstract: For the problem of finding zeroes of maximally monotone operators, we introduce a general scheme allowing both symmetric and asymmetric metrics in the proximal point algorithms. Our approach subsumes as special cases various proximal point methods in the literature. Moreover, it brings some new insights into the relationship between the hybrid proximal methods with bounded relative-error approximations and inexact proximal methods with error-summability-type conditions. In particular, introducing asymmetric metrics made it possible to cast summable errors in the framework of relative ones, asymptotically. Global convergence is established under standard assumptions. In addition, a certain generalization of Féjer-monotonicity is presented to obtain the local linear convergence rate under the calmness condition on the operator.

Talk 2: A FULL SPLITTING ALGORITHM FOR FRACTIONAL PROGRAMS 1 WITH STRUCTURED NUMERATORS AND DENOMINATORS
Speaker: Min Tao
Abstract: In this paper, we consider a class of nonconvex and nonsmooth fractional programming problems, which involve the sum of a convex, possibly nonsmooth function composed with a linear operator and a differentiable, possibly nonconvex function in the numerator and a convex, possibly nonsmooth function composed with a linear operator in the denominator. These problems have applications in various fields, including CT reconstruction and sparse signal recovery. We propose an adaptive full splitting proximal subgradient algorithm with an extrapolated step that addresses the challenge of evaluating the composition in the numerator by decoupling the linear operator from the nonsmooth component. We specifically evaluate the nonsmooth function using its proximal operator, while the linear operator is assessed through forward evaluations. Furthermore, the smooth component in the numerator is evaluated through its gradient, the nonsmooth component in the denominator is managed using its subgradient, and the linear operator in the denominator is also assessed through forward evaluations. We demonstrate subsequential convergence toward an approximate lifted stationary point and ensure global convergence under the Kurdyka-\L ojasiewicz property, all achieved {\it without relying on any full-row rank assumptions regarding the linear operators}. We provide further discussions on the tightness of the convergence results of the proposed algorithm and its related variants, and also on the reasoning behind aiming for an approximate lifted stationary point. This is exemplified by constructing a scenario illustrating that the algorithm could diverge when seeking exact solutions. Lastly, we present a practical version of the algorithm incorporating a nonmonotone line search, significantly enhancing its convergence performance. Our theoretical findings are validated through simulations involving limited-angle CT reconstruction and the robust Sharpe ratio minimization problem.

Talk 3: Inexact Proximal Point Algorithms for Zeroth-Order Global Optimization
Speaker: Minxin Zhang
Abstract: This work concerns the zeroth-order global minimization of continuous nonconvex functions with a unique global minimizer and possibly multiple local minimizers. We formulate a theoretical framework for inexact proximal point (IPP) methods for global optimization, establishing convergence guarantees under mild assumptions when either deterministic or stochastic estimates of proximal operators are used. The quadratic regularization in the proximal operator and the scaling effect of a positive parameter create a concentrated landscape of an associated Gibbs measure that is practically effective for sampling. The convergence of the expectation under the Gibbs measure is established, providing a theoretical foundation for evaluating proximal operators inexactly using sampling-based methods such as Monte Carlo (MC) integration. In addition, we propose a new approach based on tensor train (TT) approximation. This approach employs a randomized TT cross algorithm to efficiently construct a low-rank TT approximation of a discretized function using a small number of function evaluations, and we provide an error analysis for the TT-based estimation. We then propose two practical IPP algorithms, TT-IPP and MC-IPP. The TT-IPP algorithm leverages TT estimates of the proximal operators, while the MC-IPP algorithm employs MC integration to estimate the proximal operators. Both algorithms are designed to adaptively balance efficiency and accuracy in inexact evaluations of proximal operators. The effectiveness of the two algorithms is demonstrated through experiments on diverse benchmark functions and various applications. Reference: Zhang, M., Han, F., Chow, Y. T., Osher, S., & Schaeffer, H. (2024). Inexact Proximal Point Algorithms for Zeroth-Order Global Optimization. arXiv preprint arXiv:2412.11485.

Speakers
MS

Mikhail Solodov

IMPA
http://www.impa.br/~optim/solodov.html
avatar for Min Tao

Min Tao

Professor, Nanjing University
Tao Min's primary research interests lie in the theory and applications of optimization algorithms, with a particular focus on first-order methods and their applications in machine learning. Her representative works have been published in leading journals such as the SIAM Journal... Read More →
MZ

Minxin Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 2Y
Monday July 21, 2025 1:15pm - 2:30pm PDT
Monday July 21, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

2:30pm PDT

Coffee & Snack Break (Provided)
Monday July 21, 2025 2:30pm - 3:00pm PDT
Monday July 21, 2025 2:30pm - 3:00pm PDT
TBA

3:00pm PDT

Parallel Semi-Plenary Talk 1A
Monday July 21, 2025 3:00pm - 4:00pm PDT
Speakers
LX

Lin Xiao

Lin Xiao is a Research Scientist at Facebook AI Research (FAIR) in Seattle, Washington. He received BE from Beijing University of Aeronautics and Astronautics (Beihang University) and PhD from Stanford University, and was a postdoctoral fellow in the Center for the Mathematics of... Read More →
Monday July 21, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

3:00pm PDT

Parallel Semi-Plenary Talk 1B
Monday July 21, 2025 3:00pm - 4:00pm PDT
Speakers
JB

Jérôme Bolte

Jérôme Bolte is a Full Professor at the Toulouse School of Economics and holds a Chair in Artificial Intelligence at the Artificial Intelligence Institute of Toulouse (ANITI). He studied pure and applied mathematics before completing a degree in mathematics and then a doctorate... Read More →
Monday July 21, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:00pm PDT

Break
Monday July 21, 2025 4:00pm - 4:15pm PDT
Monday July 21, 2025 4:00pm - 4:15pm PDT
TBA

4:15pm PDT

Parallel Sessions 3A: Advances in Solving Large-Scale Problems: Accelerated Methods and Sharp Analyses (I)
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Advances in Solving Large-Scale Problems: Accelerated Methods and Sharp Analyses (I)
Chair: Sen Na Liwei Jiang
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Implicit Preconditioning in Stochastic Linear System Solvers
Speaker: Michał Dereziński
Abstract: In this talk, I will present new algorithms and convergence guarantees for solving linear systems via sketch-and-project, a framework which unifies many known iterative methods that use randomized sub-sampling and sketching, including randomized Kaczmarz, coordinate descent, and others. Our new results uncover a connection between stochastic iterative solvers and sketching-based randomized preconditioning algorithms: Whenever the spectral structure of a linear system is amenable to constructing a strong preconditioner via low-rank approximation, then one can construct a stochastic solver based on sketch-and-project that will implicitly take advantage of this spectral structure. In particular, I will show how this leads to solving an n x n linear system with at most k large (outlying) singular values in ~O( n^2 + nk^2 ) arithmetic operations, which is faster than the ~O( n^2 k ) cost of constructing a good preconditioner for a deterministic iterative solver such as conjugate gradient.

Talk 2: The radius of statistical efficiency
Speaker: Mateo Díaz
Abstract: Classical results in asymptotic statistics show that the Fisher information matrix controls the difficulty of estimating a statistical model from observed data. In this work, we introduce a companion measure of robustness of an estimation problem: the radius of statistical efficiency (RSE) is the size of the smallest perturbation to the problem data that renders the Fisher information matrix singular. We compute RSE up to numerical constants for a variety of test bed problems, including principal component analysis, generalized linear models, phase retrieval, bilinear sensing, and matrix completion. In all cases, the RSE quantifies the compatibility between the covariance of the population data and the latent model parameter. Interestingly, we observe a precise reciprocal relationship between RSE and the intrinsic complexity/sensitivity of the problem instance, paralleling the classical Eckart–Young theorem in numerical analysis.

Talk 3: Low rank approximation for faster optimization
Speaker: Madeleine Udell
Abstract: Low rank structure is pervasive in real-world datasets. This talk shows how to accelerate the solution of fundamental computational problems, including eigenvalue decomposition, linear system solves, composite convex optimization, and stochastic optimization (including deep learning), by exploiting this low rank structure. We present a simple method based on randomized numerical linear algebra for efficiently computing approximate top eigendecompositions, which can be used to replace large matrices (such as Hessians and constraint matrices) with low rank surrogates that are faster to apply and invert. The resulting solvers for linear systems (NystromPCG), composite convex optimization (NysADMM), and stochastic optimization (SketchySGD and PROMISE) demonstrate strong theoretical and numerical support, outperforming state-of-the-art methods in terms of speed and robustness to hyperparameters.

Speakers
MD

Michał Dereziński

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MD

Mateo Díaz

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Madeleine Udell

Madeleine Udell

Postdoctoral Fellow, Caltech Center for the Mathematics of Information
Madeleine Udell is a postdoctoral fellow at Caltech's Center for the Mathematics of Information, hosted by Joel Tropp. She will be joining Cornell as an Assistant Professor in the School of Operations Research and Information Engineering in July 2016. Her research focus is on modeling... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3B: Deterministic and Stochastic Methods for Optimization and Games- Part II
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Deterministic and Stochastic Methods for Optimization and Games- Part II
Chair: Angelia Nedich
Cluster: Multi-agent Optimization and Games

Talk 1: Frequentist Guarantees of Distributed (Non)-Bayesian Inference
Speaker: Bohan Wu
Abstract: Motivated by the need to analyze large, decentralized datasets, distributed Bayesian inference has become a critical research area across multiple fields, including statistics, electrical engineering, and economics. This paper establishes Frequentist properties, such as posterior consistency, asymptotic normality, and posterior contraction rates, for the distributed (non-)Bayes Inference problem among agents connected via a communication network. Our results show that, under appropriate assumptions on the communication graph, distributed Bayesian inference retains parametric efficiency while enhancing robustness in uncertainty quantification. We also explore the trade-off between statistical efficiency and communication efficiency by examining how the design and size of the communication graph impact the posterior contraction rate. Furthermore, We extend our analysis to time-varying graphs and apply our results to exponential family models, distributed logistic regression, and decentralized detection models.

Talk 2: Decentralized high-dimensional inference over mesh networks: a unified perspective
Speaker: Marie Maros
Abstract: We consider the problem of solving high-dimensional statistical inference problems over a network of agents (with no coordinating agent) who have exclusive access to a fraction of the total available samples. In the high-dimensional setting, the problem dimension is much larger than the total number of available samples, making the problem ill-conditioned. Despite this, we empirically observe that obtaining a statistically meaningful solution is possible with many existing decentralized schemes, given that the underlying parameter to estimate lies in a low dimensional subspace. Our observations challenge the existing theories in two key ways: (i) most decentralized schemes do not break down as the problem dimensionality increases, and (ii) decentralized schemes that are expected to behave like one another behave very differently in high dimensions. To understand the behavior of decentralized optimization methods in high-dimensional inference we introduce a unified framework and analysis, allowing us to develop an understanding of the mechanisms enabling dimension independent performance of decentralized schemes.

Talk 3: Toward Parameter-free Decentralized Optimization
Speaker: Gesualdo Scutari
Abstract: We study the minimization of (locally strongly) convex, locally smooth functions over a network of agents without a centralized server. Existing decentralized algorithms require knowledge of problem and network parameters, such as the Lipschitz constant of the global gradient and/or network connectivity, for hyperparameter tuning. Agents usually cannot access this information, leading to conservative selections, slow convergence, or divergence. We introduce a decentralized algorithm that eliminates the need for specific parameter tuning. Our approach employs an operator splitting technique with a novel variable metric, enabling a local backtracking line-search to adaptively select the stepsize without global information or extensive communications. This results in favorable convergence guarantees and dependence on optimization and network parameters compared to existing nonadaptive methods. Theoretical findings are supported by numerical experiments.

Speakers
AN

Angelia Nedich

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MM

Marie Maros

Assistant Professor, Texas A&M University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GS

Gesualdo Scutari

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3C: Methods for Large-Scale Nonlinear Optimization III
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Methods for Large-Scale Nonlinear Optimization III
Chair: Baoyu Zhou
Cluster: Nonlinear Optimization

Talk 1: A Two Stepsize SQP Method for Nonlinear Equality Constrained Stochastic Optimization
Speaker: Michael O'Neill
Abstract: We develop a Sequential Quadratic Optimization (SQP) algorithm for minimizing a stochastic objective function subject to deterministic equality constraints. The method utilizes two different stepsizes, one which exclusively scales the component of the step corrupted by the variance of the stochastic gradient estimates and a second which scales the entire step. We prove that this stepsize splitting scheme has a worst-case complexity result which improves over the best known result for this class of problems. In terms of approximately satisfying the constraint violation, this complexity result matches that of deterministic SQP methods, up to constant factors, while matching the known optimal rate for stochastic SQP methods to approximately minimize the norm of the gradient of the Lagrangian. We also propose and analyze multiple variants of our algorithm. One of these variants is based upon popular adaptive gradient methods for unconstrained stochastic optimization while another incorporates a safeguarded line search along the constraint violation. Preliminary numerical experiments show competitive performance against a state of the art stochastic SQP method. In addition, in these experiments, we observe an improved rate of convergence in terms of the constraint violation, as predicted by the theoretical results.

Talk 2: A Proximal-Stochastic-Gradient Method for Regularized Equality Constrained Problems
Speaker: Daniel P. Robinson
Abstract: I present an algorithm of the proximal-stochastic-gradient variety for minimizing the sum of a nonconvex loss function and a convex regularization function subject to nonlinear equality constraints. Motivation for the algorithm is provided, along with a theoretical analysis and preliminary numerical results.

Talk 3: Randomized Feasibility-Update Algorithms For Variational Inequality Problems
Speaker: Abhishek Chakraborty
Abstract: This paper considers a variational inequality (VI) problem arising from a game among multiple agents, where each agent aims to minimize its own cost function subject to its constrained set represented as the intersection of a (possibly infinite) number of convex functional level sets. A direct projection-based approach or Lagrangian-based techniques for such a problem can be computationally expensive if not impossible to implement. To deal with the problem, we consider randomized methods that avoid the projection step on the whole constraint set by employing random feasibility updates. In particular, we propose and analyze such random methods for solving VIs based on the projection method, Korpelevich method, and Popov method. We establish the almost sure convergence of the methods and, also, provide their convergence rate guarantees. We illustrate the performance of the methods in simulations for two-agent games.

Speakers
BZ

Baoyu Zhou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MO

Michael O'Neill

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Daniel P. Robinson

Daniel P. Robinson

Name: Daniel P. RobinsonTitle: Associate ProfessorAffiliation: Lehigh UniversityBio:Daniel P. Robinson received his Ph.D. from the University of California at San Diego in 2007. He spent the next three years working with Nicholas I. M. Gould and Jorge Nocedal as a Postdoctoral Researcher... Read More →
avatar for Abhishek Chakraborty

Abhishek Chakraborty

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3D: Semidefinite Programming - Theory and Algorithms
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Semidefinite Programming - Theory and Algorithms
Chair: Hao Hu
Cluster: Conic and Semidefinite Optimization

Talk 1: Tight error bounds for log-determinant cones without constraint qualifications
Speaker: Ting Kei Pong
Abstract: Log-determinant cone is the closure of the hypograph of the perspective function of the log-determinant function, and can be viewed as a spectral analogue of the exponential cone. In this talk, without requiring any constraint qualifications, we establish tight error bounds for the log-determinant cone. This error bound is obtained using the recently developed framework based on one-step facial residual functions, which has been successfully applied to deriving error bounds for the exponential cone. This is a joint work with Ying Lin, Scott B. Lindstrom and Bruno F. Lourenço.

Talk 2: Primal-dual interior point algorithms for nonsymmetric convex conic optimization
Speaker: Anita Varga
Abstract: In this talk, we present primal-dual interior point methods for nonsymmetric conic optimization. The proposed methods are of the path following type; we discuss different strategies for initialization and different neighborhoods to ensure fast convergence. We will also discuss some applications including sums-of-squares optimization, where the proposed methods can outperform the conventional semidefinite programming approach. We also share numerical results to illustrate the practical performance of our algorithms.

Talk 3: Ramana's exact dual for semidefinite programming, and elementary row operations
Speaker: Gabor Pataki
Abstract: Thirty years ago, in a seminal paper Ramana derived an exact dual for Semidefinite Programming (SDP). Ramana's dual has the following remarkable features: i) it assumes feasibility of the primal, but it does not make any regularity assumptions, such as strict feasibility ii) its optimal value is the same as the optimal value of the primal, so there is no duality gap. iii) it attains its optimal value when it is finite iv) it yields a number of complexity results in SDP, which are fundamental, and to date are still the best known. For example, it proves that SDP feasibility in the Turing model is not NP-complete, unless NP = co-NP. In this work we give a fairly complete analysis of Ramana's dual. First, we connect it to a seemingly very different way of inducing strong duality: reformulating the SDP into a rank revealing form using mostly elementary row operations. Second, we completely characterize the feasible set of Ramana's dual. As a corollary, we obtain a short and transparent derivation of Ramana's dual, which we hope is accessible to both the optimization and the theoretical computer science community. Our approach is combinatorial in the following sense: i) we use a minimum amount of continuous optimization theory ii) we show that feasible solutions in Ramana's dual are identified with regular facial reduction sequences, i.e., essentially discrete structures.

Speakers
TK

Ting Kei Pong

Professor, The Hong Kong Polytechnic University
Name: Dr. Ting Kei PONGTitle: ProfessorAffiliation: The Hong Kong Polytechnic UniversityBio: Please see my webpageFun Fact: :)
AV

Anita Varga

Postdoctoral Researcher, North Carolina State University
Name: Dr.Anita VargaTitle: Postdoctoral ResearcherAffiliation: North Carolina State UniversityBio:
GP

Gabor Pataki

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3E: Efficient Optimization Methods for LLMs (Part II)
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Efficient Optimization Methods for LLMs (Part II)
Chair: Xiao Li
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: BAdam: A Memory Efficient Block Coordinate Descent Method for LLM Training
Speaker: Xiao Li
Abstract: TBD

Talk 2: Pretraining of an LLM at KAUST: Mistakes and Learned Lessons
Speaker: Francesco Orabona
Abstract: TBD

Talk 3: Benchmarking Neural Network Training Algorithms
Speaker: Frank Schneider
Abstract: TBD

Speakers
avatar for Xiao Li

Xiao Li

Assistant Professor, The Chinese University of Hong Kong, Shenzhen
Name: Dr. Xiao LiTitle: Assistant Professor of School of Data ScienceAffiliation: The Chinese University of Hong Kong, ShenzhenBio:I work on (continuous) optimization. Currently, my specific focus is on developing efficient optimization methods for large language models.
FO

Francesco Orabona

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Frank Schneider

Frank Schneider

PostDoc, University of Tübingen
Name: Dr. Frank SchneiderTitle: Postdoctoral ResearcherAffiliation: University of Tübingen, GermanyBio:I’m a postdoctoral researcher at the University of Tübingen in the Methods of Machine Learning group, led by Philipp Hennig. I’m also a chair of the Algorithms working... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3F: Advances in commercial solvers
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Advances in commercial solvers
Chair: Robert Luce
Cluster: Computational Software

Talk 1: A Whole New Look for CONOPT
Speaker: Steven Dirkse
Abstract: Following GAMS' recent acquisition of CONOPT from ARKI Consulting & Development A/S, this presentation delves into the continuous evolution of this robust nonlinear optimization solver, emphasizing the significant advancements introduced in the latest release and the strategic implications of the new ownership. Recent updates have optimized the active set method at CONOPT’s core, leading to measurable performance improvements across a diverse set of test cases. These enhancements boost computational efficiency, stability and accuracy. The latest iteration of CONOPT introduces new APIs for C++ and Python, opening up new possibilities for a clean, efficient, and robust integration into various software environments and projects requiring nonlinear optimization. Finally, we will demonstrate the practical application of providing derivatives to CONOPT, an important step that is often necessary to achieve the best possible performance.

Talk 2: Continuous Optimization in MATLAB
Speaker: Shengchao Lin
Abstract: This presentation highlights MATLAB's optimization capabilities. Key features include the broad applications of the Optimization and Global Optimization Toolboxes, problem-based optimization setup for better modeling, and code generation and deployment of optimization algorithms. These improvements demonstrate MATLAB's evolving role as a powerful platform for optimization and modeling in engineering and science.

Talk 3: The Latest Developments in the Knitro Optimization Solver
Speaker: Richard Waltz
Abstract: Knitro was originally developed in the 1990s as an interior-point algorithm for general nonlinear, non-convex optimization.  Over the years, Knitro evolved into a more general optimization toolbox that includes an Active-Set LP-based solver, a Sequential Quadratic Programming (SQP) solver, specialized LP, QP, and SOCP solvers, a branch-and-bound solver for mixed integer programming, and multi-start heuristics for global optimization.  To add to this toolbox of algorithms, we have recently started developing first-order methods that do not require any matrix factorizations.  The hope is that these might be able to provide useful solutions to extremely large-scale models where the factorizations in interior-point methods become too expensive.  In this talk we will present some of this work, primarily based on Augmented Lagrangian (AL) type methods.

Speakers
RL

Robert Luce

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SD

Steven Dirkse

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SL

Shengchao Lin

Software Engineer, MathWorks
RW

Richard Waltz

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3G: Special Session in Honor of Suvrajeet Sen: Sampling-based Methods in Stochastic Programming
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Special Session in Honor of Suvrajeet Sen: Sampling-based Methods in Stochastic Programming
Chair: Harsha Gangammanavar
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: A Stochastic Decomposition Method for Multistage Distributionally Robust Optimization under Streaming Data
Speaker: Tian Xia
Abstract: TBD

Talk 2: TBD
Speaker: Di Zhang
Abstract: TBD

Talk 3: TBD
Speaker: Harsha Gangammanavar
Abstract: TBD

Speakers
DZ

Di Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HG

Harsha Gangammanavar

Associate Professor, Southern Methodist University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3H: Recent Advances in Large-scale Optimization I
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Recent Advances in Large-scale Optimization I
Chair: Salar Fattahi
Cluster: Nonlinear Optimization

Talk 1: Algorithms for nonconvex nonsmooth optimization
Speaker: Jong-Shi Pang
Abstract: This talk will be focused on algorithms for nonconvex nonsmooth optimization.

Talk 2: Discrete Optimization methods for compressing foundation models
Speaker: Rahul Mazumder
Abstract: Foundation models have achieved remarkable performance across various domains, but their large model sizes lead to high computational costs (storage, inference latency, memory, etc). Neural network pruning, roughly categorized as unstructured and structured, aims to reduce these costs by removing less-important parameters while retaining model utility as much as possible. Depending upon available hardware, different types of pruning approaches are useful. In this talk, I will discuss discrete optimization methods to address such problems. At a high-level, these are related to cardinality constrained least squares problems involving billions of variables; and require the development of large-scale algorithms that can run on GPUs.

Talk 3: A Parametric Approach for Solving Convex Quadratic Optimization with Indicators Over Trees
Speaker: Salar Fattahi
Abstract: In this talk, we discuss convex quadratic optimization problems involving indicator variables, each associated with a continuous variable, particularly focusing on scenarios where the matrix defining the quadratic term is positive definite and its sparsity pattern corresponds to the adjacency matrix of a tree graph. We introduce a graph-based dynamic programming algorithm that solves this problem in quadratic time and memory. Central to our algorithm is a precise parametric characterization of the cost function across various nodes of the graph corresponding to distinct variables. Our computational experiments conducted on both synthetic and real-world datasets demonstrate the superior performance of our proposed algorithm compared to existing algorithms and state-of-the-art mixed-integer optimization solvers. An important application of our algorithm is in the real-time inference of Gaussian hidden Markov models from data affected by outlier noise. Using a real on-body accelerometer dataset, we solve instances of this problem with over 30,000 variables in under a minute, and its online variant within milliseconds on a standard computer.

Speakers
JP

Jong-Shi Pang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RM

Rahul Mazumder

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SF

Salar Fattahi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3I: Large-scale optimization for data science
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Large-scale optimization for data science
Chair: Jason Altschuler
Cluster: Optimization For Data Science

Talk 1: Preconditioning for Linear Regression and Kernel Methods: The State of Play
Speaker: Ethan Epperly
Abstract: Simple models like linear regression and kernel methods continue to be powerful tools for learning from data. For large-scale problems, the state-of-art algorithms for these models use iterative methods with randomized preconditioning. This talk surveys the best-known preconditioners for these models, discusses recent advances, and describes open problems.

Talk 2: Balancing Regret and Runtime: Faster Iterative Projections over Submodular Base Polytopes
Speaker: Jai Moondra
Abstract: Optimization algorithms like projected Newton’s method, FISTA, and Mirror Descent achieve near-optimal regret bounds (e.g., O(sqrt(T)) for Online Mirror Descent) but face high computational costs due to Bregman projections at each iteration. By contrast, conditional gradient methods perform linear optimization at each step, achieving faster runtimes but at the expense of suboptimal regret bounds (e.g., O(T^⅔) for Online Frank-Wolfe). Motivated by this runtime-regret trade-off, we propose efficient iterative projection techniques for closely spaced points over submodular base polytopes, a widely applicable structure. Our approach, using both continuous and discrete perspectives, leads to significant runtime improvements in Online Mirror Descent, achieving up to several orders of magnitude in speed-ups in numerical experiments. For cardinality-based submodular polytopes, we further reduce Bregman projection costs by a factor of Omega(n/log n) in n-dimensions. Joint work with Hassan Mortagy and Swati Gupta.

Talk 3: TBD
Speaker: Babak Hassibi
Abstract: TBD

Speakers
JA

Jason Altschuler

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
EE

Ethan Epperly

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JM

Jai Moondra

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BH

Babak Hassibi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3J: Frontiers of Optimization for Machine Learning - Part III
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Frontiers of Optimization for Machine Learning - Part III
Chair: Fred Roosta
Cluster: Nonlinear Optimization

Talk 1: Randomized Techniques for Fast and Scalable Operator
Speaker: Michael Mahoney
Abstract: The approximation of linear operators and their inverses lies at the heart of many optimization and machine learning algorithms. This work improves the efficiency and scalability of these tasks by integrating two central computational tools of recent decades-- randomization and preconditioning. A particular focus is placed on addressing large-scale applications on modern hardware architectures with stringent communication and memory constraints. Notably, the proposed methods are designed to enable effective recycling of computations when dealing with sequences of operators with similar intrinsic properties, a scenario frequently arising in iterative optimization algorithms.

Talk 2: Approximation and Preconditioning
Speaker: Matus Telgarsky
Abstract: This talk will demonstrate a duality-based proof technique for establishing that coordinate and gradient descent follow specific paths (and not just limit points) for linear and deep network classifiers.

Talk 3: Example Selection for Distributed Learning
Speaker: Christopher de Sa
Abstract: Training example order in SGD has long been known to affect convergence rate. Recent results show that accelerated rates are possible in a variety of cases for permutation-based sample orders, in which each example from the training set is used once before any example is reused. This talk will cover a line of work in my lab on decentralized learning and sample-ordering schemes. We will discuss the limits of the classic gossip algorithm and random-reshuffling schemes and explore how both can be improved to make SGD converge faster both in theory and in practice with little overhead.

Speakers
FR

Fred Roosta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MM

Michael Mahoney

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MT

Matus Telgarsky

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CD

Christopher de Sa

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3K: Recent advances in federated learning
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Recent advances in federated learning
Chair: Minseok Ryu
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: FedSpaLLM: Federated Pruning of Large Language Models
Speaker: Yijiang Li
Abstract: Federated Learning (FL) has gained significant interest in training AI models in a distributed computing environment benefiting from its capability to maintain the privacy of sensitive data of the participating parties. However, challenges remain in effectively handling of participating parties with heterogeneous computational power, such as edge devices. In this work, we propose a federated framework that involves an adaptive global pruning scheme to enable collaborative training of large models, such as LLMs, on parties with heterogeneous computational power.

Talk 2: Balancing uneven client participation in asynchronous Federated Learning
Speaker: Charikleia Iakovidou
Abstract: Asynchronous communication is a popular approach for speeding up the convergence of Federated Learning (FL) in the presence of slowly updating clients. Existing asynchronous FL methods typically provide convergence guarantees under the assumption that each client is equally likely to participate in a global aggregation. In practice, however, due to variations in computation or communication capabilities, clients may update the server at different frequencies. We demonstrate theoretically that under uneven client participation and non-iid local data, vanilla asynchronous FedAvg cannot achieve convergence to an arbitrarily small neighborhood of the optimum of the global loss function, even when a diminishing stepsize sequence is adopted. We introduce AREA, a new asynchronous FL method that employs a memory-based correction mechanism for balancing uneven client participation, and supports a wide variety of deterministic and stochastic aggregation protocols. Without the strong assumptions of bounded maximum client delay and bounded gradients, we establish theoretically optimal convergence rates for AREA for (i) strongly convex and smooth functions, (ii) convex and nonsmooth functions, and (iii) nonconvex and smooth functions.

Talk 3: Federated Communication-Efficient Multi-Objective Optimization
Speaker: Baris Askin
Abstract: We study a federated version of multi-objective optimization (MOO), where a single model is trained to optimize multiple objective functions. MOO has been extensively studied in the centralized setting but is less explored in federated or distributed settings. We propose FedCMOO, a novel communication-efficient federated multi-objective optimization (FMOO) algorithm that improves the error convergence performance of the model compared to existing approaches. Unlike prior works, the communication cost of FedCMOO does not scale with the number of objectives, as each client sends a single aggregated gradient, obtained using randomized SVD (singular value decomposition), to the central server. We provide a convergence analysis of the proposed method for smooth non-convex objective functions under milder assumptions than in prior work. In addition, we introduce a variant of FedCMOO that allows users to specify a preference over the objectives in terms of a desired ratio of the final objective values. Through extensive experiments, we demonstrate the superiority of our proposed method over baseline approaches.

Speakers
YL

Yijiang Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CI

Charikleia Iakovidou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BA

Baris Askin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3L: Bilevel Optimization for Inverse Problems Part 1
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Bilevel Optimization for Inverse Problems Part 1
Chair: Juan Carlos de los Reyes
Cluster: PDE-constrained Optimization

Talk 1: A descent algorithm for the optimal control of ReLU neural network informed PDEs based on approximate directional derivatives
Speaker: Michael Hintermuller
Abstract: We propose and analyze a numerical algorithm for solving a class of optimal control problems for learning-informed semilinear partial differential equations. The latter is a class of PDEs with constituents that are in principle unknown and are approximated by nonsmooth ReLU neural networks. We first show that a direct smoothing of the ReLU network with the aim to make use of classical numerical solvers can have certain disadvantages, namely potentially introducing multiple solutions for the corresponding state equation. This motivates us to devise a numerical algorithm that treats directly the nonsmooth optimal control problem, by employing a descent algorithm inspired by a bundle-free method. Several numerical examples are provided and the efficiency of the algorithm is shown.

Talk 2: Differential estimates for fast first-order multilevel nonconvex optimisation
Speaker: Tuomo Valkonen
Abstract: PDE constraints appear in inverse imaging problems as physical models for measurements, while bilevel optimisation can be used for optimal experimental design and parameter learning. Such problems have been traditionally very expensive to solve, but recently, effective single-loop approaches have been introduced, both in our work, as well as in the machine learning community. In this talk, we discuss a simple gradient estimation formalisation for very general single-loop methods that include primal-dual methods for the inner problem, and conventional iterative solvers (Jacobi, Gauss–Seidel, conjugate gradients) for the adjoint problem and PDE constraints.

Talk 3: Deep Equilibrium Models for Poisson Inverse Problems via Mirror Descent
Speaker: Christian Daniele
Abstract: Inverse problems in imaging arise in a wide range of scientific and engineering applications, including medical imaging, astrophysics, and microscopy. These problems are inherently ill-posed, requiring advanced regularization techniques and optimization strategies to achieve stable and accurate reconstructions. In recent years, hybrid approaches that combine deep learning and variational methods have gained increasing attention. Well-established techniques include Algorithmic Unrolling, Plug-and-Play methods, and Deep Equilibrium Models. The latter are networks with fixed points, which are trained to match data samples from a training dataset. In this work, we focus on Deep Equilibrium Models to learn a data-driven regularization function for Poisson inverse problems, using the Kullback-Leibler divergence as the data fidelity term. To effectively handle this fidelity term, we employ Mirror Descent as the underlying optimization algorithm. We discuss theoretical guarantees of convergence, even in non-convex settings, incorporating a backtracking strategy, along with key aspects of training this class of models. To validate our approach, we evaluate its performance on a deblurring task with different kernels and varying levels of Poisson noise. Authors: Luca Calatroni, Silvia Villa, Samuel Vaiter, Christian Daniele In this work, we focus on Deep Equilibrium Models to learn a data-driven regularization function for Poisson inverse problems, using the Kullback-Leibler divergence as the data fidelity term. To effectively handle this fidelity term, we employ Mirror Descent as the underlying optimization algorithm. We discuss theoretical guarantees of convergence, even in non-convex settings, incorporating a backtracking strategy, along with key aspects of training this class of models. To validate our approach, we evaluate its performance on a deblurring task with different kernels and varying levels of Poisson noise.

Speakers
JC

Juan Carlos de los Reyes

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MH

Michael Hintermuller

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TV

Tuomo Valkonen

MODEMAT & University of Helsinki
Nonsmooth optimisation, bilevel optimisation, inverse problems, variational analysis, optimisation in measure spaces. 
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3M: Optimization Applications in Energy Systems
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Optimization Applications in Energy Systems
Chair: Sungho Shin
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Multiperiod Optimization for Power Grid Applications
Speaker: Mihai Anitescu
Abstract: There has been a growing interest in solving multi-period AC OPF problems, as the increasingly fluctuating electricity market requires operation planning over multiple periods. These problems, formerly deemed intractable, are now becoming technologically feasible to solve thanks to the advent of high-memory GPU hardware and accelerated NLP tools. This study evaluates the capability of the ExaModels.jl and MadNLP.jl tools for GPU-centered nonlinear programming to tackle previously unsolvable multi-period AC OPF instances. Our numerical experiments, run on an NVIDIA GH200, demonstrate that we can solve a multi-period OPF instance with more than 10 million variables up to 10−4 precision in less than 10 minutes.

Talk 2: Optimal Power Flow Under Constraint-Informed Uncertainty
Speaker: Anirudh Subramanyam
Abstract: Chance-constrained optimization has emerged as a promising framework for managing uncertainties in power systems. This work advances its application to DC Optimal Power Flow (DC-OPF) problems, developing a novel approach to uncertainty modeling. Current methods tackle these problems by first modeling random variables using high-dimensional statistical distributions that scale with the number of system buses, followed by deriving convex reformulations of the probabilistic constraints. We propose an alternative methodology that uses the probabilistic constraints themselves to inform the structure of uncertainty, enabling significant dimensionality reduction. Rather than learning joint distributions of wind generation forecast errors across all units, we model two key distributions: system-wide aggregate forecast errors and individual unit errors weighted by transmission line flow sensitivities. We evaluate our approach under both Gaussian and non-Gaussian uncertainty distributions, demonstrating improvements over state-of-the-art in both statistical accuracy and optimization performance.

Talk 3: Characterizing marginal value of storage in distribution grid operations
Speaker: Dirk Lauinger
Abstract: Electricity distribution companies invest in storage to shave peak load and reduce investments into substation and distribution line upgrades. In deregulated electricity markets, storage assets owned by distribution companies are not allowed to participate in electricity markets, which leads to the assets sitting idle most of the time. Contracting for storage could provide investors with additional value streams, distribution companies with cheaper storage, and rate payers with reduced prices. We integrate contracted storage into distribution company investment planning problems and find that peak shaving reduces profits from market participation by about 1% in a Massachusetts case study. Capital investment savings from contracted storage more than compensate for this reduction. Both distribution companies and storage investors could thus benefit from contracted storage.

Speakers
SS

Sungho Shin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MA

Mihai Anitescu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AS

Anirudh Subramanyam

Assistant Professor, Pennsylvania State University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DL

Dirk Lauinger

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3N: Adjustable Robust Optimization: Theories and Algorithms
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Adjustable Robust Optimization: Theories and Algorithms
Chair: Ahmadreza Marandi
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: A semi-infinite Benders’ cut approach for adjustable robust optimization
Speaker: Ayse Nur Arslan
Abstract: In this talk we consider two-stage linear adjustable robust optimization problems with continuous recourse. These problems have been the subject of exact solution algorithms, notably, Benders decomposition and constraint-and-column generation (CCG) approaches. Here, we present an alternative decomposition approach reposing on a novel reformulation of the problem using semi-infinite Benders’ cuts. We argue that this approach will enjoy the same quality of dual bounds as the CCG approach while requiring to solve a smaller number of separation problems. We additionally study the formulation and solution of separation problems under different assumptions on the form of the uncertainty set and the feasibility of the recourse problem. We perform a detailed numerical study that showcases the superior performance of our proposed approach as well as compares the performances of different formulations for the separation problem.

Talk 2: Robust Bilevel Optimization with Wait-and-See Follower: A Column-and-Constraint Generation Approach
Speaker: Henri Lefebvre
Abstract: Bilevel optimization is a classical framework for modeling hierarchical decision-making processes. Typically, it is assumed that all input parameters for both the leader and the follower are known when the leader makes a decision. However, in many real-world applications, the leader must decide without fully anticipating the follower's response due to uncertainties in the follower's problem. In this talk, we address robust bilevel optimization problems in which the follower adopts a ``wait-and-see'' approach. Thus, the leader decides without knowledge of the explicit realization of the uncertainty, then the uncertainty realizes in a worst-case manner, and afterward the follower's decisions are made. For this challenging problem class, we discuss mathematical properties and present a corresponding solution approach based on column-and-constraint generation. The convergence of the proposed algorithm is discussed along with its practical implementation including numerical results. We finally outline potential research directions.

Talk 3: The Value of Flexibility in Robust Supply Chain Network Design
Speaker: Amir Ardestani-Jaafari
Abstract: A supply chain network design problem (SCNDP) involves making long-term and mostly irreversible strategic decisions, requiring the utilization of various sources of flexibility and demand information. The cost efficiency of this process hinges, to a large extent, on how, among other factors, flexibilities from strategic and operational perspectives are tailored and how demand information is leveraged. In this paper, we investigate five distinct policies for our SCNDP, stemming from incorporating new flexibilities at both the strategic and operational levels. We commence with a localized production model where local production satisfies the demand. We then extend the model to cases where the production capacity in one location can be utilized to meet the demand of other nodes (Policy II). Furthermore, the capacity can be shared among open facilities if \textit{capacity-sharing links} have already been arranged (Policy III). In contrast to these policies, where the set capacity in the first stage serves as a surrogate for production amount in the second stage, we allow production decisions to be postponed until after the realization of demand, leading to a make-to-order rather than a make-to-stock production strategy (Policies IV and V). To analyze each of these policies, we develop a two-stage robust optimization framework for which we introduce a novel computationally efficient exact (based on Column-and-Constraint Generation (C\&CG)) and multiple approximation techniques (based on Linear Decision Rules). These techniques effectively solve realistically sized instances of the problem under varying demand uncertainty budgets, enabling us to derive managerial insights that would otherwise be unattainable. We demonstrate, among other results, that (i) the existence of capacity-sharing links among facilities yields (a) the highest percentage of cost-saving due to the pooling effect for all uncertainty budget values (b) significantly reduces shortage probability and (ii) production postponement brings only a marginal improvement in the results, suggesting that upstream supply chain connections (capacity-sharing links among facilities) are more critical than postponing production decisions, especially for moderate budgets of uncertainty. Finally, we apply our most effective policies to a real-world case study to contextualize these concepts, quantify their values, and formulate our design recommendations.

Speakers
AN

Ayse Nur Arslan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Henri Lefebvre

Henri Lefebvre

Post-doc, Trier University
Henri is currently a post-doctoral researcher at Trier University (Germany) in the "Nonlinear Optimization" group. He earned his PhD at the University of Bologna (DEI) under the supervision of Michele Monaci and Enrico Malaguti where he studied optimization problems under uncertainty... Read More →
AA

Amir Ardestani-Jaafari

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3O: Advances in Discrete Optimization: From Relaxation to Convex Hull Exactness
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Advances in Discrete Optimization: From Relaxation to Convex Hull Exactness
Chair: Akang Wang
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: ROS: A GNN-based Relax-Optimize-and-Sample Framework for Max-k-Cut Problems
Speaker: Yeqing Qiu
Abstract: The Max-k-Cut problem is a fundamental combinatorial optimization challenge that generalizes the classic NP-complete Max-Cut problem. While relaxation techniques are commonly employed to tackle Max-k-Cut, they often lack guarantees of equivalence between the solutions of the original problem and its relaxation. To address this issue, we introduce the Relax-Optimize-and-Sample (ROS) framework. In particular, we begin by relaxing the discrete constraints to the continuous probability simplex form. Next, we pre-train and fine-tune a graph neural network model to efficiently optimize the relaxed problem. Subsequently, we propose a sampling-based construction algorithm to map the continuous solution back to a high-quality Max-k-Cut solution. By integrating geometric landscape analysis with statistical theory, we establish the consistency of function values between the continuous solution and its mapped counterpart. Extensive experimental results on random regular graphs and the Gset benchmark demonstrate that the proposed ROS framework effectively scales to large instances with up to 20,000 nodes in just a few seconds, outperforming state-of-the-art algorithms. Furthermore, ROS exhibits strong generalization capabilities across both in-distribution and out-of-distribution instances, underscoring its effectiveness for large-scale optimization tasks.

Talk 2: Solving Sparse & High-Dimensional-Output Regression via Compression
Speaker: Guanyi Wang
Abstract: Multi-Output Regression (MOR) has been widely used in scientific data analysis for decision-making. Unlike traditional regression models, MOR aims to simultaneously predict multiple real-valued outputs given an input. However, the increasing dimensionality of the outputs poses significant challenges regarding interpretability and computational scalability for modern MOR applications. As a first step to address these challenges, this paper proposes a Sparse & High-dimensional-Output REgression (SHORE) model by incorporating additional sparsity requirements to resolve the output interpretability, and then designs a computationally efficient two-stage framework capable of handling SHORE with provable accuracy via compression on outputs. Theoretically, we show that the proposed framework is computationally scalable while maintaining the same order of training loss and prediction loss before-and-after compression under arbitrary or relatively weak sample set conditions. Empirically, numerical results further validate the theoretical findings, showcasing the efficiency and accuracy of the proposed framework.

Talk 3: On the Exactness of Partial Convexification
Speaker: Rui Chen
Abstract: The idea of partial convexification is very commonly used in discrete optimization and global optimization (e.g., Dantzig-Wolfe decomposition, cutting planes from substructures etc.). We provide sufficient conditions and necessary conditions under which partial convexification gives the exact convex hull. Our results generalize the simplex lemma (folklore), and several known convexification results.

Speakers
GW

Guanyi Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RC

Rui Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3P: Fast Algorithms for Data Science
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Fast Algorithms for Data Science
Chair: Shota Takahashi
Cluster: Optimization For Data Science

Talk 1: Sparsity Constrained and Sparsity Regularized Piecewise Linear Minimization: Optimality Conditions and Algorithms
Speaker: Mustafa Celebi Pinar
Abstract: Motivated by some applications such as sparsity regularized boosting where the so called boosting algorithms maximize the soft margin of produced linear combination of base hypotheses (given an oracle for producing base hypotheses) while dealing with a training set of plus-minus labeled examples, we study optimality conditions for the resulting piecewise-linear and sparsity imposed minimization problems. Different concepts of optimality are defined and developed. Algorithms are proposed and tested. Numerical results will be reported.

Talk 2: Adaptive first-order method for nonconvex optimization derived from vanishing damping continuous-time dynamics
Speaker: Kansei Ushiyama
Abstract: We propose a new first-order algorithm for nonconvex functions with Lipschitz continuous gradients and Hessian matrices. Existing first-order methods use momentum to achieve the lowest known computational complexity for finding a stationary point. The limitation of these methods is that they either require the knowledge of parameters, including Lipschitz constants, or rely on the restart strategy that resets the momentum and can slow down the algorithm. Our method has the lowest known complexity, does not require the knowledge of parameters, and uses a strategy other than restart that does not reset the momentum. This algorithm is derived from a continuous-time algorithm that can be interpreted as a dynamics with vanishing damping. We show numerically that our method works efficiently for some problems.

Talk 3: Accelerated Convergence of Frank–Wolfe Algorithms with Adaptive Bregman Step-Size Strategy
Speaker: Shota Takahashi
Abstract: We propose a Frank–Wolfe (FW) algorithm with an adaptive Bregman step-size strategy for smooth adaptable (weakly-) convex functions. This means that the gradient of the objective function is not necessarily Lipschitz continuous and we only require the smooth adaptable property. Compared to existing FW algorithms, our assumptions are thus less restrictive. We establish convergence guarantees in various settings, such as sublinear to linear convergence rates depending on the assumptions. Assuming that the objective function is weakly convex, we also provide both local sublinear and local linear convergence in terms of the primal gap under the (local) quadratic growth condition. We also propose a variant of the away-step FW algorithm using Bregman distances and establish its global linear convergence for convex optimization and its local linear convergence for nonconvex optimization under the (local) quadratic growth condition over polytopes. Numerical experiments demonstrate that our proposed FW algorithms outperform existing methods. This talk is based on joint works with Akiko Takeda and Sebastian Pokutta.

Speakers
MC

Mustafa Celebi Pinar

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KU

Kansei Ushiyama

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ST

Shota Takahashi

Assistant Professor, The University of Tokyo
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3Q: Fast Algorithmic Frameworks for Semidefinite Programming and Applications
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Fast Algorithmic Frameworks for Semidefinite Programming and Applications
Chair: Ling Liang
Cluster: Conic and Semidefinite Optimization

Talk 1: Convex relaxation for quantum many-body physics
Speaker: Yuehaw Khoo
Abstract: In this talk, we explore adaptations of semidefinite programming relaxations for solving many-body physics problems. Our approach transforms a high-dimensional PDE problem into a convex optimization problem, setting it apart from traditional non-convex methods that rely on nonlinear re-parameterizations of the solution. For quantum mechanical systems, we present a convex program to obtain the ground state in terms of its moments. We further introduce a near-linear time algorithm for solving the convex program using hierarchical matrices.

Talk 2: Fast and Certifiable Trajectory Optimization
Speaker: Shucheng Kang
Abstract: We propose semidefinite trajectory optimization (STROM), a framework that computes fast and certifiably optimal solutions for nonconvex trajectory optimization problems defined by polynomial objectives and constraints. STROM employs sparse second-order Lasserre's hierarchy to generate semidefinite program (SDP) relaxations of trajectory optimization. Different from existing tools (e.g., YALMIP and SOSTOOLS in Matlab), STROM generates chain-like multiple-block SDPs with only positive semidefinite (PSD) variables. Moreover, STROM does so two orders of magnitude faster. Underpinning STROM is cuADMM, the first ADMM-based SDP solver implemented in CUDA and runs in GPUs. cuADMM builds upon the symmetric Gauss-Seidel ADMM algorithm and leverages GPU parallelization to speedup solving sparse linear systems and projecting onto PSD cones. In five trajectory optimization problems (inverted pendulum, cart pole, vehicle landing, flying robot, and car back-in), cuADMM computes optimal trajectories (with certified suboptimality below 1%) in minutes (when other solvers take hours or run out of memory) and seconds (when others take minutes). Further, when warmstarted by data-driven initialization in the inverted pendulum problem, cuADMM delivers real-time performance: providing certifiably optimal trajectories in 0.66 seconds despite the SDP has 49,500 variables and 47,351 constraints.

Talk 3: Exploring chordal sparsity in semidefinite programming with sparse plus low-rank data matrices
Speaker: Tianyun Tang
Abstract: Semidefinite programming (SDP) problems are challenging to solve because of their high dimensionality. However, solving sparse SDP problems with small tree-width are known to be relatively easier because: (1) they can be decomposed into smaller multi-block SDP problems through chordal conversion; (2) they have low-rank optimal solutions. In this paper, we study more general SDP problems whose coefficient matrices have sparse plus low-rank (SPLR) structure. We develop a unified framework to convert such problems into sparse SDP problems with bounded tree-width. Based on this, we derive rank bounds for SDP problems with SPLR structure, which are tight in the worst case.

Speakers
LL

Ling Liang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YK

Yuehaw Khoo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SK

Shucheng Kang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TT

Tianyun Tang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3R: Manifold optimization with special metrics
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Manifold optimization with special metrics
Chair: André Uschmajew
Cluster: Optimization on Manifolds

Talk 1: Optimal transport barycenter via nonconvex concave minimax optimization
Speaker: Xiaohui Chen
Abstract: The optimal transport barycenter is a fundamental notion of averaging that extends from the Euclidean space to the Wasserstein space of probability distributions. Computation of the unregularized barycenter for discretized probability distributions on point clouds is a challenging task when the domain dimension $d > 1$. Most practical algorithms approximating the barycenter problem are based on entropic regularization. In this paper, we introduce a nearly linear time $O(m \log{m})$ primal-dual algorithm for computing the exact barycenter when the input probability density functions are discretized on an $m$-point grid. The key success of our Wasserstein-Descent $\dot{\mathbb{H}}^1$-Ascent (WDHA) algorithm hinges on alternating between two different yet closely related Wasserstein and Sobolev optimization geometries for the primal barycenter and dual Kantorovich potential subproblems. Under reasonable assumptions, we establish the convergence rate and iteration complexity of the proposed algorithm to its stationary point when the step size is appropriately chosen for the gradient updates. Superior computational efficacy and approximation accuracy over the existing Wasserstein gradient descent and Sinkhorn's algorithms are demonstrated on 2D synthetic and real data.

Talk 2: Information geometry of operator scaling
Speaker: Tasuku Soma
Abstract: Matrix scaling is a classical problem with a wide range of applications. It is known that the Sinkhorn algorithm for matrix scaling is interpreted as alternating e-projections from the viewpoint of classical information geometry. Recently, a generalization of matrix scaling to completely positive maps called operator scaling has been found to appear in various fields of mathematics and computer science, and the Sinkhorn algorithm has been extended to operator scaling. In this talk, we discuss operator scaling from the viewpoint of quantum information geometry. For example, the operator Sinkhorn algorithm is shown to coincide with alternating e-projections with respect to the symmetric logarithmic derivative metric, which is a Riemannian metric on the space of quantum states relevant to quantum estimation theory.

Talk 3: Operator Sinkhorn iteration with overrelaxation
Speaker: André Uschmajew
Abstract: We propose accelerated versions of the operator Sinkhorn iteration for operator scaling using successive overrelaxation. We analyze the local convergence rates of these accelerated methods via linearization, which allows us to determine the asymptotically optimal relaxation parameter based on Young's SOR theorem. Based on the Hilbert metric on positive definite cones, we also obtain a global convergence result for a geodesic version of overrelaxation in a specific range of relaxation parameters. Numerical experiments demonstrate that the proposed methods outperform the original operator Sinkhorn iteration in certain applications.

Speakers
XC

Xiaohui Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TS

Tasuku Soma

Associate Professor, The Institute of Statistical Mathematics
AU

André Uschmajew

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3S: Manifold optimization with special metrics
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Manifold optimization with special metrics
Chair: Max Pfeffer
Cluster: Optimization on Manifolds

Talk 1: Riemannian optimization methods for ground state computations of multicomponent Bose-Einstein condensates
Speaker: Tatjana Stykel
Abstract: In this talk, we address the computation of ground states of multicomponent Bose-Einstein condensates by solving the underlying energy minimization problem on the infinite-dimensional generalized oblique manifold. First, we discuss the existence and uniqueness of a ground state with non-negative components and its connection to the coupled Gross-Pitaevskii eigenvector problem. Then, we study the Riemannian structure of the generalized oblique manifold by introducing several Riemannian metrics and computing important geometric tools such as orthogonal projections and Riemannian gradients. This allows us to develop the Riemannian gradient descent methods based on different metrics. Exploiting first- and second-order information of the energy functional for the construction of appropriate metrics makes it possible to incorporate preconditioning into Riemannian optimization, which significantly improves the performance of the optimization schemes. A collection of numerical experiments demonstrates the computational efficiency of the proposed methods. (Joint work with R. Altmann, M. Hermann, and D. Peterseim)

Talk 2: Approximating maps into manifolds with multiple tangent spaces
Speaker: Hang Wang
Abstract: A manifold-valued function takes values from a Euclidean domain into a manifold. Approximating a manifold-valued function from input-output samples consists of modeling the relationship between an output on a Riemannian manifold and the Euclidean input vector. In this talk, I will present algorithms for building a surrogate model to approximate either a known or an unknown manifold-valued function. The proposed methods are based on pullbacks to multiple tangent spaces and the Riemannian center of mass, hereby relying on Riemannian optimization algorithms. The effectiveness of this scheme will be illustrated with numerical experiments for a few model problems.

Talk 3: The injectivity radii of the Stiefel manifold under a one-parameter family of deformation metrics
Speaker: Ralf Zimmermann
Abstract: The injectivity radius of a manifold is an important quantity, both from a theoretical point of view and in terms of numerical applications. It is the largest possible radius within which all geodesics are unique and length-minimizing. In consequence, it is the largest possible radius within which calculations in Riemannian normal coordinates are well-defined. A matrix manifold that arises frequently in a wide range of practical applications is the compact Stiefel manifold of orthogonal p-frames in the Euclidean n-space. Its geometry may be considered under a one-parameter family of deformation metrics. We observe that the associated geodesics are space curves of constant Frenet curvatures. In combination with tight sectional curvature bounds, this allows us to determine the injectivity radius of the Stiefel manifold for a large subset of the one-parameter family of metrics that includes the Euclidean metric.

Speakers
TS

Tatjana Stykel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HW

Hang Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RZ

Ralf Zimmermann

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3T: Variational Analysis: Theory and Applications I
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Variational Analysis: Theory and Applications I
Chair: Walaa Moursi
Cluster: Nonsmooth Optimization

Talk 1: TBA
Speaker: Heinz Bauschke
Abstract: TBA

Talk 2: TBA
Speaker: Radu Bot
Abstract: TBA

Talk 3: TBA
Speaker: Xianfu Wang
Abstract: TBA

Speakers
WM

Walaa Moursi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HB

Heinz Bauschke

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RB

Radu Bot

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XW

Xianfu Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3U: Recent Advances on PDE-constrained optimization packages and libraries: Part II
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Recent Advances on PDE-constrained optimization packages and libraries: Part II
Chair: Umberto Villa
Cluster: Computational Software

Talk 1: hIPPYlib: An Extensible Software Framework for Large-Scale Inverse Problems Governed by PDEs
Speaker: Noemi Petra
Abstract: We present an extensible software framework, hIPPYlib, for solution of large-scale deterministic and Bayesian inverse problems governed by partial differential equations (PDEs) with infinite-dimensional parameter fields (which are high-dimensional after discretization). hIPPYlib overcomes the prohibitive nature of Bayesian inversion for this class of problems by implementing state-of-the-art scalable algorithms for PDE-based inverse problems that exploit the structure of the underlying operators, notably the Hessian of the log-posterior. The key property of the algorithms implemented in hIPPYlib is that the solution of the deterministic and linearized Bayesian inverse problem is computed at a cost, measured in linearized forward PDE solves, that is independent of the parameter dimension. The mean of the posterior is approximated by the MAP point, which is found by minimizing the negative log-posterior. This deterministic nonlinear least-squares optimization problem is solved with an inexact matrix-free Newton-CG method. The posterior covariance is approximated by the inverse of the Hessian of the negative log posterior evaluated at the MAP point. The construction of the posterior covariance is made tractable by invoking a low-rank approximation of the Hessian of the log-likelihood. hIPPYlib makes all of these advanced algorithms easily accessible to domain scientists and provides an environment that expedites the development of new algorithms. Authors: Umberto Villa (UT Austin), Noemi Petra (UC Merced), Omar Ghattas (UT Austin)

Talk 2: SimPEG: an open-source framework for simulation and parameter estimation in geophysics
Speaker: Lindsey Heagy
Abstract: Geophysical data can provide insights about the subsurface in a range of applications. A few examples include locating critical minerals, monitoring geologic storage of CO2, managing groundwater, and characterizing changes to permafrost. The geophysical inverse problem is posed as a PDE-constrained optimization problem where we aim to fit the observed data and incorporate additional information that may include petrophysical, geologic, and geochemical measurements, as well as additional geophysical data sets. We started the SimPEG project with the aim of accelerating research and education in geophysics and enabling researchers to build upon and contribute to a modular toolbox for solving problems in geophysics. At the core is a framework for finite volume forward simulations and gradient-based inversions. SimPEG currently supports simulation and inversion of gravity, magnetic, electrical and electromagnetic data. In this talk, I will provide an overview of how we have broken down inverse problems in geophysics into modular components and how working in an open-source paradigm has facilitated our research, collaborations with industry, and dissemination of educational resources in classrooms and for humanitarian projects.

Talk 3: TBA
Speaker: TBA TBA
Abstract: TBA

Speakers
NP

Noemi Petra

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LH

Lindsey Heagy

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3V: Derivative-free optimization for special classes of problems I
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Derivative-free optimization for special classes of problems I
Chair: Clément Royer
Cluster: Derivative-free Optimization

Talk 1: A derivative-free algorithm for continuous submodular optimization
Speaker: Clément Royer
Abstract: Submodular functions are a classical concept of discrete optimization, that can also be extended to the continuous setting. In particular, the class of continuous submodular functions encompasses some nonconvex functions arising in natural language processing, which partly explains renewed interest for this topic in recent years. In this talk, we propose a derivative-free algorithm for submodular optimization over compact sets, adapted from a classical framework for bound-constrained derivative-free optimization. By leveraging properties of submodular functions, we obtain complexity guarantees for this method, that represent a significant improvement over guarantees in the general, nonconvex setting. We then investigate the practical behavior of our method on our problems of interest.

Talk 2: The cosine measure of a function
Speaker: Gabriel Jarry-Bolduc
Abstract: The cosine measure of a set of vectors is a valuable tool in derivative-free optimization to judge the quality of a set of vectors. It gives information on how uniformly the set of vectors is covering the space R^n. A set of vectors is a positive spanning set of R^n if and only if its cosine measure is greater than zero. An important property of positive spanning sets is that when the gradient of a function at a point is well-defined and not equal to the zero vector, then there is at least one descent direction (ascent direction) of the function at the point contained in the set. This is not necessarily true if the gradient is equal to the zero vector or if the gradient does not exist. To characterize the previous two cases, the novel concept of cosine measure of a function is introduced in this talk. It provides an infimum on the value of the cosine measure of a set of vectors guaranteed to contain a descent direction of the function at the point of interest. It is shown how to theoretically compute the cosine measure of a function for popular classes of nonsmooth functions.

Talk 3: Using resilient positive spanning sets to deal with stragglers
Speaker: Sébastien Kerleau
Abstract: Positive spanning sets (PSSs) are families of vectors that span a given linear space through non-negative linear combinations. Such sets are of particular interest for their use in derivative-free optimization algorithms. In that context, the cost of determining the value of the objective function at a given point can be particularly expensive, taking up to weeks for a single function evaluation. Although time can partly be saved by conducting multiple function evaluations in parallel, the issue of dealing with stragglers - function evaluations that take significantly longer than others - remains to be solved. With that goal in mind, this talk will study a subclass of PSSs whose properties can allow for an improvement of the classical DFO algorithms, making them resilient to the presence of stragglers. The talk will end with numerical experiments studying the efficiency of the new-found resilient algorithm.

Speakers
CR

Clément Royer

Associate professor, Université Paris Dauphine-PSL
Clément W. Royer is an associate professor of computer science at Université Paris Dauphine-PSL. Clément received his Ph.D. from the University of Toulouse, France, and was then a postdoctoral research associate at the Wisconsin Institute of Discovery, University of Wisconsin-Madison... Read More →
GJ

Gabriel Jarry-Bolduc

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SK

Sébastien Kerleau

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3W: Quantum Linear Algebra and Optimization (Part 2)
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Quantum Linear Algebra and Optimization (Part 2)
Chair: Mohammadhossein Mohammadisiahroudi
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Quantum Linear Algebra for Interior Point Methods
Speaker: Mohammadhossein Mohammadisiahroudi
Abstract: Quantum computing has recently emerged as a promising avenue for accelerating large-scale optimization. One notable direction is the use of quantum linear system algorithms to enhance the efficiency of Interior Point Methods. However, a key challenge in Quantum Interior Point Methods (QIPMs) lies in the tomography process, which extracts a classical representation of the Newton system solution from a quantum state. In this talk, we introduce a novel hybrid iterative approach that significantly improves the precision of Newton system solutions, achieving exponential accuracy gains. Additionally, we explore the integration of advanced quantum linear algebra techniques, such as quantum matrix-vector and matrix-matrix multiplications, to further accelerate QIPMs and enhance their practical feasibility.

Talk 2: Quantum Multiplicative Weights SOCP Solver
Speaker: Maria Isabel Franco Garrido
Abstract: Second-order cone programming (SOCP) is a key optimization framework that extends linear and quadratic programming and can be seen as a subset of semidefinite programming (SDP). Its wide applicability in fields such as machine learning, portfolio optimization, robotics, engineering design, and power systems underscores the critical need for faster and scalable solvers. State-of-the-art methods, such as interior-point methods (IPMs), are efficient and widely used in modern solvers; however, as these problems are large-scale, the development of more optimized solvers remains an important area of interest. In this work, we explore multiplicative weights (MW)-based methods as an alternative approach for solving SOCPs, addressing a gap in the classical optimization literature while also investigating quantum speedups. Although MW methods have been successfully applied to linear and semidefinite programs, their potential for SOCPs has remained largely unexplored. We analyze the complexity of our proposed algorithms in terms of query complexity, assuming oracle access to problem data. Our results demonstrate a quadratic quantum speed-up over classical implementations, contingent on the availability of quantum random access memory (QRAM), a common assumption in prior asymptotic analyses.

Talk 3: Exponentially Better Bounds for Quantum Optimization via Dynamical Simulation
Speaker: Sophia Simon
Abstract: In this talk, we present several quantum algorithms for continuous optimization that do not require any gradient estimation. Instead, we encode the optimization problem into the dynamics of a physical system and then leverage existing Hamiltonian simulation algorithms to efficiently simulate the time evolution in a coherent manner. This allows us, in certain cases, to obtain exponentially better query upper bounds relative to the best known upper bounds for optimization schemes based on gradient descent which utilize the quantum computer only for estimating gradients of the objective function.

Speakers
MM

Mohammadhossein Mohammadisiahroudi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3X: Mechanism and Pricing Designs in Stochastic Decision Making
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Mechanism and Pricing Designs in Stochastic Decision Making
Chair: Helene Le Cadre
Cluster: nan

Talk 1: Incentive Design in Nonsmooth Games
Speaker: Helene Le Cadre
Abstract: Considering the growing trend towards integrated human-in-the-loop systems, incorporating irrational behaviors into game-theoretic models that allow to closely reflect human-beings attitudes towards risk is of high relevance. Ultimately, understanding how agents with different risk preferences interact can better inform the mechanism designer and provide guidelines on how to effectively steer agents towards improved collective and individual outcomes. To this end, we study non-cooperative stochastic games, where agents display irrational behaviors in response to underlying risk factors. Our formulation incorporates Prospect Theory (PT), a behavioral model used to describe agents’ risk attitude. We show that the resulting nonconvex nonsmooth game admits equilibria and we quantify the suboptimality induced by irrational behaviors. Then, we extend our PT-based game to an incentive-design problem formulated as a decision-dependent learning game, enabling us to cope with the multiplicity of solutions of the lower-level problem. In this setting, we provide a distributed algorithm with provable convergence, allowing the incentives to adapt dynamically to the information received in a feedback-loop approach. The results are applied to a local energy community involving strategic end users exposed to two-part tariffs.

Talk 2: Distributionally Fair Two-stage Stochastic Programming by Bilevel Optimization
Speaker: Yutian He
Abstract: Two-stage stochastic programming (TSSP) is a fundamental framework for decision-making under uncertainty, where a first-stage decision is made before uncertainty is realized, followed by scenario-dependent second-stage decisions. While most TSSP literature focuses on cost minimization, fairness considerations in decision-making have largely been overlooked. Recently, Ye et al (2025) studied a one-stage stochastic program subject to a distributional fairness constraint, but similar development under the two-stage setting is still unavailable. In this work, we propose two models of TSSP under distributional fairness constraints: one where the first- and second-stage decision-makers collaborate to ensure fairness, and another where only the first-stage decision-maker wants to ensure fairness, while the second-stage decision-maker only aims at minimizing the cost. To solve these models, we approximate the expectations by sample average and then reformulate them as mixed integer nonlinear programs. For large instances, we further develop an alternating minimization method to efficiently solve our problems, providing faster solutions.

Talk 3: Competitive Demand Learning: A Non-cooperative Pricing Algorithm with Coordinated Price Experimentation
Speaker: Yu-Ching Lee
Abstract: We consider a periodical equilibrium pricing problem for multiple firms over a planning horizon of $T$ periods. At each period, firms set their selling prices and receive stochastic demand from consumers. Firms do not know their underlying demand curve, but they wish to determine the selling prices to maximize total revenue under competition. Hence, they have to do some price experiments such that the observed demand data are informative to make price decisions. However, uncoordinated price updating can render the demand information gathered by price experimentation less informative or inaccurate. We design a nonparametric learning algorithm to facilitate coordinated dynamic pricing, in which competitive firms estimate their demand functions based on observations and adjust their pricing strategies in a prescribed manner. We show that the pricing decisions, determined by estimated demand functions, converge to underlying equilibrium as time progresses. {We obtain a bound of the revenue difference that has an order of $\mathcal{O}(F^2T^{3/4})$ and a regret bound that has an order of $\mathcal{O}(F\sqrt{T})$ with respect to the number of the competitive firms~$F$ and $T$.} We also develop a modified algorithm to handle the situation where some firms may have the knowledge of the demand curve.

Speakers
HL

Helene Le Cadre

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Yutian He

Yutian He

University of Iowa
Monday July 21, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 3Y: Machine Learning - Privacy and Learning Theory
Monday July 21, 2025 4:15pm - 5:30pm PDT
Session: Machine Learning - Privacy and Learning Theory
Chair: Devansh Gupta
Cluster: nan

Talk 1: Inherent Privacy of Zeroth Order Projected Gradient Descent
Speaker: Devansh Gupta
Abstract: Differentially private zeroth-order optimization methods have recently gained popularity in private fine tuning of machine learning models due to their reduced memory requirements. Current approaches for privatizing zeroth-order methods rely on adding Gaussian noise to the estimated zeroth-order gradients. However, since the search direction in the zeroth-order methods is inherently random, researchers including Tang et. Al [1] and Zhang et. Al [2] have raised an important question: is the inherent noise in zeroth-order estimators sufficient to ensure the overall differential privacy of the algorithm? This work settles this question for a class of oracle-based optimization algorithms where the oracle returns zeroth-order gradient estimates. In particular, we show that for a fixed initialization, there exist strongly convex objective functions such that running (Projected) Zeroth-Order Gradient Descent (ZO-GD) is not differentially private. Furthermore, we show that even with random initialization and without revealing intermediate iterates, the privacy loss in ZO-GD can grow superlinearly with the number of iterations when minimizing convex objective functions. [1] Tang, X., Panda, A., Nasr, M., Mahloujifar, S., and Mittal, P. (2024). Private fine-tuning of large language models with zeroth-order optimization. [2] Zhang, L., Li, B., Thekumparampil, K. K., Oh, S., and He, N. (2024a). DPZero: Private fine-tuning of language models without backpropagation. In International Conference on Machine Learning. PMLR.

Talk 2: Near-Optimal and Tractable Estimation under Shift-Invariance
Speaker: Dmitrii Ostrovskii
Abstract: In 1990s, Arkadi Nemirovski asked the following question: How hard is it to estimate a sequence of length N satisfying an unknown linear recurrence relation of order S and observed in i.i.d. Gaussian noise? The class of all such sequences is parametric but extremely rich: it contains all exponential polynomials with total degree S, including harmonic oscillations with s arbitrary frequencies. Geometrically, this class corresponds to the projection onto R^N of the union of all shift-invariant subspaces of R^Z of dimension S. In this work, we show that the statistical complexity of this class, as measured by the squared minimax radius of the (1−P)-confidence Euclidean ball, is nearly the same as for the class of S-sparse signals, namely (S\log(N) + \log(1/P)) \log^2(S) \log(N/S) up to a constant factor. Moreover, the corresponding near-minimax estimator is tractable, and it can be used to build a test statistic with a near-minimax detection threshold in the associated detection problem. These statistical results rest upon an approximation-theoretic one: we show that finite-dimensional shift-invariant subspaces admit compactly supported reproducing kernels whose Fourier spectra have the smallest possible p-norms, simultaneously for all p >= 1.

Talk 3: On stochastic mirror descent under relative nonconvexity and nonsmoothness
Speaker: Philip Thompson
Abstract: In this talk, we review recent convergence analysis of the stochastic mirror descent method and present novel convergence analysis within the framework of relative variation (e.g. Lipschitness, smoothness, etc) and relative notions of the PL inequality. We also present some applications in machine learning.

Speakers
DG

Devansh Gupta

PhD Student, University of Southern California
I am a Computer Science PhD student at University of Southern California in the Theory Group and Optimization for Data Driven Science where I am advised by Meisam Razaviyayn and Vatsal Sharan. I am broadly interested in working on fundamental problems in Optimization and Differential... Read More →
DO

Dmitrii Ostrovskii

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PT

Philip Thompson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Monday July 21, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

5:30pm PDT

Break
Monday July 21, 2025 5:30pm - 5:45pm PDT
Monday July 21, 2025 5:30pm - 5:45pm PDT
TBA

5:45pm PDT

Best Paper Session
Monday July 21, 2025 5:45pm - 7:15pm PDT
Finalists (in alphabetical order):

Guy Kornowski for the paper "Oracle Complexity in Nonsmooth Nonconvex Optimization”, co-authored with Ohad Shamir.
Abstract: It is well-known that given a smooth, bounded-from-below, and possibly nonconvex function, standard gradient-based methods can find $\epsilon$-stationary points (with gradient norm less than $\epsilon$) in $\mathcal{O}(1/\epsilon^2)$ iterations. However, many important nonconvex optimization problems, such as those associated with training modern neural networks, are inherently not smooth, making these results inapplicable. In this paper, we study nonsmooth nonconvex optimization from an oracle complexity viewpoint, where the algorithm is assumed to be given access only to local information about the function at various points. We provide two main results: First, we consider the problem of getting \emph{near} $\epsilon$-stationary points. This is perhaps the most natural relaxation of \emph{finding} $\epsilon$-stationary points, which is impossible in the nonsmooth nonconvex case. We prove that this relaxed goal cannot be achieved efficiently, for any distance and $\epsilon$ smaller than some constants. Our second result deals with the possibility of tackling nonsmooth nonconvex optimization by reduction to smooth optimization: Namely, applying smooth optimization methods on a smooth approximation of the objective function. For this approach, we prove under a mild assumption an inherent trade-off between oracle complexity and smoothness: On the one hand, smoothing a nonsmooth nonconvex function can be done very efficiently (e.g., by randomized smoothing), but with dimension-dependent factors in the smoothness parameter, which can strongly affect iteration complexity when plugging into standard smooth optimization methods. On the other hand, these dimension factors can be eliminated with suitable smoothing methods, but only by making the oracle complexity of the smoothing process exponentially large.

Naoki Marumo for the paper “Parameter-free Accelerated Gradient Descent for nonconvex optimization", co-authored with Akiko Takeda.
Abstract: We propose a new first-order method for minimizing nonconvex functions with a Lipschitz continuous gradient and Hessian. The proposed method is an accelerated gradient descent with two restart mechanisms and finds a solution where the gradient norm is less than $\epsilon$ in $O(\epsilon^{-7/4})$ function and gradient evaluations. Unlike existing first-order methods with similar complexity bounds, our algorithm is parameter-free because it requires no prior knowledge of problem-dependent parameters, e.g., the Lipschitz constants and the target accuracy $\epsilon$. The main challenge in achieving this advantage is estimating the Lipschitz constant of the Hessian using only first-order information. To this end, we develop a new Hessian-free analysis based on two technical inequalities: a Jensen-type inequality for gradients and an error bound for the trapezoidal rule. Several numerical results illustrate that the proposed method performs comparably to existing algorithms with similar complexity bounds, even without parameter tuning.

Lai Tian for the paper "Testing Approximate Stationarity Concepts for Piecewise Affine Functions", co-authored with Anthony Man-Cho So.
Abstract: We study the basic computational problem of detecting approximate stationary points for continuous piecewise affine (PA) functions. Our contributions span multiple aspects, including complexity, regularity, and algorithms. Specifically, we show that testing first-order approximate stationarity concepts, as defined by commonly used generalized subdifferentials, is computationally intractable unless $\cP=\cNP$. To facilitate computability, we consider a polynomial-time solvable relaxation by abusing the convex subdifferential sum rule and establish a tight characterization of its exactness. Furthermore, addressing an open issue motivated by the need to terminate the subgradient method in finite time, we introduce the first oracle-polynomial-time algorithm to detect so-called near-approximate stationary points for PA functions. A notable byproduct of our development in regularity is the first necessary and sufficient condition for the validity of an equality-type (Clarke) subdifferential sum rule. Our techniques revolve around two new geometric notions for convex polytopes and may be of independent interest in nonsmooth analysis. Moreover, some corollaries of our work on complexity and algorithms for stationarity testing address open questions in the literature. To demonstrate the versatility of our results, we complement our findings with applications to a series of structured piecewise smooth functions, including $\rho$-margin-loss SVM, piecewise affine regression, and nonsmooth neural networks.

Speakers
NM

Naoki Marumo

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Guy Kornowski

Guy Kornowski

Weizmann Institute of Science
Talk title:Dimension dependence in nonconvex optimizationBio:Guy Kornowski is a PhD student at the Weizmann Institute of Science, advised by Prof. Ohad Shamir. During his PhD he interned at Apple ML Research, where he worked with Kunal Talwar and Vitaly Feldman. His research focuses... Read More →
Monday July 21, 2025 5:45pm - 7:15pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089
 
Tuesday, July 22
 

8:30am PDT

Auditorium Opens Doors for seating
Tuesday July 22, 2025 8:30am - 9:00am PDT
Tuesday July 22, 2025 8:30am - 9:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

9:00am PDT

Plenary 2
Tuesday July 22, 2025 9:00am - 10:00am PDT
Speakers
KT

Kim-Chuan Toh

Kim-Chuan Toh is the Leo Tan Professor in the Department of Mathematics at the National University of Singapore. He works extensively on convex programming, particularly large-scale matrix optimization problems such as semidefinite programming, and optimization problems arising... Read More →
Tuesday July 22, 2025 9:00am - 10:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

10:00am PDT

Coffee & Snack Break (Provided)
Tuesday July 22, 2025 10:00am - 10:30am PDT
Tuesday July 22, 2025 10:00am - 10:30am PDT
TBA

10:30am PDT

Parallel Sessions 4A: Progress in Nonsmooth Optimization
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Progress in Nonsmooth Optimization
Chair: Feng Ruan
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Subgradient Convergence Implies Subdifferential Convergence on Weakly Convex Functions: With Uniform Rates Guarantees
Speaker: Feng Ruan
Abstract: In nonsmooth, nonconvex stochastic optimization, understanding the uniform convergence of subdifferential mappings is crucial for analyzing stationary points of sample average approximations of risk as they approach the population risk. Yet, characterizing this convergence remains a fundamental challenge. This work introduces a novel perspective by connecting the uniform convergence of subdifferential mappings to that of subgradient mappings as empirical risk converges to the population risk. We prove that, for stochastic weakly-convex objectives, and within any open set, a uniform bound on the convergence of subgradients -- chosen arbitrarily from the corresponding subdifferential sets -- translates to a uniform bound on the convergence of the subdifferential sets itself, measured by the Hausdorff metric. Using this technique, we derive uniform convergence rates for subdifferential sets of stochastic convex-composite objectives. Our results do not rely on key distributional assumptions in the literature, which require the population and finite sample subdifferentials to be continuous in the Hausdorff metric, yet still provide tight convergence rates. These guarantees lead to new insights into the nonsmooth landscapes of such objectives within finite samples.

Talk 2: Variational Theory and Algorithms for a Class of Asymptotically Approachable Nonconvex Problems
Speaker: Ying Cui
Abstract: We investigate a class of composite nonconvex functions, where the outer function is the sum of univariate extended-real-valued convex functions and the inner function is the limit of difference-of-convex functions. A notable feature of this class is that the inner function can be merely lower semicontinuous instead of continuously differentiable. It covers a range of important yet challenging applications, including the composite value functions of nonlinear programs and the value-at-risk constraints. We propose an asymptotic decomposition of the composite function that guarantees epi-convergence to the original function, leading to necessary optimality conditions for the corresponding minimization problem. The proposed decomposition also enables us to design a numerical algorithm such that any accumulation point of the generated sequence, if exists, satisfies the newly introduced optimality conditions. These results expand on the study of so-called amenable functions introduced by Poliquin and Rockafellar in 1992, which are compositions of convex functions with smooth maps, and the prox-linear methods for their minimization.

Talk 3: Survey Descent: a Case-Study in Amplifying Optimization Research with Modern ML Workflows
Speaker: X.Y. Han
Abstract: Within the classic optimization, one learns that for strongly convex objectives that are smooth, gradient descent ensures linear convergence of iterates and objective values relative to the number of gradient evaluations. Nonsmooth objective functions are more challenging: existing solutions typically invoke cutting plane methods whose complexities are difficult to bound, leading to convergence guarantees that are sublinear in the cumulative number of gradient evaluations. We instead propose a multipoint generalization of the gradient descent called Survey Descent. In this method, one first leverages a one-time initialization procedure to gather a "survey" of points. Then, during each iteration of the method, the survey points are updated in parallel using a simple, four-line procedure inspired by gradient descent. Under certain regularity conditions, we prove that Survey Descent then achieves a desirable performance by converging linearly to the optimal solution in the nonsmooth setting. Despite being an nominally mathematical endeavor, we discuss how the development of Survey Descent was significantly accelerated by a frictionless computational workflow made possible by tools from modern machine learning (ML); how this model of applying new ML workflows to solve open questions in optimization and applied probability could amplify the researchers' productivity; practical computational bottlenecks that could hinder this integration; and what tools are needed to overcome those obstacles.

Speakers
FR

Feng Ruan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YC

Ying Cui

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for X.Y. Han

X.Y. Han

Assistant Professor, Chicago Booth
Name: Prof. X.Y. HanTitle: Assistant Professor of Operations Management and Applied AIAffiliation: Chicago BoothBio:X.Y. Han is an assistant professor of Operations Management and Applied Artificial Intelligence at the University of Chicago, Booth School of Business. His research... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4B: Optimization for Neural Network Pruning and Quantization
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Optimization for Neural Network Pruning and Quantization
Chair: Lin Xiao
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Understanding Neural Network Quantization and its Robustness Against Data Poisoning Attacks
Speaker: Yiwei Lu
Abstract: Neural network quantization, exemplified by BinaryConnect (BC) and its variants, has become a standard approach for model compression. These methods typically employ the sign function in the forward pass, with various approximations used for gradient computation during backpropagation. While effective, these techniques often rely on heuristics or "training tricks" that lack theoretical grounding. This talk explores the optimization perspective of these quantization methods, introducing forward-backward quantizers as a principled framework. We present ProxConnect++ (PC++), a generalization that encompasses existing quantization techniques and provides automatic theoretical guarantees. Furthermore, we reveal an unexpected benefit of neural network quantization: enhanced robustness against data poisoning attacks.

Talk 2: Convex Regularizations for Pruning- and Quantization-Aware Training of Neural Networks
Speaker: Lin Xiao
Abstract: We present a convex regularization approach for pruning- and quantization-aware training of deep neural networks. While it is well-known that group Lasso can induce structured pruning, we show that convex, piece-wise affine regularizations (PAR) can effectively induce quantization. Previous work have limited success in practice due to the challenge of integrating structured regularization with stochastic gradient methods. We derive an aggregate proximal stochastic gradient method (AProx) that can successfully produce desired pruning and quantization results. Moreover, we establish last-iterate convergence of the method, which better supports the computational practice than the classical theory of average-iterate convergence.

Talk 3: Quantization through Piecewise-Affine Regularization: Optimization and Statistical Guarantees
Speaker: Jianhao Ma
Abstract: Optimization problems involving discrete or quantized variables can be very challenging due to the combinatorial nature of the design space. We show that (coordinate-wise) piecewise-affine regularization (PAR) can effectively induce quantization in the optimization variables. PAR provides a flexible modeling and computational framework for quantization based on continuous and convex optimization. In addition, for linear regression problems, we can approximate $\ell_1$- and squared $\ell_2$-regularizations using different parameterizations of PAR, and obtain statistical guarantees that are similar to those of Lasso and ridge regression, all with quantized regression variables.

Speakers
YL

Yiwei Lu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LX

Lin Xiao

Lin Xiao is a Research Scientist at Facebook AI Research (FAIR) in Seattle, Washington. He received BE from Beijing University of Aeronautics and Astronautics (Beihang University) and PhD from Stanford University, and was a postdoctoral fellow in the Center for the Mathematics of... Read More →
JM

Jianhao Ma

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4C: Methods for Large-Scale Nonlinear Optimization IV
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Methods for Large-Scale Nonlinear Optimization IV
Chair: Albert Berahas
Cluster: Nonlinear Optimization

Talk 1: High Probability Analysis for Negative Curvature Methods with Probabilistic Oracles
Speaker: Wanping Dong
Abstract: We consider a negative curvature method for continuous nonlinear nonconstrained optimization problems under a stochastic setting where the function values, gradients, and Hessian (products) are available only through inexact probabilistic oracles. Our goal is to develop algorithms that have high probabilistic second-order convergence and affordable complexity so that they can be used for large-scale problems. We introduce general conditions on the probabilistic oracles and propose a method that dynamically chooses between negative curvature and descent steps. We derive a high probability tail bound on the iteration complexity of the algorithm and show improvements compared to our previous negative curvature method. A practical variant is implemented to illustrate the power of the proposed algorithm.

Talk 2: Retrospective Approximation for Stochastic Constrained Problems Using Sequential Quadratic Programming
Speaker: Shagun Gupta
Abstract: Sequential Quadratic Programming (SQP) is one of the state-of-the-art algorithms used to solve deterministic constrained nonlinear optimization problems. In recent years, the framework has been extended to solve deterministic equality and inequality constrained problems with stochastic objective functions. In response to the challenges posed by stochasticity, various schemes have been incorporated into SQP algorithms to adapt key parameters, such as step size and merit parameter, from the deterministic setting to the stochastic setting. These include stochastic line search, Lipschitz constant estimation, and Hessian averaging. In our work, we leverage SQP algorithms within the innovative framework of Retrospective Approximation. This framework introduces a novel approach to solving stochastic constrained problems by allowing the SQP algorithm to solve a series of subsampled deterministic subproblems. Each deterministic subproblem is solved not to optimality, but to a specified accuracy with an increasing sample size for the subsampled deterministic problems. This strategic decoupling of stochasticity from the SQP algorithm proves instrumental, enabling the utilization of legacy deterministic solvers. Thus, by decoupling stochasticity, the Retrospective Approximation framework facilitates the integration of legacy deterministic solvers, reducing requirements for hyper-parameter tuning in stochastic settings. We provide theoretical convergence requirements for the increase in the subsampling batch size and required solution accuracy for deterministic subproblems. We also conduct numerical experiments to showcase the utilization of legacy deterministic solvers for stochastic constrained problems.

Talk 3: Stochastic Second-order Inexact Augmented Lagrangian Framework for Nonconvex Expectation Constrained Optimization
Speaker: Yash Kumar
Abstract: In this talk, we present methods for solving stochastic nonconvex optimization problems where both the objective function and the constraints are expectations of stochastic functions. We consider an inexact Augmented Lagrangian framework for solving these problems, employing stochastic second-order methods for the subproblems instead of first-order methods. This framework ensures convergence to second-order stationary points instead of approximate first-order stationary points. Furthermore, these methods do not require access to full Hessians but only Hessian-vector products, which are typically twice the computational cost of gradients. We provide convergence guarantees for the stochastic second-order inexact Augmented Lagrangian framework, along with total computational complexity guarantees for various second-order subproblem solvers. Numerical experiments on constrained machine learning classification problems demonstrate the efficiency of the proposed framework.

Speakers
AB

Albert Berahas

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WD

Wanping Dong

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SG

Shagun Gupta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YK

Yash Kumar

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4D: Recent Advances in Large-scale Optimization II
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Recent Advances in Large-scale Optimization II
Chair: Salar Fattahi
Cluster: Nonlinear Optimization

Talk 1: Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Space
Speaker: Yao Xie
Abstract: We consider a minimax problem motivated by distributionally robust optimization (DRO) when the worst-case distribution is continuous, leading to significant computational challenges due to the infinite-dimensional nature of the optimization problem. Recent research has explored learning the worst-case distribution using neural network-based generative models to address these computational challenges but lacks algorithmic convergence guarantees. This paper bridges this theoretical gap by presenting an iterative algorithm to solve such a minimax problem, achieving global convergence under mild assumptions and leveraging technical tools from vector space minimax optimization and convex analysis in the space of continuous probability densities. In particular, leveraging Brenier's theorem, we represent the worst-case distribution as a transport map applied to a continuous reference measure and reformulate the regularized discrepancy-based DRO as a minimax problem in the Wasserstein space. Furthermore, we demonstrate that the worst-case distribution can be efficiently computed using a modified Jordan-Kinderlehrer-Otto (JKO) scheme with sufficiently large regularization parameters for commonly used discrepancy functions linked to the radius of the ambiguity set. Additionally, we derive the global convergence rate and quantify the total number of subgradient and inexact modified JKO iterations required to obtain approximate stationary points. These results are potentially apply to nonconvex and nonsmooth scenarios, with broad relevance to modern machine learning applications.

Talk 2: Federated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning
Speaker: Yuejie Chi
Abstract: Federated reinforcement learning (RL) enables collaborative decision making of multiple distributed agents without sharing local data trajectories. In this work, we consider a multi-task setting, in which each agent has its own private reward function corresponding to different tasks, while sharing the same transition kernel of the environment. Focusing on infinite-horizon Markov decision processes, the goal is to learn a globally optimal policy that maximizes the sum of the discounted total rewards of all the agents in a decentralized manner, where each agent only communicates with its neighbors over some prescribed graph topology. We develop federated vanilla and entropy-regularized natural policy gradient (NPG) methods in the tabular setting under softmax parameterization, where gradient tracking is applied to estimate the global Q-function to mitigate the impact of imperfect information sharing. We establish non-asymptotic global convergence guarantees under exact policy evaluation, where the rates are nearly independent of the size of the state-action space and illuminate the impacts of network size and connectivity. To the best of our knowledge, this is the first time that near dimension-free global convergence is established for federated multi-task RL using policy optimization. We further go beyond the tabular setting by proposing a federated natural actor critic (NAC) method for multi-task RL with function approximation, and establish its finite-time sample complexity taking the errors of function approximation into account.

Talk 3: Invariant Low-Dimensional Subspaces in Gradient Descent for Learning Deep Networks
Speaker: Qing Qu
Abstract: Over the past few years, an extensively studied phenomenon in training deep networks is the implicit bias of gradient descent towards parsimonious solutions. In this work, we first investigate this phenomenon by narrowing our focus to deep linear networks. Through our analysis, we reveal a surprising "law of parsimony" in the learning dynamics when the data possesses low-dimensional structures. Specifically, we show that the evolution of gradient descent starting from orthogonal initialization only affects a minimal portion of singular vector spaces across all weight matrices. In other words, the learning process happens only within a small invariant subspace of each weight matrix, even though all weight parameters are updated throughout training. This simplicity in learning dynamics could have significant implications for both efficient training and a better understanding of deep networks. First, the analysis enables us to considerably improve training efficiency by taking advantage of the low-dimensional structure in learning dynamics. We can construct smaller, equivalent deep linear networks without sacrificing the benefits associated with the wider counterparts. Moreover, we demonstrate the potential implications for efficient training deep nonlinear networks. Second, it allows us to better understand deep representation learning by elucidating the progressive feature compression and discrimination from shallow to deep layers. The study paves the foundation for understanding hierarchical representations in deep nonlinear networks.

Speakers
SF

Salar Fattahi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YX

Yao Xie

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YC

Yuejie Chi

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
QQ

Qing Qu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4E: Random subspace algorithms in optimization
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Random subspace algorithms in optimization
Chair: Pierre-Louis Poirion
Cluster: Optimization For Data Science

Talk 1: Zeroth-order Random Subspace AAlgorithm for Non-smooth Convex Optimization
Speaker: Akiko Takeda
Abstract: Zeroth-order optimization, which does not use derivative information, is one of the significant research areas in the field of mathematical optimization and machine learning. Although various studies have explored zeroth-order algorithms, one of the theoretical limitations is that oracle complexity depends on the dimension, i.e., on the number of variables, of the optimization problem. In this talk, we propose a zeroth-order random subspace algorithm by combining a gradient-free algorithm (specifically, Gaussian randomized smoothing with central differences) with random projection to reduce the dependency of the dimension in oracle complexity. The algorithm has a local convergence rate independent of the original dimension under some local assumptions.

Talk 2: Random matrix to solve large linear system
Speaker: Jiaming Yang
Abstract: Large matrices arise in real-world applications in the areas of machine learning, data analysis and optimization: from the representation of massive datasets with high dimensional features, to the first and second-order derivatives of an objective function that needs to be optimized. How can we capture the intrinsic patterns and physics efficiently and accurately? As an interdisciplinary research area that roots in theoretical computer science (TCS), random matrix theory (RMT), and recently thrives in scientific computing and machine learning, Randomized Numerical Linear Algebra (RNLA) offers a promising avenue to alleviate computational challenges by introducing randomness to large-scale problems, with the guarantee of creating efficient approximate solutions that retain high accuracy. Specifically, randomized subspace methods are one of the most popular methods that are well established in theory but only start flourishing in the area of optimization recently. In my recent work [Derezinski,Yang, STOC 2024] and [Derezinski, Musco, Yang, SODA 2025], we focus on the problem of solving a linear system and develope different types of stochastic optimization algorithms. Our algorithms provably achieve better time complexity results, and are linear in the input matrix size when we assume that it has a flat tail. As one of our key techniques, we construct a low- rank Nystrom approximation with sparse random sketching, resulting in an easy-to-construct preconditioner with the effective guarantee from the randomized subspace theory.

Talk 3: Random Subspace Homogenized Trust Region Method
Speaker: Pierre-Louis Poirion
Abstract: We proposes the Random Subspace Homogenized Trust Region (RSHTR) method with the best theoretical guarantees among random subspace algorithms for nonconvex optimization. Furthermore, under rank-deficient conditions, RSHTR converge to a second-order stationary point quadratically. Experiments on real-world datasets verify the benefits of RSHTR.

Speakers
AT

Akiko Takeda

University of Tokyo/RIKEN
Akiko Takeda received the Doctor of Science degree in information science from the Tokyo Institute of Technology, Japan, in 2001. She is currently a professor in the Department of Mathematical Informatics, The University of Tokyo, and the team leader of Continuous Optimization Team... Read More →
avatar for Jiaming Yang

Jiaming Yang

PhD Student, University of Michigan
PP

Pierre-Louis Poirion

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4F: Contextual Stochastic Optimization under Streaming Data and Decision Dependency
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Contextual Stochastic Optimization under Streaming Data and Decision Dependency
Chair: Guzin Bayraksan
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Residuals-Based Contextual Distributionally Robust Optimization with Decision-Dependent Uncertainty
Speaker: Xian Yu
Abstract: We consider a residuals-based distributionally robust optimization model, where the underlying uncertainty depends on both covariate information and our decisions. We adopt regression models to learn the latent decision dependency and construct a nominal distribution (thereby ambiguity sets) around the learned model using empirical residuals from the regressions. Ambiguity sets can be formed via the Wasserstein distance, a sample robust approach, or with the same support as the nominal empirical distribution (e.g., phi-divergences), where both the nominal distribution and the radii of the ambiguity sets could be decision- and covariate-dependent. We provide conditions under which desired statistical properties, such as asymptotic optimality, rates of convergence, and finite sample guarantees, are satisfied. Via cross-validation, we devise data-driven approaches to find the best radii for different ambiguity sets, which can be decision-(in)dependent and covariate-(in)dependent. Through numerical experiments, we illustrate the effectiveness of our approach and the benefits of integrating decision dependency into a residuals-based DRO framework.

Talk 2: Distribution-Free Algorithms for Predictive Stochastic Programming in the Presence of Streaming Data
Speaker: Suvrajeet Sen
Abstract: This work studies a fusion of concepts from stochastic programming and nonparametric statistical learning in which data is available in the form of covariates interpreted as predictors and responses. Such models are designed to impart greater agility, allowing decisions under uncertainty to adapt to the knowledge of predictors (leading indicators). This work studies two classes of methods: one of the methods may be classified as a first-order method, whereas the other studies piecewise linear approximations. In addition, our study incorporates several non-parametric estimation schemes, including k nearest neighbors (kNN) and other standard kernel estimators. Our computational results demonstrate that the new algorithms outperform traditional approaches which were not designed for streaming data applications requiring simultaneous estimation and optimization. (This work was performed as part of the first author's doctoral dissertation.)

Talk 3: An Alternating Optimization Method for Contextual Distributionally Robust Optimization under Streaming Data
Speaker: Guzin Bayraksan
Abstract: We consider data-driven decision-making that incorporates a prediction model within the 1-Wasserstein distributionally robust optimization (DRO) given joint observations of uncertain parameters and covariates using regression residuals in a streaming-data setting. In this setting, additional data become available and allow decisions to adapt to the growing knowledge of the underlying uncertainty. The ambiguity set shrinks as more data is observed. We propose an efficient online optimization method for this streaming-data contextual DRO setting, which iteratively alternates between optimizing the decision and determining the worst-case distribution. We analyze the asymptotic convergence properties of this algorithm and establish dynamic regret bounds to certify the performance of online solutions. Through numerical experiments, we validate our theoretical findings and demonstrate that our approach significantly enhances computational efficiency while maintaining high solution quality under streaming data.

Speakers
XY

Xian Yu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SS

Suvrajeet Sen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GB

Guzin Bayraksan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4G: Analysis and Design of First Order Methods for Convex Optimization
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Analysis and Design of First Order Methods for Convex Optimization
Chair: Bartolomeo Stellato
Cluster: Optimization For Data Science

Talk 1: Toward a grand unified theory of accelerations in optimization and machine learning
Speaker: Ernest Ryu
Abstract: Momentum-based acceleration of first-order optimization methods, first introduced by Nesterov, has been foundational to the theory and practice of large-scale optimization and machine learning. However, finding a fundamental understanding of such acceleration remains a long-standing open problem. In the past few years, several new acceleration mechanisms, distinct from Nesterov’s, have been discovered, and the similarities and dissimilarities among these new acceleration phenomena hint at a promising avenue of attack for the open problem. In this talk, we discuss the envisioned goal of developing a mathematical theory unifying the collection of acceleration mechanisms and the challenges that are to be overcome.

Talk 2: Data-driven performance estimation of first-order methods
Speaker: Jisun Park
Abstract: We introduce a data-driven approach to analyze the probabilistic performance of first-order optimization algorithms. Combining Wasserstein Distributionally Robust Optimization to the performance estimation framework, we derive probabilistic performance guarantees to a wide range first-order methods. We show that our method is able to achieve significant reductions in conservatism compared to classical worst-case performance analysis tools.

Talk 3: Exact Verification of First-Order Methods for Quadratic Optimization via Mixed-Integer Programming
Speaker: Vinit Ranjan
Abstract: We present a mixed-integer programming based framework to exactly verify the convergence of first-order methods for parametric convex quadratic and linear optimization. We formulate the verification problem as a mathematical optimization problem where we maximize the infinity norm of the fixed-point residual at the last iteration subject to constraints on the parameters and initial iterates. Our approach uses affine and piecewise affine steps to exactly represent a wide range of gradient, projection, and proximal steps. We scale the mixed-integer formulation using advanced bound tightening and strong formulations for the piecewise affine steps. Numerical examples show orders of magnitude lower worst-case residuals that more closely match the practical convergence.

Speakers
BS

Bartolomeo Stellato

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ER

Ernest Ryu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JP

Jisun Park

Postdoctoral Research Fellow, Princeton University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4H: Advances in Network Optimization and Cooperative Learning
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Advances in Network Optimization and Cooperative Learning
Chair: Cesar A Uribe
Cluster: Multi-agent Optimization and Games

Talk 1: Optimally Improving Cooperative Learning in a Social Settin
Speaker: Shahrzad Haddadan
Abstract: We consider a cooperative learning scenario where a collection of networked agents with individually owned classifiers update their predictions, for the same classification task, through communication or observations of each other’s predictions. Clearly if highly influential vertices use erroneous classifiers, there will be a negative effect on the accuracy of all the agents in the network. We ask the following question: how can we optimally fix the prediction of a few classifiers so as maximize the overall accuracy in the entire network. To this end we consider an aggregate and an egalitarian objective function. We show a polynomial time algorithm for optimizing the aggregate objective function, and show that optimizing the egalitarian objective function is NP-hard. Furthermore, we develop approximation algorithms for the egalitarian improvement. The performance of all of our algorithms are guaranteed by mathematical analysis and backed by experiments on synthetic and real data.

Talk 2: The engineering potential of fish research: swimming upstream to new solutions
Speaker: Daniel Burbano
Abstract: Millions of years of evolution have endowed animals with refined and elegant mechanisms to orient and navigate in complex environments. Elucidating the underpinnings of these processes is of critical importance not only in biology to understand migration and survival but also for engineered network systems to aid the development of bio-inspired algorithms for estimation and control. Particularly interesting is the study of fish navigation where different cues, such as vision and hydrodynamics are integrated and fed back to generate locomotion. Little is known, however, about the information pathways and the integration process underlying complex navigation problems. This talk will discuss recent advances in data-driven mathematical models based on potential flow theory, stochastic differential equations, and control theory describing fish navigation. In addition, we will discuss how biological insights gained from this research can be applied to robot navigation with zero-order optimization and estimation and control problems in network systems

Talk 3: An Optimal Transport Approach for Network Regression
Speaker: Alex Zalles
Abstract: We study the problem of network regression, where one is interested in how the topology of a network changes as a function of Euclidean covariates. We build upon recent developments in generalized regression models on metric spaces based on Fr\'echet means and propose a network regression method using the Wasserstein metric. We show that when representing graphs as multivariate Gaussian distributions, the network regression problem requires the computation of a Riemannian center of mass (i.e., Fr\'echet means). Fr\'echet means with non-negative weights translates into a barycenter problem and can be efficiently computed using fixed point iterations. Although the convergence guarantees of fixed-point iterations for the computation of Wasserstein affine averages remain an open problem, we provide evidence of convergence in a large number of synthetic and real-data scenarios. Extensive numerical results show that the proposed approach improves existing procedures by accurately accounting for graph size, topology, and sparsity in synthetic experiments. Additionally, real-world experiments using the proposed approach result in higher Coefficient of Determination () values and lower mean squared prediction error (MSPE), cementing improved prediction capabilities in practice.

Speakers
SH

Shahrzad Haddadan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DB

Daniel Burbano

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AZ

Alex Zalles

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4I: Optimization for Large Language Models and Kernels
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Optimization for Large Language Models and Kernels
Chair: Ming Yin
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Optimizing for a Proxy Reward in RLHF
Speaker: Banghua Zhu
Abstract: Reinforcement Learning from Human Feedback (RLHF) has become an important technique in post-training of Larger Language Models (LLM). During RLHF, one usually first trains a reward model from human preference data, and then optimizes the LLM for the proxy reward signal predicted by the reward model. In this talk, I'll discuss what makes a good reward model for RLHF from both theoretical and empirical observations.

Talk 2: Self-Play Preference Optimization for Language Model Alignment
Speaker: Yue Wu
Abstract: In this paper, we propose a self-play-based method for language model alignment, which treats the problem as a constant-sum two-player game aimed at optimizing the model to approximate the Nash equilibrium. Our approach, dubbed SPPO, is based on a new alignment objective derived from L2 regression. Interestingly, this new objective has a deep connection with the KL-regularized policy gradient and natural gradient methods, and can guarantee the convergence to the optimal solution. In our experiments, this theoretically motivated objective turns out highly effective. By leveraging a small pre-trained preference model, SPPO can obtain a highly-aligned model without additional external supervision from human or stronger language models.

Talk 3: Learning Counterfactual Distributions via Kernel Nearest Neighbors
Speaker: Kyuseong Choi
Abstract: Consider a setting with multiple units (e.g., individuals, cohorts, geographic locations) and outcomes (e.g., treatments, times, items), where the goal is to learn a multivariate distribution for each unit-outcome entry, such as the distribution of a user's weekly spend and engagement under a specific mobile app version. A common challenge is the prevalence of missing not at random data---observations are available only for certain unit-outcome combinations---where the observed distributions can be correlated with properties of distributions themselves, i.e., there is unobserved confounding. An additional challenge is that for any observed unit-outcome entry, we only have a finite number of samples from the underlying distribution. We tackle these two challenges by casting the problem into a novel distributional matrix completion framework and introduce a kernel-based distributional generalization of nearest neighbors to estimate the underlying distributions. By leveraging maximum mean discrepancies and a suitable factor model on the kernel mean embeddings of the underlying distributions, we establish consistent recovery of the underlying distributions even when data is missing not at random and positivity constraints are violated. Furthermore, we demonstrate that our nearest neighbors approach is robust to heteroscedastic noise, provided we have access to two or more measurements for the observed unit-outcome entries—a robustness not present in prior works on nearest neighbors with single measurements.

Speakers
BZ

Banghua Zhu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YW

Yue Wu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KC

Kyuseong Choi

PhD, Cornell Tech
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4J: Dynamic Optimization: Deterministic and Stochastic Continuous-time Models I
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Dynamic Optimization: Deterministic and Stochastic Continuous-time Models I
Chair: Cristopher Hermosilla
Cluster: Multi-agent Optimization and Games

Talk 1: A Minimality Property of the Value Function in Optimal Control over the Wasserstein Space
Speaker: Cristopher Hermosilla
Abstract: In this talk we study an optimal control problem with (possibly) unbounded terminal cost in the space of Borel probability measures with finite second moment. We consider the metric geometry associated with the Wasserstein distance, and a suitable weak topology rendering this space locally compact. In this setting, we show that the value function of a control problem is the minimal viscosity supersolution of an appropriate Hamilton-Jacobi-Bellman equation. Additionally, if the terminal cost is bounded and continuous, we show that the value function is the unique viscosity solution of the Hamilton-Jacobi-Bellman equation.

Talk 2: Principal-Multiagents problem in continuous-time
Speaker: Nicolás Hernández
Abstract: We study a general contracting problem between the principal and a finite set of competitive agents, who perform equivalent changes of measure by controlling the drift of the output process and the compensator of its associated jump measure. In this setting, we generalize the dynamic programming approach developed by Cvitanić, Possamaï, and Touzi (2017) and we also relax their assumptions. We prove that the problem of the principal can be reformulated as a standard stochastic control problem in which she controls the continuation utility (or certainty equivalent) processes of the agents. Our assumptions and conditions on the admissible contracts are minimal to make our approach work. We also present a smoothness result for the value function of a risk–neutral principal when the agents have exponential utility functions. This leads, under some additional assumptions, to the existence of an optimal contract.

Talk 3: Unbounded viscosity solutions of Hamilton-Jacobi equations in the 2-Wasserstein space
Speaker: Othmane Jerhaoui
Abstract: In this talk, we study unbounded viscosity solutions of Hamilton-Jacobi equations in the 2-Wasserstein space over the Euclidean space. The notion of viscosity is defined by taking test functions that are locally Lipschitz and can be respresented as a difference of two geodesically semiconvex functions. First, We establish a comparison result for a general Hamiltonian sat- isfying mild hypotheses. Then, we will discuss well-posedness of a class of Hamilton-Jacobi equations with a Hamiltonian arising from classical mechanics.

Speakers
CH

Cristopher Hermosilla

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
NH

Nicolás Hernández

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
OJ

Othmane Jerhaoui

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4K: Convex approaches to SDP
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Convex approaches to SDP
Chair: Richard Zhang
Cluster: Conic and Semidefinite Optimization

Talk 1: Fitting Tractable Convex Sets to Support Function Data
Speaker: Venkat Chandrasekaran
Abstract: The geometric problem of estimating an unknown compact convex set from evaluations of its support function arises in a range of scientific and engineering applications. Traditional approaches typically rely on estimators that minimize the error over all possible compact convex sets; in particular, these methods do not allow for much incorporation of prior structural information about the underlying set and the resulting estimates become increasingly more complicated to describe as the number of measurements available grows.  We address both of these shortcomings by describing a new framework for estimating tractably specified convex sets from support function evaluations.  Along the way, we also present new bounds on how well arbitrary convex bodies can be approximated by elements from structured non-polyhedral families of convex sets.  Our numerical experiments highlight the utility of our framework over previous approaches in settings in which the measurements available are noisy or small in number as well as those in which the underlying set to be reconstructed is non-polyhedral. (Joint work with Yong Sheng Soh and Eliza O'Reilly.)

Talk 2: Iterative methods for primal-dual scalings in conic optimization
Speaker: Lieven Vandenberghe
Abstract: A central element in the design of primal-dual interior-point methods for conic optimization is the definition of a suitable primal-dual scaling or metric. The talk will discuss simple iterative methods for computing primal-dual scalings. We will consider the Nesterov-Todd scaling for symmetric cones and extensions to sparse positive semidefinite matrix cones.

Talk 3: Generalized Cuts and Grothendieck Covers: a Primal-Dual Approximation Framework Extending the Goemans--Williamson Algorithm
Speaker: Nathan Benedetto Proenca
Abstract: We provide a primal-dual framework for randomized approximation algorithms utilizing semidefinite programming (SDP) relaxations. Our framework pairs a continuum of APX-complete problems including MaxCut, Max2Sat, MaxDicut, and more generally, Max-Boolean Constraint Satisfaction and MaxQ (maximization of a positive semidefinite quadratic form over the hypercube) with new APX-complete problems which are stated as convex optimization problems with exponentially many variables. These new dual counterparts, based on what we call Grothendieck covers, range from fractional cut covering problems (for MaxCut) to tensor sign covering problems (for MaxQ). For each of these problem pairs, our framework transforms the randomized approximation algorithms with the best known approximation factors for the primal problems to randomized approximation algorithms for their dual counterparts with reciprocal approximation factors which are tight with respect to the Unique Games Conjecture. For each APX-complete pair, our algorithms solve a single SDP relaxation and generate feasible solutions for both problems which also provide approximate optimality certificates for each other. Our work utilizes techniques from areas of randomized approximation algorithms, convex optimization, spectral sparsification, as well as Chernoff-type concentration results for random matrices.

Speakers
RZ

Richard Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
VC

Venkat Chandrasekaran

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LV

Lieven Vandenberghe

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
NB

Nathan Benedetto Proenca

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4L: First-order methods for nonsmooth optimization
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: First-order methods for nonsmooth optimization
Chair: Digvijay Boob
Cluster: Nonlinear Optimization

Talk 1: Efficient Subgradient Method for Minimizing Nonsmooth Maximal Value Functions
Speaker: Hanyang Li
Abstract: We consider the minimization of a class of nonsmooth maximal value functions that are piecewise-smooth. Recent implementable Goldstein-style subgradient methods for general Lipschitz functions involve computationally intensive inner loops to approximate the descent direction. In this paper, we introduce a novel subgradient method that eliminates the sophisticated inner loops by employing a regularization technique, significantly improving computational efficiency for minimizing nonsmooth maximal value functions.

Talk 2: Dimension dependence in nonconvex optimization
Speaker: Guy Kornowski
Abstract: Optimization problems that arise in modern machine learning are often neither smooth nor convex, and typically high-dimensional. In this talk we will discuss the intricate role of the dimension in such problems, and compare it to smooth optimization. We will see that some approaches lead to significant gaps in dimension dependencies, yet sometimes these can be eliminated altogether. In particular, we will examine fundamental concepts such as stationarity, smoothing, and zero-order optimization, and show they exhibit exponential, polynomial, and no such gaps, respectively.

Talk 3: On the Sample Complexity of Imitation Learning for Smoothed Model Predictive Control
Speaker: Swati Padmanabhan
Abstract: Recent work in imitation learning has shown that having an expert controller that is both suitably smooth and stable enables stronger guarantees on the performance of the learned controller. However, constructing such smoothed expert controllers for arbitrary systems remains challenging, especially in the presence of input and state constraints. As our primary contribution, we show how such a smoothed expert can be designed for a general class of systems using a log-barrier-based relaxation of a standard Model Predictive Control (MPC) optimization problem. At the crux of this theoretical guarantee on smoothness is a new lower bound we prove on the optimality gap of the analytic center associated with a convex Lipschitz function, which we hope could be of independent interest. We validate our theoretical findings via experiments, demonstrating the merits of our smoothing approach over randomized smoothing.

Speakers
DB

Digvijay Boob

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HL

Hanyang Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Guy Kornowski

Guy Kornowski

Weizmann Institute of Science
Talk title:Dimension dependence in nonconvex optimizationBio:Guy Kornowski is a PhD student at the Weizmann Institute of Science, advised by Prof. Ohad Shamir. During his PhD he interned at Apple ML Research, where he worked with Kunal Talwar and Vitaly Feldman. His research focuses... Read More →
SP

Swati Padmanabhan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4M: Bilevel Optimization for Inverse Problems Part 2
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Bilevel Optimization for Inverse Problems Part 2
Chair: Juan Carlos de los Reyes
Cluster: PDE-constrained Optimization

Talk 1: Linesearch-enhanced inexact forward–backward methods for bilevel optimization
Speaker: Marco Prato
Abstract: Bilevel optimization problems arise in various real-world applications, often being characterized by the impossibility of having the exact objective function and its gradient available. Developing mathematically sound optimization methods that effectively handle inexact information is crucial for ensuring reliable and efficient solutions. In this talk we propose a line-search based algorithm for solving a bilevel optimization problem, where the approximate gradient and function evaluation obeys an adaptive tolerance rule. Our method is based on implicit differentiation under some standard assumptions, and its main novelty with respect to similar approaches is the well posed, inexact line-search procedure using only approximate function values and adaptive accuracy control. This work is partially supported by the PRIN project 20225STXSB, under the National Recovery and Resilience Plan (NRRP) funded by the European Union - NextGenerationEU.

Talk 2: Tensor train solution to uncertain optimization problems with shared sparsity penalty
Speaker: Akwum Onwunta
Abstract: We develop first- and second-order numerical optimization methods to solve non-smooth optimization problems featuring a shared sparsity penalty, constrained by differential equations with uncertainty. To alleviate the curse of dimensionality, we use tensor product approximations. To handle the non-smoothness of the objective function, we introduce a smoothed version of the shared sparsity objective. We consider both a benchmark elliptic PDE constraint and a more realistic topology optimization problem. We demonstrate that the error converges linearly in iterations and the smoothing parameter and faster than algebraically in the number of degrees of freedom, consisting of the number of quadrature points in one variable and tensor ranks. Moreover, in the topology optimization problem, the smoothed shared sparsity penalty reduces the tensor ranks compared to the unpenalized solution. This enables us to find a sparse high-resolution design under a high-dimensional uncertainty.

Talk 3: TBD
Speaker: Juan Carlos de los Reyes
Abstract: TBD

Speakers
avatar for Marco Prato

Marco Prato

Associate Professor, Università di Modena e Reggio Emilia, Italy
Name: Dr. Marco PratoTitle: Associate Professor in Numerical AnalysisAffiliation: Department of Physics, Informatics and Mathematics, University of Modenanad Reggio Emilia, ItalyBio:Dr. Marco Prato was born in Varazze, on the Italian Riviera, in 1980. He received the MSc Degree and... Read More →
JC

Juan Carlos de los Reyes

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4N: Derivative-free stochastic optimization methods I
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Derivative-free stochastic optimization methods I
Chair: Stefan Wild
Cluster: Derivative-free Optimization

Talk 1: Improving the robustness of zeroth-order optimization solvers
Speaker: Stefan Wild
Abstract: Zeroth-order optimization solvers are often deployed in settings where little information regarding a problem's conditioning or noise level is known. An ideal solver will perform well in a variety of challenging settings. We report on our experience developing adaptive algorithms, which leverage information learned online to adapt critical algorithmic features. We illustrate our approach in trust-region-based reduced-space methods and show how trained policies can even be deployed effectively in nonstationary cases, where the noise seen changes over the decision space.

Talk 2: A noise-aware stochastic trust-region algorithm using adaptive random subspaces
Speaker: Kwassi Joseph Dzahini
Abstract: We introduce ANASTAARS, a noise-aware stochastic trust-region algorithm using adaptive random subspace strategies, that is effective not only for low- and moderate-dimensional cases, but also for potentially high-dimensional ones. The proposed method achieves scalability by optimizing random models that approximate the objective within low-dimensional affine subspaces, thereby significantly reducing per-iteration costs in terms of function evaluations. These subspaces and their dimension are defined via Johnson--Lindenstrauss transforms such as those obtained from Haar-distributed orthogonal random matrices. In contrast to previous work involving random subspaces with fixed dimensions, ANASTAARS introduces an adaptive subspace selection strategy. Instead of generating a completely new poised set of interpolation points at each iteration, the proposed method updates the model by generating only a few or even a single new interpolation point, reusing past points (and their corresponding function values) from lower-dimensional subspaces in such a way that the resulting set remains poised. This approach not only introduces a novel way to reduce per-iteration costs in terms of function evaluations, but also avoids constructing random models in fixed-dimension subspaces, resulting in a more efficient and optimization process through the use of adaptive subspace models. Furthermore, to address the observation that model-based trust-region methods perform optimally when the signal-to-noise ratio is high, ANASTAARS incorporates a strategy from noise-aware numerical optimization literature by utilizing an estimate of the noise level in each function evaluation. The effectiveness of the method is demonstrated through numerical experiments conducted on problems within the "quantum approximate optimization algorithm" (QAOA) framework.

Talk 3: A consensus-based global optimization method with noisy objective function
Speaker: Greta Malaspina
Abstract: Consensus based optimization is a derivative-free particles-based method for the solution of global optimization problems. Several versions of the method have been proposed in the literature, and different convergence results have been proved, with varying assumptions on the regularity of the objective function and the initial distribution of the particles. However, all existing results assume the objective function to be evaluated exactly at each iteration of the method. In this work, we extend the convergence analysis of a discrete-time CBO method to the case where only a noisy stochastic estimator of the objective function can be computed at a given point. In particular we prove that under suitable assumptions on the oracle’s noise, the expected value of the mean squared distance of the particles from the solution can be made arbitrarily small in a finite number of iterations. Some numerical results showing the impact of noise are also given.

Speakers
SW

Stefan Wild

Lawrence Berkeley National Laboratory
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KJ

Kwassi Joseph Dzahini

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GM

Greta Malaspina

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4O: Feasible and infeasible methods for optimization on manifolds I
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Feasible and infeasible methods for optimization on manifolds I
Chair: Bin Gao
Cluster: Optimization on Manifolds

Talk 1: A double tracking method for optimization with decentralized generalized orthogonality constraints
Speaker: Xin Liu
Abstract: We consider the decentralized optimization problems with generalized orthogonality constraints, where both the objective function and the constraint exhibit a distributed structure. Such optimization problems, albeit ubiquitous in practical applications, remain unsolvable by existing algorithms in the presence of distributed constraints. To address this issue, we convert the original problem into an unconstrained penalty model by resorting to the recently proposed constraint-dissolving operator. However, this transformation compromises the essential property of separability in the resulting penalty function, rendering it impossible to employ existing algorithms to solve. We overcome this difficulty by introducing a novel algorithm that tracks the gradient of the objective function and the Jacobian of the constraint mapping simultaneously. The global convergence guarantee is rigorously established with an iteration complexity. To substantiate the effectiveness and efficiency of our proposed algorithm, we present numerical results on both synthetic and real-world datasets.

Talk 2: A projected gradient descent algorithm for ab initio fixed-volume crystal structure relaxation
Speaker: Yukuan Hu
Abstract: This paper is concerned with ab initio crystal structure relaxation under a fixed unit cell volume, which is a step in calculating the static equations of state and forms the basis of thermodynamic property calculations for materials. The task can be formulated as an energy minimization with a determinant constraint. Widely used line minimization-based methods (e.g., conjugate gradient method) lack both efficiency and convergence guarantees due to the nonconvex nature of the determinant constraint as well as the significant differences in the curvatures of the potential energy surface with respect to atomic and lattice components. To this end, we propose a projected gradient descent algorithm named PANBB. It is equipped with (i) search direction projections for lattice vectors, (ii) distinct curvature-aware initial trial step sizes for atomic and lattice updates, and (iii) a nonrestrictive line minimization criterion as the stopping rule for the inner loop. It can be proved that PANBB favors theoretical convergence to equilibrium states. Across a benchmark set containing 223 structures from various categories, PANBB achieves average speedup factors of approximately 1.41 and 1.45 over the conjugate gradient method and direct inversion in the iterative subspace implemented in off-the-shelf simulation software, respectively. Moreover, it normally converges on all the systems, manifesting its robustness. As an application, we calculate the static equations of state for the high-entropy alloy AlCoCrFeNi, which remains elusive owing to 160 atoms representing both chemical and magnetic disorder and the strong local lattice distortion. The results are consistent with the previous calculations and are further validated by experimental thermodynamic data.

Talk 3: Efficient optimization with orthogonality constraints via random submanifold approach
Speaker: Andi Han
Abstract: Optimization problems with orthogonality constraints are commonly addressed using Riemannian optimization, which leverages the geometric structure of the constraint set as a Riemannian manifold. This method involves computing a search direction in the tangent space and updating via a retraction. However, the computational cost of the retraction increases with problem size. To improve scalability, we propose a method that restricts updates to random submanifolds, reducing per-iteration complexity. We introduce two submanifold selection strategies and analyze the convergence for nonconvex functions, including those satisfying the Riemannian Polyak–Łojasiewicz condition, as well as for stochastic optimization problems. The approach generalizes to quotient manifolds derived from the orthogonal manifold.

Speakers
BG

Bin Gao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XL

Xin Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YH

Yukuan Hu

Postdoctoral Fellow, CERMICS, École des Ponts, IP Paris
AH

Andi Han

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4P: Recent Advances in Conic Optimization and Machine Learning
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Recent Advances in Conic Optimization and Machine Learning
Chair: Godai Azuma
Cluster: Conic and Semidefinite Optimization

Talk 1: Procrustean Differential Privacy: A Parameter-Scalable Method for Privacy-Preserving Collaborative Learning
Speaker: Keiyu Nosaka
Abstract: Privacy-preserving collaborative learning enables multiple parties to jointly train machine learning models without directly sharing sensitive data. While approaches such as Federated Learning and Homomorphic Encryption offer robust privacy guarantees, they often incur significant computational and communication overhead as model complexity increases. In contrast, multiplicative perturbation techniques promise enhanced efficiency; however, they are typically hampered by increased privacy risks from collusion or by a reliance on extensive deep learning-based training to achieve satisfactory performance. In this work, we introduce a novel framework that bridges these extremes by integrating the analytic Gaussian mechanism of differential privacy with the Generalized Orthogonal Procrustes Problem. Our method delivers adjustable privacy–performance trade-offs through tunable differential privacy parameters, allowing practitioners to balance protection and efficiency according to specific application needs. We substantiate our approach with theoretical guarantees and numerical analyses that evaluate its performance across varying privacy levels, data dimensions, and numbers of collaborating parties.

Talk 2: Facial Structure of Copositive and Completely Positive Cones over a Second-Order Cone
Speaker: Mitsuhiro Nishijima
Abstract: A copositive cone over a second-order cone is the closed convex cone of real symmetric matrices whose associated quadratic forms are nonnegative over the given second-order cone. In this talk, we classify the faces of those copositive cones and their duals (i.e., completely positive cones over a second-order cone), and investigate their dimension and exposedness properties. Then we compute two parameters related to chains of faces of both cones. At the end, we discuss some possible extensions of the results with a view toward analyzing the facial structure of general copositive and completely positive cones.

Talk 3: Exact Semidefinite Relaxations for Safety Verification of Neural Network
Speaker: Godai Azuma
Abstract: We study the accuracy of DeepSDP which is proposed as a semidefinite programming (SDP) based method to measure the safety and the robustness of given neural networks by guaranteeing the bounds of their outputs. The dual problem of the DeepSDP is in fact an SDP relaxation of quadratic constraints representing ReLU activation functions. In this talk, we investigate the exactness of the DeepSDP by using exactness conditions for the general SDP relaxation so that the estimated robustness is improved on. We also discuss our assumptions and some results on the accuracy.

Speakers
KN

Keiyu Nosaka

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MN

Mitsuhiro Nishijima

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GA

Godai Azuma

Assistant Professor, Aoyama Gakuin University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4Q: Moment-SOS Hierarchy: From Theory to Computation in the Real World (I)
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Moment-SOS Hierarchy: From Theory to Computation in the Real World (I)
Chair: Heng Yang
Cluster: Conic and Semidefinite Optimization

Talk 1: Polynomial Matrix Optimization, Matrix-Valued Moments, and Sparsity
Speaker: Jie Wang
Abstract: This talk will introduce the matrix version of the moment-SOS hierarchy for solving polynomial matrix optimization problems. Various types of sparsities will be discussed to improve the scalability of the approach.

Talk 2: Practical non-symmetric cone programming algorithms for sums-of-squares optimization
Speaker: Dávid Papp
Abstract: Sums-of-squares relaxations of polynomial optimization problems are usually solved using off-the-shelf semidefinite programming (SDP) methods, which often leads to at least one of the following two problems when the relaxation order (that is, the degree of the SOS polynomials) is high: (1) the time and space complexity of SDP algorithms grow too fast to be practical; (2) the numerical conditioning of the SDPs is prohibitive to obtain accurate solutions. The talk focuses on how both can be mitigated using polynomial interpolation and replacing all-purpose semidefinite programming algorithms with non-symmetric cone programming methods that can optimize directly over sums-of-squares cones. Generalizations to other nonnegativity certificates (SONC, SAGE) will also be briefly discussed.

Talk 3: A squared smoothing Newton Method for semidefinite programming
Speaker: Ling Liang
Abstract: This paper proposes a squared smoothing Newton method via the Huber smoothing function for solving semidefinite programming problems (SDPs). We first study the fundamental properties of the matrix-valued mapping defined upon the Huber function. Using these results and existing ones in the literature, we then conduct rigorous convergence analysis and establish convergence properties for the proposed algorithm. In particular, we show that the proposed method is well-defined and admits global convergence. Moreover, under suitable regularity conditions, i.e., the primal and dual constraint nondegenerate conditions, the proposed method is shown to have a superlinear convergence rate. To evaluate the practical performance of the algorithm, we conduct extensive numerical experiments for solving various classes of SDPs. Comparison with the state-of-the-art SDP solvers demonstrates that our method is also efficient for computing accurate solutions of SDPs.

Speakers
avatar for Heng Yang

Heng Yang

Assistant Professor, Harvard University
Assistant Professor at Harvard University working on polynomial optimization and semidefinite programming.
JW

Jie Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DP

Dávid Papp

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LL

Ling Liang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4R: Variational Analysis: Theory and Applications II
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Variational Analysis: Theory and Applications II
Chair: Walaa Moursi
Cluster: Nonsmooth Optimization

Talk 1: TBA
Speaker: Stephen Simons
Abstract: TBA

Talk 2: TBA
Speaker: Jon Vanderwerff
Abstract: TBA

Talk 3: TBA
Speaker: Gerald Beer
Abstract: TBA

Speakers
WM

Walaa Moursi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Stephen Simons

Stephen Simons

Professor Emeritus, University of California, Santa Barbara
Name: Dr. Stephen SimonsTitle: Professor Emeritus, University of California, Santa BarbaraAffiliation: University of California, Santa Barbara, Department of MathematicsBio:I am interested in the theory of monotone operators on general Banach space.
JV

Jon Vanderwerff

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GB

Gerald Beer

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4S: Recent Advances on PDE-constrained optimization packages and libraries : Part I
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Recent Advances on PDE-constrained optimization packages and libraries : Part I
Chair: Noemi Petra
Cluster: Computational Software

Talk 1: Empowering Numerical Optimization Across Disciplines with the Rapid Optimization Library (ROL)
Speaker: Denis Ridzal
Abstract: The Rapid Optimization Library (ROL) is a versatile, high-performance C++ library designed to address the complex demands of numerical optimization in various scientific and engineering disciplines. As an open-source effort through the Trilinos Project, ROL provides an extensive collection of state-of-the-art optimization algorithms capable of handling any application, hardware architecture, and problem size. This talk introduces ROL's key features, including its abstract linear algebra interface for universal applicability, modern algorithms for smooth, constrained, stochastic, risk-aware, and nonsmooth optimization, and its unique PDE-OPT application development kit for PDE-constrained optimization. Additionally, ROL offers an easy-to-use Python interface, which enhances its accessibility and usability across a wider range of applications and user communities. We highlight ROL's successful uses in fields ranging from electrodynamics and fluid dynamics to super-resolution imaging and machine learning. ROL's design philosophy, emphasizing vector abstractions, matrix-free interfaces, and a comprehensive suite of optimization algorithms, positions it as an important tool for researchers seeking to push the boundaries of numerical optimization. Authors: Robert Baraldi, Brian Chen, Aurya Javeed, Drew Kouri, Denis Ridzal, Greg von Winckel, Radoslav Vuchkov

Talk 2: CLAIRE: Constrained Large Deformation Diffeomorphic Image Registration
Speaker: Andreas Mang
Abstract: We present numerical methods for optimal control problems governed by geodesic flows of diffeomorphisms. Our work focuses on designing efficient numerical methods and fast computational kernels for high-performance computing platforms. We aim to compute a flow map that establishes spatial correspondences between two images of the same scene, modeled as geodesic flows of diffeomorphisms. The related optimization problem is non-convex and non-linear, leading to high-dimensional, ill-conditioned systems. Our solvers use advanced algorithms for rapid convergence and employ adjoint-based, second-order methods for numerical optimization. We provide results from real and synthetic data to assess convergence rates, time to solution, accuracy, and scalability of our methods. We also discuss strategies for preconditioning the Hessian and showcase results from our GPU-accelerated software package termed CLAIRE, which is optimized for clinically relevant problems and scales to hundreds of GPUs on modern architectures.

Talk 3: PyOED: An Extensible Suite for Data Assimilation and Model-Constrained Optimal Design of Experiments
Speaker: Ahmed Attia
Abstract: This talk gives a high-level overview of PyOED, a highly extensible scientific package that enables rapid development, testing, and benchmarking of model-based optimal experimental design (OED) methods for inverse problems. PyOED brings together variational and Bayesian data assimilation (DA) algorithms for inverse problems, optimal design of experiments, and novel optimization, statistical, and machine learning methods, into an integrated extensible research environment. PyOED is continuously being expanded with a plethora of Bayesian inference, DA, and OED methods as well as new scientific simulation models, observation error models, priors, and observation operators. These pieces are added such that they can be permuted to enable developing and testing OED methods in various settings of varying complexities. Authors: Ahmed Attia (ANL), Abhijit Chowdhary (NCSU), Shady Ahmed (PNNL)

Speakers
NP

Noemi Petra

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AA

Ahmed Attia

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4T: Parametric Optimization Problems
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Parametric Optimization Problems
Chair: Xiaoqi Yang
Cluster: Fixed Points and Variational Inequalities

Talk 1: Recent stability results in parametric optimization
Speaker: Xiaoqi Yang
Abstract: Recover bounds/relative calmness for sparse optimization models are important for algorithmic convergence analysis in machine learning and compressed sensing. Some recent results in sparse optimization obtained by restricted isometry property and restricted eigenvalue conditions will be reviewed. Lipschitz-like property and Lipschitz continuity are at the core of stability analysis. A projectional coderivative of set-valued mappings will be discussed and and be applied to obtain a complete characterization for a set-valued mapping to have the Lipschitz-property relative to a closed and convex set.

Talk 2: Lipschitz continuity of solution multifunctions of extended l_1 regularization problems
Speaker: Kaiwen Meng
Abstract: In this paper we obtain a verifiable sufficient condition for a polyhedral multifunction to be Lipschitz continuous on its domain. We apply this sufficient condition to establish the Lipschitz continuity of the solution multifunction for an extended l_1 regularization problem with respect to the regularization parameter and the observation parameter under the assumption that the data matrix is of full row rank.

Talk 3: Existence of a solution for stochastic multistage vector variational inequality problems
Speaker: Shengkun Zhu
Abstract: In this talk, we will discuss basic form and extensive forms of a stochastic vector variational inequality problem. We will apply KKM theorem to prove the existence of a solution under the monotonicity and hemi-continuity assumptions.

Speakers
XY

Xiaoqi Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Kaiwen Meng

Kaiwen Meng

Professor, Southwestern University of Finance and Economics
Name: Dr. Kaiwen MENGTitle: Professor of Continuous OptimizationAffiliation: Southwestern University of Finance and EconomicsBio:Dr. Kaiwen MENG is a Fun Fact:  
SZ

Shengkun Zhu

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4U: Computational Frameworks and Applications: Hybridizing Continuous and Discrete Optimization in Practice
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Computational Frameworks and Applications: Hybridizing Continuous and Discrete Optimization in Practice
Chair: Jordan Jalving & Marta D'Elia
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Optimal Conceptual Design using Generalized Disjunctive Programming in IDAES/Pyomo
Speaker: E. Soraya Rawlings
Abstract: The need to design more sustainable and energy-efficient processes requires the ability to efficiently pose and solve optimization problems with both discrete and continuous decisions. Traditionally, design optimization problems have been expressed as mixed-integer nonlinear optimization problems (MINLP) using algebraic modeling languages (AMLs; e.g., AIMMS, AMPL, GAMS, JuMP, Pyomo, etc.). A challenge with AMLs is that, not only they combine the structure of the optimization problem with binary variables, but also lack libraries for fast development of process models. An alternative approach is to use an extended mathematical programming environment that provides supplemental capabilities. Relevant in conceptual design are environments with the ability to construct hierarchical models, similar to what is supported in process simulation environments, and the ability to express logical restrictions explicitly in the model. If the environment incorporates both abilities, it can support the construction of design superstructures or topologies that represent all possible combinations of process configurations that we would like to consider in the design process. In this work, we present the application of one such environment to design separation and integrated energy systems. We leverage the open source IDAES platform, which supports the construction of hierarchical process models in Pyomo. We then leverage Generalized Disjunctive Programming to construct design superstructures and systematically convert them to MINLP models that can be solved with standard MINLP solvers. References [1] M. L. Bynum, G. A. Hackebeil, W. E. Hart, C. D. Laird, B. L. Nicholson, J. D. Siirola, J.-P. Watson and D. L. Woodruff, Pyomo - Optimization Modeling in Python, 3rd ed., vol. 67, Springer International Publishing, 2021. [2] Lee, A., Ghouse, J. H., Eslick, J.C., Laird, C.D., Siirola, J.D., Zamarripa, M.A., Gunter, D., Shinn, J. H., Dowling, A. W., Bhattacharyya, D., Biegler, L. T., Burgard, A. P., & Miller, D.C., “The IDAES process modeling framework and model library—Flexibility for process simulation and optimization,” Journal of Advanced Manufacturing and Processing, vol. 3, no. 3, pp. 1-30, 2021. https://doi.org/10.1002/amp2.10095 Disclaimer Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government.

Talk 2: Bridging Continuous and Discrete Optimization in Julia
Speaker: Thibaut Cuvelier
Abstract: TBA

Talk 3: Computation Design: Integrating Implicit Modeling, Meshless Simulation, and Machine Learning for Optimization with Real-World Efficiency
Speaker: Todd Mcdevitt
Abstract: Across every industry, engineers face ever-shrinking time-to-market demands, often needing more time to optimize their designs fully. Traditional CAD and simulation tools fall short of enabling practical usage of optimization and generative design due to persistent bottlenecks in geometry regeneration and meshing. These limitations prevent engineers from fully leveraging the design space while meeting project timelines. In this presentation, we introduce a new paradigm in mechanical design optimization that integrates implicit modeling, meshless simulation methods, and machine learning to address these challenges. Through diverse use cases across industries, we demonstrate how these techniques unlock the practical use of optimization while adhering to project timelines. We will also compare the computational costs of surrogate models versus the direct evaluation of objective functions with the original high-fidelity model. Attendees will gain insights into the practical deployment of optimization workflows in industrial, fast-paced, multi-disciplinary settings.

Speakers
avatar for Thibaut Cuvelier

Thibaut Cuvelier

Software engineer, operations research, Google
Thibaut Cuvelier is currently a software engineer at Google Research, in the Operations Research team. He received a PhD in telecommunications from CentraleSupélec (université Paris-Saclay, France). He is currently working on applications of operations research and reinforcement... Read More →
TM

Todd Mcdevitt

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4V: Recent advances in open-source continuous solvers
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Recent advances in open-source continuous solvers
Chair: Charlie Vanaret
Cluster: Computational Software

Talk 1: A new parallel interior point solver for HiGHS
Speaker: Filippo Zanetti
Abstract: HiGHS is open-source software for linear, mixed-integer and quadratic optimization. It includes an interior point method based on an iterative Krylov solver, that has attracted much attention, in particular from the energy modelling community. In this talk, the latest news on the development of a new interior point solver, based on a direct factorization, are presented. The talk discusses multiple approaches that have been considered and highlights the improvements compared to the existing solver. Issues regarding regularization, accuracy, dealing with dense columns and parallelization are also mentioned.

Talk 2: Uno, a modern Lagrange-Newton solver for nonconvex constrained optimization
Speaker: Charlie Vanaret
Abstract: Derivative-based iterative methods for nonlinearly constrained nonconvex optimization usually share common algorithmic components, such as strategies for computing a descent direction and mechanisms that promote global convergence. Based on this observation, we introduce an abstract framework with eight building blocks that describes most derivative-based iterative methods and unifies their workflows. We then present Uno, a Lagrange-Newton solver that implements our abstract framework and allows the automatic generation of various strategy combinations with no programming effort from the user. Uno is meant to (1) organize mathematical optimization strategies into a coherent hierarchy; (2) offer a wide range of efficient and robust methods that can be compared for a given instance; (3) reduce the cost of development and maintenance of multiple optimization solvers; and (4) enable researchers to experiment with novel optimization strategies while leveraging established subproblem solvers and interfaces to modeling languages. We demonstrate that Uno is highly competitive against state-of-the-art solvers such as filterSQP, IPOPT, SNOPT, MINOS, LANCELOT, LOQO and CONOPT. Uno is available as open-source software under the MIT license at https://github.com/cvanaret/Uno

Talk 3: Solving Direct Optimal Control Problems in Real-Time: A Byrd-Omojokun Funnel Solver for acados
Speaker: David Kiessling
Abstract: We present a nonlinear optimization SQP-type method within the acados software framework for solving direct optimal control in real time. acados is a versatile software framework that implements fast solvers exploiting the specific optimal control problem structure using tailored linear algebra and numerical integration methods. This talk focuses on a novel SQP implementation using a funnel to drive global convergence. We present a novel implementation of a Byrd-Omojokun method avoiding infeasible subproblems. Additionally, we will present numerical results comparing our solver against other state-of-the art optimization solvers.

Speakers
avatar for Filippo Zanetti

Filippo Zanetti

University of Edinburgh
Name: Dr. Filippo ZanettiTitle: HiGHS optimization developerAffiliation: HiGHS - University of EdinburghBio:I am working on developing an interior point solver for LP and convex QP for HiGHS. My main interests are in interior point methods and their implementation, numerical linear... Read More →
avatar for Charlie Vanaret

Charlie Vanaret

Argonne & ZIB
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4W: Nonsmooth Optimization - Theory and Algorithms
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Nonsmooth Optimization - Theory and Algorithms
Chair: Shaoning Han
Cluster: nan

Talk 1: Analysis of a Class of Heaviside Composite Minimization Problems
Speaker: Shaoning Han
Abstract: The minimization of nonclosed functions is a difficult topic that has been minimally studied. Among such functions is a Heaviside composite function that is the composition of a Heaviside function with a possibly nonsmooth multivariate function. Unifying a statistical estimation problem with hierarchical selection of variables and a sample average approximation of composite chance constrained stochastic programs, a Heaviside composite optimization problem is one whose objective and constraints are defined by sums of possibly nonlinear multiples of such composite functions. In this talk, we first present difficulties of tackling such problems using existing optimization techniques. Then we establish stationarity conditions for the problem, and introduce a sufficient condition, termed local convex-like property, under which the proposed stationary point is a local minimizer. Finally, we briefly discuss solution strategies via lifting and reformulation techniques. For details of this work, please refer to "Han S, Cui Y, Pang J-S (2024) Analysis of a class of minimization problems lacking lower semicontinuity. Mathematics of Operations Research."

Talk 2: Non-smooth stochastic gradient descent using smoothing functions
Speaker: Jingfu Tan
Abstract: In this talk, we address stochastic optimization problems involving a composition of a non-smooth outer function and a smooth inner function, a formulation frequently encountered in machine learning and operations research. To deal with the non-differentiability of the outer function, we approximate the original non-smooth function using smoothing functions, which are continuously differentiable and approach the original function as a smoothing parameter goes to zero (at the price of increasingly higher Lipschitz constants). The proposed smoothing stochastic gradient method iteratively drives the smoothing parameter to zero at a designated rate. We establish convergence guarantees under strongly convex, convex, and nonconvex settings, proving convergence rates that match known results for non-smooth stochastic compositional optimization. In particular, for convex objectives, smoothing stochastic gradient achieves a~$1/T^{1/4}$ rate in terms of the number of stochastic gradient evaluations. We further show how the general compositional and finite sum compositional problems (widely used frameworks in large-scale machine learning and risk-averse optimization) fit the assumptions needed for the rates (unbiased gradient estimates, bounded second moments, and accurate smoothing errors). We will present numerical results indicating that smoothing stochastic gradient descent is a competitive method for certain classes of problems.

Talk 3: (Canceled) Convergence analysis of the 9th Chebyshev Method for Nonconvex, Nonsmooth Optimization Problems
Speaker: Jiabao Yang
Abstract: Ushiyama-Sato-Matsuo (2022) (hereafter referred to as [USM]) showed that some optimization methods can be regarded as numerical analysis methods for ordinary differential equations. [USM] proposed the 9th Chebyshev method, an optimization method that allows a larger step width than the steepest descent method, under the assumption that the objective function is convex and differentiable. Observing the left edge of the stability domain, we see the 9th Chebyshev method should allow about 78 times larger steps than the steepest descent method. On the other hand, the 9th Chebyshev method requires 9 evaluations of the gradient per iteration, while the steepest descent method requires only one. Considering most calculation time is spent on the gradient, we expect that the 9th Chebyshev method converges 78/9 = 8.7 times faster. However, there are many cases in the world where differentiable, smooth, convex, etc. cannot be used. For example, problems in image analysis and voice recognition require the use of nondifferentiable, nonsmooth, and nonconvex functions. The steepest descent method or the 9th Chebyshev method proposed by [USM] assumes that the objective function is convex and differentiable, and thus cannot be applied to problems with an objective function that is not necessarily differentiable. For nonconvex optimization problems, finding a good local optimal solution is a realistic goal, and Riis-Ehrhardt-Quispel-Scholieb (2022) (hereafter referred to as [REQS]) introduced an alternative concept of differentiation called “Clarke subdifferential” for functions that are not always differentiable, and proposed an optimization method that can be applied to objective functions that are not necessarily differentiable and convex. Although the objective function is not assumed to be differentiable or convex, such as being locally Lipschitz continuous, [REQS] prove that the optimization method converges to the set of Clarke stationary points defined in the framework of Clarke subdifferential under the condition that some assumptions are satisfied. In this talk, based on the proof method of [REQS], we propose a method that combines the method of [REQS] and the method of [USM] and prove that the proposed method converges to the set of Clarke stationary points under the same objective function assumption as [REQS]. Refernces: 1, Kansei Ushiyama, Shun Sato, Takayasu Matsuo, "Deriving efficient optimization methods based on stable explicit numerical methods", JSIAM Letters Vol.14 (2022) pp.29–32 2, Erlend S.Riis, Matthias J.Ehrhardt, G.R.W.Quispel, Carola-Bibiane Schonlieb, "A Geometric Integration Approach to Nonsmooth, Nonconvex Optimisation", Foundations of Computational Mathematics (2022) 22:1351–1394

Speakers
SH

Shaoning Han

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JT

Jingfu Tan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JY

Jiabao Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4X: Algorithms for Nonconvex Problems
Tuesday July 22, 2025 10:30am - 11:45am PDT
Session: Algorithms for Nonconvex Problems
Chair: Yulin Peng
Cluster: nan

Talk 1: NEW ALGORITHMS FOR HARD OPTIMIZATION problems PROBLEMS.
Speaker: Aharonl Ben-Tal
Abstract: NEW ALGORITHMS FOR HARD (NONCONVEX) OPTIMIZATION PROBLEMS. The problems addressed in this talk are: (1) Max of convex function (2) Max of max of convex function (3) Max of Difference of convex functions. Almost all existing algorithms for such problems suffer from might be called “the curse of obtaining a good starting point”. In our algorithms a starting point is computed by employing only tractable methods for convex problems. The core algorithm on which the algorithms for problems (2) and (3) are based, is the COMAX algorithm developed for problem (1), See Ben-Tal, A. and Roos E., "An Algorithm for Maximizing a Convex Function Based on its Minimizer". INFORMS Journal on Computing Volume: 34, Number: 6 (November-December 2022): 3200-

Talk 3: Conditional Infimum, Hidden Convexity and the S-Procedure
Speaker: Michel De Lara

Speakers
AB

Aharonl Ben-Tal

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MD

Michel De Lara

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 4Y
Tuesday July 22, 2025 10:30am - 11:45am PDT
Tuesday July 22, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

11:45am PDT

Lunch 2 (provided)
Tuesday July 22, 2025 11:45am - 1:15pm PDT
Taco Buffet
Tuesday July 22, 2025 11:45am - 1:15pm PDT
USC Founder's / Hutton Park 3551 Trousdale Pkwy, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5A: Methods for Large-Scale Nonlinear Optimization V
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Methods for Large-Scale Nonlinear Optimization V
Chair: Raghu Bollapragada
Cluster: Nonlinear Optimization

Talk 1: The Interior-Point Frank-Wolfe Method for Minimizing Smooth Convex Functionals over Linearly Constrained Probability Spaces
Speaker: Di Yu
Abstract: Several important and large problem classes including optimal experimental design, emergency volunteer response, spectral optimal transport, and certain queuing problems can be posed as that of minimizing a smooth convex functional over the space of compactly supported probability measures subject to a linear constraint map. We introduce the interior-point Frank-Wolfe (IPFW) method for solving such problems, where a sequence of barrier problems is constructed as a way to handle the constraints, each of which is solved to increasing accuracy using a Frank-Wolfe method. Importantly, the Frank-Wolfe sub-problems are shown to have a ``closed-form'' solution expressed in terms of the constraint functionals and the influence function of the objective. The sequence of probability measures resulting from the IPFW framework is shown to converge in a certain sense to the correct solution, along with a complexity calculation. The problem and the proposed solution is illustrated through examples.

Talk 2: Local convergence of adaptively regularized tensor methods
Speaker: Karl Welzel
Abstract: Tensor methods are methods for unconstrained continuous optimization that can incorporate derivative information of up to order p > 2 by computing a step based on the pth-order Taylor expansion at each iteration. The most important among them are regularization-based tensor methods which have been shown to have optimal worst-case iteration complexity of finding an approximate minimizer. Moreover, as one might expect, this worst-case complexity improves as p increases, highlighting the potential advantage of tensor methods. Still, the global complexity results only guarantee pessimistic sublinear rates, so it is natural to ask how local rates depend on the order of the Taylor expansion p. In the case of functions that are uniformly convex of order q (ones that around the minimizer x* grow like the distance to x* to the qth power) and a fixed regularization parameter, the answer is given in a paper by Doikov and Nesterov from 2022: we get (p/(q-1))th-order local convergence of function values and gradient norms, if p > q-1. In particular, when the Hessian is positive definite at the minimizer (q=2) we get pth-order convergence, but also when the Hessian is singular at x* (q > 2) superlinear convergence (compared to Newton's linear convergence) is possible as long as enough derivatives are available. The value of the regularization parameter in their analysis depends on the Lipschitz constant of the pth derivative. Since this constant is not usually known in advance, adaptive regularization methods are more practical. We extend the local convergence results to locally uniformly convex functions and fully adaptive methods. We discuss how for p > 2 it becomes crucial to select the "right" minimizer of the regularized local model in each iteration to ensure all iterations are eventually successful. Counterexamples show that in particular the global minimizer of the subproblem is not suitable in general. If the right minimizer is used, the (p/(q-1))th-order local convergence is preserved, otherwise the rate stays superlinear but with an exponent arbitrarily close to one depending on the algorithm parameters.

Talk 3: Noise-Aware Sequential Quadratic Programming for Equality Constrained Optimization with Rank-Deficient Jacobians
Speaker: Jiahao Shi
Abstract: We propose and analyze a sequential quadratic programming algorithm for minimizing a noisy nonlinear smooth function subject to noisy nonlinear smooth equality constraints. The algorithm uses a step decomposition strategy and, as a result, is robust to potential rank-deficiency in the constraints, allows for two different step size strategies, and has an early stopping mechanism. Under linear independence constraint qualifications, convergence is established to a neighborhood of a first-order stationary point, where the radius of the neighborhood is proportional to the noise level in the objective function and constraints. Moreover, in the rank deficient setting, convergence to a neighborhood of an infeasible stationary point is established. Numerical experiments demonstrate the efficiency and robustness of the proposed method.

Speakers
RB

Raghu Bollapragada

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DY

Di Yu

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KW

Karl Welzel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JS

Jiahao Shi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5B: Continuous and Discrete Optimization
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Continuous and Discrete Optimization
Chair: Marcia Fampa
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Stochastic Iterative Methods for Linear Problems with Adversarial Corruption
Speaker: Jamie Haddock
Abstract: Stochastic iterative methods, like stochastic gradient descent or the randomized Kaczmarz method, have gained popularity in recent times due to their amenability to large-scale data and distributed computing environments. This talk will focus on variants of the randomized Kaczmarz methods developed for problems with significant or adversarial corruption present in the problem-defining data. This type of corruption arises frequently in applications like medical imaging, sensor networks, error correction, and classification of mislabeled data. We will focus on recent results for linear feasibility problems and tensor regression problems.

Talk 2: Volume formulae for the convex hull of the graph of a $n$-monomial in some special cases
Speaker: Emily Speakman
Abstract: The spatial branch-and-bound algorithmic framework, used for solving non-convex mixed-integer nonlinear optimization problems, relies on obtaining quality convex approximations of the non-convex substructures in a problem formulation. A common example is a simple monomial, $y=x_1x_2, \dots, x_n$, defined over the box $[a_1, b_1] \times [a_2, b_2] \times \dots \times [a_n, b_n] \subseteq \R^n$. There are many choices of convex set that could be used to approximate this solution set, with the (polyhedral) convex hull giving the “tightest” or best possible outer approximation. By computing the volume of the convex hull, we obtain a measure that can be used to evaluate other convex outer approximations in comparison. In previous work, we have obtained a formula for the volume of the convex hull of the graph of a trilinear monomial (i.e., $n=3$) in terms of the $6 = 2n$ box parameters. Here, we seek to extend our work to the case of general $n$ by making additional assumptions on the box domain. In particular, we assume that only $k$ variables have a non-zero lower bound. In this work, we consider $k=1,2,3$, and conjecture the volume of the convex hull in each case. Moreover, we provide a proof for the case $k=1$.

Talk 3: New hierarchies for disjoint bilinear programs
Speaker: Mohit Tawarmalani
Abstract: Disjoint bilinear programs are mathematical optimization problems involving minimization of a bilinear function over a Cartesian product of polytope. In this paper, we iteratively obtain, in closed-form, rational functions that are barycentric coordinates of successively tighter outer-approximations of a polytope. This procedure converges in m steps, where m is the number of constraints describing the polytope. Using this procedure, we construct a finite hierarchy of relaxations that in m steps describes the convex hull of bilinear functions over the feasible region providing a linear reformulation for disjoint bilinear programming.

Speakers
JH

Jamie Haddock

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ES

Emily Speakman

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MT

Mohit Tawarmalani

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5C: AI Meets Optimization (Part 2)
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: AI Meets Optimization (Part 2)
Chair: Wotao Yin
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: TBD
Speaker: Jan Drgona
Abstract: TBD

Talk 2: TBD
Speaker: Jialin Liu
Abstract: TBD

Talk 3: TBD
Speaker: Junchi Yan
Abstract: TBD

Speakers
JD

Jan Drgona

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Jialin Liu

Jialin Liu

Assistant Professor, University of Central Florida
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JY

Junchi Yan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5D: Recent advances in algorithms for large-scale optimization (I)
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Recent advances in algorithms for large-scale optimization (I)
Chair: Xudong Li
Cluster: Computational Software

Talk 1: Infeasible Model Analysis In the Optverse Solver
Speaker: Zirui Zhou
Abstract: Isolating an Irreducible Infeasible Subset (IIS) of constraints is the best way to analyze an infeasible optimization model. The OptVerse solver incorporates very fast algorithms for this purpose. The LP analyzer takes advantage of the presolver to isolate a small subset of constraints for conventional analysis, whether or not the presolve detects infeasibility. The MIP analyzer uses new techniques to very quickly find an IIS that includes the integrality restrictions. Experimental results are given.

Talk 2: HOT: An Efficient Halpern Accelerating Algorithm for Optimal Transport Problems
Speaker: Yancheng Yuan
Abstract: This talk introduces an efficient HOT algorithm for solving the optimal transport (OT) problems with finite supports. We particularly focus on an efficient implementation of the HOT algorithm for the case where the supports are in $\mathbb{R}^2$ with ground distances calculated by $L_2^2$-norm. Specifically, we design a Halpern accelerating algorithm to solve the equivalent reduced model of the discrete OT problem. Moreover, we derive a novel procedure to solve the involved linear systems in the HOT algorithm in linear time complexity. Consequently, we can obtain an $\varepsilon$-approximate solution to the optimal transport problem with $M$ supports in $O(M^{1.5}/\varepsilon)$ flops, which significantly improves the best-known computational complexity. We further propose an efficient procedure to recover an optimal transport plan for the original OT problem based on a solution to the reduced model, thereby overcoming the limitations of the reduced OT model in applications that require the transport map. We implement the HOT algorithm in PyTorch and extensive numerical results show the superior performance of the HOT algorithm compared to existing state-of-the-art algorithms for solving the OT problems.

Talk 3: DNNLasso: Scalable Graph Learning for Matrix-Variate Data
Speaker: Meixia Lin
Abstract: We consider the problem of jointly learning row-wise and column-wise dependencies of matrix-variate observations, which are modelled separately by two precision matrices. Due to the complicated structure of Kronecker-product precision matrices in the commonly used matrix-variate Gaussian graphical models, a sparser Kronecker-sum structure was proposed recently based on the Cartesian product of graphs. However, existing methods for estimating Kronecker-sum structured precision matrices do not scale well to large scale datasets. In this work, we introduce DNNLasso, a diagonally non-negative graphical lasso model for estimating the Kronecker-sum structured precision matrix, which outperforms the state-of-the-art methods by a large margin in both accuracy and computational time.

Speakers
XL

Xudong Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZZ

Zirui Zhou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YY

Yancheng Yuan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ML

Meixia Lin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5E: Advances in Mixed-Integer Optimization
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Advances in Mixed-Integer Optimization
Chair: Alberto Del Pia
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Convexification Techniques For Logical Implication Constraints Involving Cardinality Requirements
Speaker: Jean-Philippe Richard
Abstract: Cardinality requirements and implications between groups of distinct variables are pervasive in applications and are often modeled through the use of integer programming techniques. We describe a general constructive scheme that allows for the convex hull of sets involving logical implication constraints relating the cardinality of groups of variables to be derived in a higher dimensional space. We also discuss aspects of projecting the formulations. We provide simple illustrative applications of the scheme, which subsume existing results in the literature.

Talk 2: Transfer Theorems in Mixed-Integer Convex Optimization
Speaker: Phillip Kerger
Abstract: In this talk, we will present two lines of work that explore the transferability of results between different settings in optimization. First, we will show how performance guarantees from noise-free convex optimization can be adapted to the stochastic setting, even when mixed-integer variables are present. This is achieved through a black-box transfer approach that applies broadly to first-order methods. Second, we will discuss how complexity results from continuous convex optimization can be extended to the mixed-integer setting, which leads to new lower bounds under various oracles, such as those with partial first-order information. Such black-box approaches are especially appealing to have results in easier-to-analyze cases immediately transfer to more complex ones. Finally, some remaining open questions will be discussed.

Talk 3: Extended formulations for some class of Delta-modular IPs
Speaker: Luze Xu
Abstract: Conforti et al. give a compact extended formulation for a class of bimodular-constrained integer programs, namely those that model the stable set polytope of a graph with no disjoint odd cycles. We extend their techniques to design compact extended formulations for the integer hull of translated polyhedral cones whose constraint matrix is strictly $\Delta$-modular and has rows that represent a cographic matroid. Our work generalizes the important special case from Conforti et al. concerning 4-connected graphs with odd cycle transversal number at least 4. We also discuss the necessity of our assumptions. This is joint work with Joseph Paat and Zach Walsh.

Speakers
JR

Jean-Philippe Richard

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PK

Phillip Kerger

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LX

Luze Xu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5F: Adaptive Methods
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Adaptive Methods
Chair: Ilyas Fatkhullin
Cluster: Optimization For Data Science

Talk 1: The Price of Adaptivity in Stochastic Convex Optimization
Speaker: Oliver Hinder
Abstract: We prove impossibility results for adaptivity in non-smooth stochastic convex optimization. Given a set of problem parameters we wish to adapt to, we define a ``price of adaptivity'' (PoA) that, roughly speaking, measures the multiplicative increase in suboptimality due to uncertainty in these parameters. When the initial distance to the optimum is unknown but a gradient norm bound is known, we show that the PoA is at least logarithmic for expected suboptimality, and double-logarithmic for median suboptimality. When there is uncertainty in both distance and gradient norm, we show that the PoA must be polynomial in the level of uncertainty. Our lower bounds nearly match existing upper bounds, and establish that there is no parameter-free lunch. En route, we also establish tight upper and lower bounds for (known-parameter) high-probability stochastic convex optimization with heavy-tailed and bounded noise, respectively.

Talk 2: Adaptive Online Learning and Optimally Scheduled Optimization
Speaker: Ashok Cutkosky
Abstract: In this talk I will describe some recent advances in online learning, and how these advances result in improved algorithms for stochastic optimization. We will first describe new online optimization algorithms that achieve optimal regret with neither prior knowledge of Lipschitz constants nor bounded domain assumptions, which imply stochastic optimization algorithms that perform as well as SGD with an optimally-tuned learning rate. We will then survey new and improved conversions from online to stochastic optimization that shed light on heuristic learning rate schedules popular in practice, and illustrate how this analysis allows us to begin investigation into identifying an optimal schedule of learning rates. This is in contrast to most literature on adaptive stochastic optimization that typically seeks to compete only with a single fixed learning rate. We will conclude by highlighting open problems in both online and stochastic optimization.

Talk 3: Unveiling the Power of Adaptive Methods Over SGD: A Parameter-Agnostic Perspective
Speaker: Junchi Yang
Abstract: Adaptive gradient methods are popular in optimizing modern machine learning models, yet their theoretical benefits over vanilla Stochastic Gradient Descent (SGD) remain unclear. We examines the convergence of SGD and adaptive methods when their hyperparameters are set without knowledge of problem-specific parameters. First, for smooth functions, we compare SGD to well-known adaptive methods like AdaGrad, Normalized SGD with Momentum (NSGD-M), and AMSGrad. While untuned SGD attains the optimal convergence rate, it comes at the expense of an unavoidable exponential dependence on the smoothness constant. In contrast, several adaptive methods reduce this exponential dependence to polynomial. Secondly, for a broader class of functions characterized by (L0, L1) smoothness, SGD fail without proper tuning. We show NSGD-M achieves a near-optimal rate, despite an exponential dependence on the L1 constant, which we show is unavoidable for a family of normalized momentum methods.

Speakers
IF

Ilyas Fatkhullin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
OH

Oliver Hinder

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AC

Ashok Cutkosky

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JY

Junchi Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5G: Recent Advances in Large-scale Optimization III
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Recent Advances in Large-scale Optimization III
Chair: Salar Fattahi
Cluster: Nonlinear Optimization

Talk 1: Convexification of a class of optimization problems with step functions
Speaker: Andres Gomez
Abstract: We study nonlinear optimization problems with discontinuous step functions. This class of problems arises often in statistical problems, modeling correct/incorrect predictions with linear classifiers, fairness and nearly isotonic regression. We discuss special classes of problems where convex hulls can be computed explicitly, and discuss algorithmic implementations of the convexifications. Our computational results indicate a good statistical estimators can be obtained from the optimal solutions of the convex relaxations, and that exact methods can be substantially improved in some cases.

Talk 2: AGDA+: Proximal Alternating Gradient Descent Ascent Method with a Nonmonotone Adaptive Step-Size Search for Nonconvex Minimax Problems
Speaker: Xuan Zhang
Abstract: We consider double-regularized nonconvex-strongly concave (NCSC) minimax problems of the form (P) : minx maxy g(x) + f (x,y) − h(y), where g, h are closed convex, f is L-smooth in (x, y) and strongly concave in y. We propose a proximal alternating gradient descent ascent method AGDA+ that can adaptively choose nonmonotone primal-dual stepsizes to compute an approximate stationary point for (P) without requiring the knowledge of the global Lipschitz constant L. Using a nonmonotone step-size search (backtracking) scheme, AGDA+ stands out by its ability to exploit the local Lipschitz structure and eliminates the need for precise tuning of hyper-parameters. AGDA+ achieves the optimal iteration complexity of O(ε-2) and it is the first step-size search method for NCSC minimax problems that require only O(1) calls to ∇f per backtracking iteration. We also discuss how AGDA+ can easily be extended to search for μ as well. The numerical experiments demonstrate its robustness and efficiency.

Talk 3: Towards Weaker Variance Assumptions for Stochastic Optimization
Speaker: Ahmet Alacaoglu
Abstract: We revisit a classical assumption for analyzing stochastic gradient algorithms where the squared norm of the stochastic subgradient (or the variance for smooth problems) is allowed to grow as fast as the squared norm of the optimization variable. We contextualize this assumption in view of its inception in the 1960s, its seemingly independent appearance in the recent literature, its relationship to weakest-known variance assumptions for analyzing stochastic gradient algorithms, and its relevance even in deterministic problems for non-Lipschitz nonsmooth convex optimization. We build on and extend a connection recently made between this assumption and the Halpern iteration. For convex nonsmooth, and potentially stochastic, optimization we provide horizon-free algorithms with last-iterate rates. For problems beyond simple constrained optimization, such as convex problems with functional constraints, we obtain rates for optimality measures that do not require boundedness of the feasible set.

Speakers
SF

Salar Fattahi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AG

Andres Gomez

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XZ

Xuan Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AA

Ahmet Alacaoglu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5H: Preference Robust Optimization
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Preference Robust Optimization
Chair: Wenjie Huang
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: conformal inverse optimization
Speaker: Erick Delage
Abstract: TBA

Talk 2: Preference robust optimization with quasi-concave choice functions in multi-attribute decision making: characterization and computation
Speaker: Wenjie Huang
Abstract: TBA

Talk 3: Adaptive Preference Elicitation in Preference Robust CPT-Based Shortfall
Speaker: Sainan Zhang
Abstract: TBA

Speakers
ED

Erick Delage

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Wenjie Huang

Wenjie Huang

The University of Hong Kong
Dr. Wenjie Huang is in Department of Data and Systems Engineering and Musketeers Foundation Institute of Data Science, The University of Hong Kong (HKU). He received B.S. degree in Industrial Engineering and Management, with minor in Economics, from Shanghai Jiao Tong University in... Read More →
SZ

Sainan Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5I: Dynamic Optimization: Deterministic and Stochastic Continuous-time Models II
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Dynamic Optimization: Deterministic and Stochastic Continuous-time Models II
Chair: Cristopher Hermosilla
Cluster: Multi-agent Optimization and Games

Talk 1: Average Cost Problems Subject to State Constraints
Speaker: Nathalie Khalil
Abstract: Control systems with unknown parameters provide a natural framework for modeling uncer- tainties in various applications. In this work, we focus on pathwise state-constrained optimal control problems where these unknown parameters affect the system’s dynamics, cost function, endpoint constraint, and state constraint. The objective is to minimize a cost criterion expressed in integral form, the so-called “average cost”, with the cost function evaluated relative to a refer- ence probability measure defined over the set of unknown parameters. For this class of problems, we derive necessary optimality conditions. By using an average cost criterion, this approach offers an alternative to traditional minimax or robust optimization methods.

Talk 2: Degeneration in Optimal Control Problems with Non-regular Mixed Constraints
Speaker: Karla Cortez
Abstract: In this talk we will discuss the emergence of the degeneration phenomenon in the necessary conditions derived in recent literature on optimal control problems with non-regular mixed constraints. We will examine how the lack of regularity in these constraints can lead to trivial multipliers, hindering the applicability of classical optimality conditions. To address this issue, we will present non-degeneration conditions that ensure the existence of non-trivial multipliers in these problems and we will illustrate their potential through examples and preliminary results. The talk will conclude with a discussion on future directions for extending and validating these findings.

Talk 3: Uniqueness of Multipliers in Optimal Control Problems
Speaker: Jorge Becerril
Abstract: In this talk, we explore conditions for the uniqueness of multipliers in optimal control problems across different settings. We start with the simplest case involving only isoperimetric constraints, highlighting the connection between uniqueness and key concepts such as regularity and normality. Next, we examine optimal control problems with regular mixed constraints, focusing on piecewise continuous controls. In this context, the continuity assumption allows us to relate uniqueness to the observability of a certain no-input linear system. Under additional assumptions, using results from viability theory, we can establish a one-to-one correspondence between the set of Lagrange multipliers and the initial value of certain set-valued function. Lastly, we discuss ongoing efforts to extend these findings to the case of essentially bounded controls.

Speakers
CH

Cristopher Hermosilla

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Nathalie Khalil

Nathalie Khalil

Researcher, University of Porto, Portugal
I am a Researcher and Lecturer at the Department of Electrical Engineering and Computer Science of the Faculty of Engineering at the University of Porto. I specialize in optimal control theory and its applications to engineering challenges.I have over 10 years of research experience... Read More →
KC

Karla Cortez

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JB

Jorge Becerril

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5J: Models and Algorithms for Optimization with Rare Events
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Models and Algorithms for Optimization with Rare Events
Chair: Anirudh Subramanyam
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Decision-Making under Extreme Risks: Configuring Optimization Algorithms for Rare-Event Optimization
Speaker: Henry Lam
Abstract: We consider stochastic optimization where the goal is not only to optimize an average-case objective, but also mitigate the occurrence of rare but catastrophic events. This problem, which is motivated from emerging applications such as safe AI, requires an integration of variance reduction methods into sampling-based optimization algorithms in order to attain sufficient solution accuracy. However, we explain how natural variance-reduction-optimization integration, even executed in an adaptive fashion studied by recent works, encounters fundamental challenges. On a high level, the challenge arises from the extreme sensitivity of tail-based objectives with respect to the decision variables, which renders the failure of traditional Lipschitz-based analyses. We offer some potential remedies and supporting numerical results.

Talk 2: Risk-Aware Path Integral Diffusion to Sample Rare Events
Speaker: Michael Chertkov
Abstract: TBD

Talk 3: Scaling Scenario-Based Chance-Constrained Optimization under Rare Events
Speaker: Jaeseok Choi
Abstract: Chance-constrained optimization is a suitable modeling framework for mitigating extreme event risk in many practical settings. The scenario approach is a popular solution method for chance-constrained problems, due to its straightforward implementation and ability to preserve problem structure. However, for safety-critical applications where violating constraints is nearly unacceptable, the scenario approach becomes computationally infeasible due to the excessively large sample sizes it demands. We address this limitation with a new yet straightforward decision-scaling technique. Our method leverages large deviation principles and relies on only mild nonparametric assumptions about the underlying uncertainty distributions. The method achieves an exponential reduction in sample size requirements compared to the classical scenario approach for a wide variety of constraint structures, while also guaranteeing feasibility with respect to the uncertain constraints. Numerical experiments spanning engineering and management applications show that our decision-scaling technique significantly expands the scope of problems that can be solved both efficiently and reliably.



Speakers
AS

Anirudh Subramanyam

Assistant Professor, Pennsylvania State University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HL

Henry Lam

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MC

Michael Chertkov

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5K: First-order methods for nonsmooth and constrained optimization problems
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: First-order methods for nonsmooth and constrained optimization problems
Chair: Digvijay Boob
Cluster: Nonlinear Optimization

Talk 1: Linearly Convergent Algorithms for Nonsmooth Problems with Unknown Smooth Pieces
Speaker: Zhe Zhang
Abstract: Non-smoothness is a major bottleneck to efficient optimization. In the absence of smoothness, the theoretical convergence rates drop from linear to sublinear for convex programs, and become orders of magnitude worse for nonconvex programs. Moreover, this huge is known to be unimprovable in general. We focus on mild, structured non-smooth problems: piecewise smooth (PWS) functions whose domain can be partitioned into subsets such that restricted to each subset the function is smooth. PWS functions cover ReLU functions, L1-penalties, and min-max saddle point problems as special cases. In particular, we study convex PWS functions for which we present globally linear convergent methods under the quadratic growth condition; as a corollary, we improve the iteration complexity for solving weakly convex PWS problems by orders of magnitude. Importantly, our method does not require any knowledge about individual smooth pieces, and is thus applicable even to general non-smooth programs exhibiting local PWS structure.

Talk 2: Primal-Dual Algorithm with Last iterate Convergence Guarantees for Stochastic Convex Optimization Problems
Speaker: Mohammad Khalafi
Abstract: We provide the first method that obtains the best-known convergence rate guarantees on the last iterate for stochastic composite nonsmooth convex function-constrained optimization problems. Our novel and easy-to-implement algorithm is based on the augmented Lagrangian technique and uses a new linearized approximation of constraint functions, leading to its name, the Augmented Constraint Extrapolation (Aug-ConEx) method. We show that Aug-ConEx achieves convergence rate in the nonsmooth stochastic setting without any strong convexity assumption and for the same problem with strongly convex objective function. While optimal for nonsmooth and stochastic problems, the Aug-ConEx method also accelerates convergence in terms of Lipschitz smoothness constants to and in the aforementioned cases, respectively. To our best knowledge, this is the first method to obtain such differentiated convergence rate guarantees on the last iterate for a composite nonsmooth stochastic setting without additional factors. We validate the efficiency of our algorithm by comparing it with a state-of-the-art algorithm through numerical experiments.

Talk 3: An Optimal Method for Minimizing Heterogeneously Smooth and Convex Compositions
Speaker: Aaron Zoll
Abstract: This talk will discuss a universal, optimal algorithm for convex minimization problems of the composite form f(x) + h(g_1(x), ..., g_m(x)). We allow each component function g_i(x) to independently range from being nonsmooth Lipschitz to smooth and from convex to strongly convex, described by notions of Holder continuous gradients and uniform convexity. We note that although the objective is built from a heterogeneous combination of components, it does not necessarily possess any smoothness, Lipschitzness, or any favorable structural properties. Regardless, our proposed sliding accelerated gradient method converges at least as fast as the optimal rate guarantees in terms of oracle access to (sub)gradients of each g_i seperately. Furthermore, given access to an estimate of the initial distance to optimal, we provide a “mostly parameter-free” variant. As a key application, fixing h as a nonpositive indicator function, this model readily captures functionally constrained minimization f(x) subject to g_i(x) \leq 0. Our algorithm and analysis are directly inspired by the Q-analysis technique developed for such smooth constrained minimization by Zhang and Lan. Our theory recovers their accelerated guarantees and extends them to benefit from heterogeneously smooth and convex constraints.

Speakers
DB

Digvijay Boob

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZZ

Zhe Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MK

Mohammad Khalafi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AZ

Aaron Zoll

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5L: Optimization for Machine Learning and AI
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Optimization for Machine Learning and AI
Chair: Tianbao Yang
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Model Developmental Safety: A Constrained Optimization Approach
Speaker: Gang Li
Abstract: In this talk, we introduce introduce model developmental safety as a guarantee of a learning system such that in the model development process the new model should strictly preserve the existing protected capabilities of the old model while improving its performance on target tasks. To ensure the model developmental safety, we present a safety-centric framework by formulating the model developmental safety as data-dependent constraints. Under this framework, we study how to develop a pretrained vision-language model (aka the CLIP model) for acquiring new capabilities or improving existing capabilities of image classification. We propose an efficient constrained optimization algorithm with theoretical guarantee and use its insights to finetune a CLIP model with task-dependent heads for promoting the model developmental safety. Our experiments on improving vision perception capabilities on autonomous driving and scene recognition datasets demonstrate the efficacy of the proposed approach.

Talk 2: Provable Accelerated Convergence of Nesterov’s Momentum for Deep ReLU Neural Networks
Speaker: Jasper Liao
Abstract: Current state-of-the-art analyses on the convergence of gradient descent for training neural networks focus on characterizing properties of the loss landscape, such as the Polyak-Lojaciewicz (PL) condition and the restricted strong convexity. While gradient descent converges linearly under such conditions, it remains an open question whether Nesterov’s momentum enjoys accelerated convergence under similar settings and assumptions. In this work, we consider a new class of objective functions, where only a subset of the parameters satisfies strong convexity, and show Nesterov’s momentum achieves acceleration in theory for this objective class. We provide two realizations of the problem class, one of which is deep ReLU networks, which constitutes this work as the first that proves an accelerated convergence rate for non-trivial neural network architectures.

Talk 3: In-processing Methods for Training Partially Fair Predictive Models Based on Difference-of-Convex Constraints
Speaker: Qihang Lin
Abstract: Fairness in machine learning has become a critical concern, particularly in high-stakes applications. Existing approaches often focus on achieving full fairness across all score ranges generated by predictive models, ensuring fairness in both high and low-scoring populations. However, this stringent requirement can compromise predictive performance and may not align with the practical fairness concerns of stakeholders. In this work, we propose a novel framework for building partially fair machine learning models, which enforce fairness within a specific score range of interest, such as the middle range where decisions are most contested, while maintaining flexibility in other regions. We introduce two statistical metrics to rigorously evaluate partial fairness within a given score range, such as the top 20%–60% of scores. To achieve partial fairness, we propose an in-processing method by formulating the model training problem as constrained optimization with difference-of-convex constraints, which can be solved by an inexact difference-of-convex algorithm (IDCA). We provide the complexity analysis of IDCA for finding a nearly KKT point. Through numerical experiments on real-world datasets, we demonstrate that our framework achieves high predictive performance while enforcing partial fairness where it matters most.

Speakers
TY

Tianbao Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GL

Gang Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jasper Liao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
QL

Qihang Lin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5M: Recent progresses in derivative-free optimization II
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Recent progresses in derivative-free optimization II
Chair: Jeffrey Larson
Cluster: Derivative-free Optimization

Talk 1: Some Available Derivative (SAD) Optimization
Speaker: Jeffrey Larson
Abstract: Many practical optimization problems involve objective functions where gradient information is unavailable or expensive to compute for certain variables but is readily available for other variables. This talk presents an optimization method that effectively incorporates available partial gradient information with state-of-the-art derivative-free optimization methods. We introduce a new algorithm tailored to this setting, demonstrate its effectiveness through numerical experiments, and discuss its application to optimizing fusion stellarator designs, where coil parameter gradients are accessible, but plasma parameter gradients are not.

Talk 2: Barycenter of weight coefficient region of least norm updating quadratic models with vanishing trust-region radius
Speaker: Pengcheng Xie
Abstract: Derivative-free optimization problems, where objective function derivatives are unavailable, can be addressed using local quadratic models within a trust-region algorithm. We propose a model updating approach when high accuracy is demanded, such as when the trust-region radius vanishes. Our approach uses the barycenter of a particular coefficient region and is shown to be advantageous in numerical results.

Speakers
JL

Jeffrey Larson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Pengcheng Xie

Pengcheng Xie

Postdoctoral Scholar, Lawrence Berkeley National Laboratory
Pengcheng Xie works for Lawrence Berkeley National Laboratory in the United States as a postdoctoral scholar working with Dr. Stefan M. Wild, specializing in computational mathematics, mathematical optimization, machine learning, and numerical analysis. He holds a PhD from the Chinese... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5N: Optimization on Riemannian manifolds and stratified sets (1/2)
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Optimization on Riemannian manifolds and stratified sets (1/2)
Chair: Guillaume Olikier
Cluster: Optimization on Manifolds

Talk 1: First-order optimization on stratified sets
Speaker: Guillaume Olikier
Abstract: This talk considers the problem of minimizing a differentiable function with locally Lipschitz continuous gradient on a closed subset of a Euclidean vector space that can be partitioned into finitely many smooth submanifolds. The partition is called a stratification and the submanifolds are called the strata. Under suitable assumptions on the stratification, first-order methods are proposed that generate a sequence in the set whose accumulation points are guaranteed to be Bouligand stationary. These methods combine retracted line searches along descent directions selected in the Bouligand tangent cone and projections onto the strata. Examples of a set satisfying the assumptions include the algebraic variety of real matrices of upper-bounded rank and its intersection with the cone of symmetric positive-semidefinite matrices. On these sets, Bouligand stationarity is the strongest necessary condition for local optimality.

Talk 2: The Effect of Smooth Parametrizations on Nonconvex Optimization Landscapes
Speaker: Joe Kileel
Abstract: Given a constrained optimization problem, we often tackle it by choosing a parameterization of the constraint set and then optimizing over the parameters. This lets us to approach optimization problems over manifolds or stratified sets through problems over simpler sets, such as Euclidean space without constraints. For example, if we need to optimize a real-valued cost over bounded-rank matrices, then we can parameterize the domain using a low-rank factorization (e.g., the SVD) and then optimize over the factors which are unconstrained. Alternatively, in deep learning when optimizing over the function space represented by a neural network, we have parameterized the space by the weights and biases in the network. In these situations, a natural question is: does the choice of parameterization affect the nonconvex optimization landscape? And: are some parameterizations “better” than others? In this talk, I'll present a geometric framework to formalize such questions and new analysis tools to help answer them. The theory will be applied to several examples, including the aforementioned ones as well as optimization problems from tensor decomposition and semidefinite programming. Based on joint works with Eitan Levin (Caltech), Nicolas Boumal (EPFL), and Chris Criscitiello (EPFL).

Talk 3: Geodesic convexity of polar decomposition
Speaker: Foivos Alimisis
Abstract: In this talk, we will analyze a hidden convexity-like structure for the polar decomposition problem in the orthogonal group. This turns out to be similar to convexity-like structures that have been discovered for the symmetric eigenvalue problem. We will talk about the theoretical, but as well practical implications of this structure in polar decomposition problems under uncertainty.

Speakers
GO

Guillaume Olikier

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JK

Joe Kileel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FA

Foivos Alimisis

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5O: Global Optimization Theory and Algorithms
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Global Optimization Theory and Algorithms
Chair: Sergiy Butenko
Cluster: Global Optimization

Talk 1: Solving bilevel polynomial optimization by disjunctive decomposition
Speaker: Suhan Zhong
Abstract: We consider bilevel polynomial optimization problems whose lower level constraints are linear on lower level variables. We show that such a bilevel program can be reformulated as a disjunctive program using partial Lagrange multiplier expressions. Each branch problem of this disjunctive program can be efficiently solved by polynomial optimization techniques. We give necessary and sufficient conditions for solutions of branch problems to be global optimizer of original bilevel optimization, and sufficient conditions for the local optimality.

Talk 2: Adaptive Bilevel Knapsack Interdiciton 
Speaker: Jourdain Lamperski
Abstract: We consider adaptive bilevel knapsack interdiction. A leader aims to interdict items that a follower aims to fill their knapsack with. The leader does not have full information, as captured by a finite number of possible realizations of the follower. The leader can adapt to uncertainty by executing one of a budgeted number of precomputed interdiction policies.  We propose exact and approximate solution methods. In particular, we present theoretical performance guarantees for the approximate solution methods and evalutate their empirical performance against the exact methods through computational experiments. 

Talk 3: Continuous Approaches to Cluster-Detection Problems in Networks
Speaker: Sergiy Butenko
Abstract: We discuss continuous formulations for several important cluster-detection problems in networks. More specifically, the problems of interested are formulated as quadratic, cubic, or higher-degree polynomial optimization problems subject to linear constraints. The proposed formulations are used to develop analytical bounds as well as effective algorithms for some of the problems. Moreover, a novel hierarchy of nonconvex continuous reformulations of optimization problems on networks is discussed.

Speakers
SZ

Suhan Zhong

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jourdain Lamperski

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SB

Sergiy Butenko

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5P: Feasible and infeasible methods for optimization on manifolds II
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Feasible and infeasible methods for optimization on manifolds II
Chair: Bin Gao
Cluster: Optimization on Manifolds

Talk 1: A decentralized proximal gradient tracking algorithm for composite optimization on Riemannian manifolds
Speaker: Lei Wang
Abstract: This talk focuses on minimizing a smooth function combined with a nonsmooth regularization term on a compact Riemannian submanifold embedded in the Euclidean space under a decentralized setting. Typically, there are two types of approaches at present for tackling such composite optimization problems. The first, subgradient-based approaches, rely on subgradient information of the objective function to update variables, achieving an iteration complexity of $O(\epsilon^{-4}\log^2(\epsilon^{-2}))$. The second, smoothing approaches, involve constructing a smooth approximation of the nonsmooth regularization term, resulting in an iteration complexity of $O(\epsilon^{-4})$. This paper proposes a proximal gradient type algorithm that fully exploits the composite structure. The global convergence to a stationary point is established with a significantly improved iteration complexity of $O(\epsilon^{-2})$. To validate the effectiveness and efficiency of our proposed method, we present numerical results from real-world applications, showcasing its superior performance compared to existing approaches.

Talk 2: A low-rank augmented Lagrangian method for SDP-RLT relaxations of mixed-binary quadratic programs
Speaker: Di Hou
Abstract: The mixed-binary quadratic program (MBQP) with both equality and inequality constraints is a well-known NP-hard problem that arises in various applications. In this work, we focus on two relaxations: the doubly nonnegative (DNN) relaxation and the SDP-RLT relaxation, which combines the Shor relaxation with a partial first-order reformulation-linearization technique (RLT). We demonstrate the equivalence of these two relaxations by introducing slack variables. Furthermore, we extend RNNAL—a globally convergent Riemannian augmented Lagrangian method (ALM) originally developed for solving DNN relaxations—to handle SDP-RLT relaxations. RNNAL penalizes the inequality constraints while keeping the equality constraints in the ALM subproblems. By applying a low-rank decomposition in each ALM subproblem, the feasible region is transformed into an algebraic variety with advantageous geometric properties for us to apply a Riemannian gradient descent method. Our algorithm can efficiently solve general semidefinite programming (SDP) problems, including relaxations for quadratically constrained quadratic programming (QCQP). Extensive numerical experiments confirm the efficiency of the proposed method.

Talk 3: An improved unconstrained approach for bilevel optimization
Speaker: Nachuan Xiao
Abstract: In this talk, we focus on the nonconvex-strongly-convex bilevel optimization problem (BLO). In this BLO, the objective function of the upper-level problem is nonconvex and possibly nonsmooth, and the lower-level problem is smooth and strongly convex with respect to the underlying variable. We show that the feasible region of BLO is a Riemannian manifold. Then we transform BLO to its corresponding unconstrained constraint dissolving problem (CDB), whose objective function is explicitly formulated from the objective functions in BLO. We prove that BLO is equivalent to the unconstrained optimization problem CDB. Therefore, various efficient unconstrained approaches, together with their theoretical results, can be directly applied to BLO through CDB. We propose a unified framework for developing subgradient-based methods for CDB. Remarkably, we show that several existing efficient algorithms can fit the unified framework and be interpreted as descent algorithms for CDB. These examples further demonstrate the great potential of our proposed approach.

Speakers
BG

Bin Gao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LW

Lei Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DH

Di Hou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
NX

Nachuan Xiao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5Q: Moment-SOS Hierarchy: From Theory to Computation in the Real World (II)
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Moment-SOS Hierarchy: From Theory to Computation in the Real World (II)
Chair: Jie Wang
Cluster: Conic and Semidefinite Optimization

Talk 1: Bregman primal--dual first-order method and application to sparse semidefinite programming
Speaker: Xin Jiang
Abstract: We discuss the centering problem in large-scale semidefinite programming with sparse coefficient matrices. The logarithmic barrier function for the cone of positive semidefinite completable sparse matrices is used as the distance-generating kernel. For this distance, the complexity of evaluating the Bregman proximal operator is shown to be roughly proportional to the cost of a sparse Cholesky factorization. This is much cheaper than the standard proximal operator with Euclidean distances, which requires an eigenvalue decomposition. Then primal-dual proximal algorithm with Bregman distances are applied to solve large-scale sparse semidefinite programs efficiently.

Talk 2: Moment-SOS hierarchies for variational problems and PDE control
Speaker: Giovanni Fantuzzi
Abstract: Moment-SOS hierarchies are an established tool to compute converging sequences of lower bounds on the global minimum of finite-dimensional polynomial optimization problems. In this talk, I will show that they can be combined with finite-element discretizations to give a "discretize-then-relax" framework for solving two classes of infinite-dimensional problems: minimization problems for nonconvex integral functionals over functions from a Sobolev space, and some optimal control problems for nonlinear partial differential equations. For each class of problems, I will discuss conditions ensuring that this "discretize-then-relax" framework produces converging approximations (sometimes, bounds) to the global minimum and to the corresponding optimizer. Gaps between theory and practice will be illustrated by means of examples.

Talk 3: Inexact Augmented Lagrangian Methods for Semidefinite Optimization: Quadratic Growth and Linear Convergence
Speaker: Feng-Yi Liao
Abstract: Augmented Lagrangian Methods (ALMs) are widely employed in solving constrained optimizations, and some efficient solvers are developed based on this framework. Under the quadratic growth assumption, it is known that the dual iterates and the Karush–Kuhn–Tucker (KKT) residuals of ALMs applied to semidefinite programs (SDPs) converge linearly. In contrast, the convergence rate of the primal iterates has remained elusive. In this talk, we resolve this challenge by establishing new quadratic growth and error-bound properties for primal and dual SDPs under the strict complementarity condition. Our main results reveal that both primal and dual iterates of the ALMs converge linearly contingent solely upon the assumption of strict complementarity and a bounded solution set. This finding provides a positive answer to an open question regarding the asymptotically linear convergence of the primal iterates of ALMs applied to semidefinite optimization. 

Speakers
JW

Jie Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XJ

Xin Jiang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GF

Giovanni Fantuzzi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5R: Variational Analysis: Theory and Applications III
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Variational Analysis: Theory and Applications III
Chair: Walaa Moursi
Cluster: Nonsmooth Optimization

Talk 1: TBA
Speaker: Sedi Bartz
Abstract: TBA

Talk 2: TBA
Speaker: Taeho Yoon
Abstract: TBA

Talk 3: TBA
Speaker: Scott Lindstrom
Abstract: TBA

Speakers
WM

Walaa Moursi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SB

Sedi Bartz

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for TaeHo Yoon

TaeHo Yoon

Postdoctoral fellow, Johns Hopkins University
Hi, I'm a Rufus Isaacs Postdoctoral Fellow at Johns Hopkins University, Department of Applied Mathematics & Statistics.I work on optimization with general interest in topics including acceleration, minimax optimization, variational inequality, fixed point problems, learning in multiplayer... Read More →
SL

Scott Lindstrom

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5S: Semidefinite Relaxations in Inference Problems
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Semidefinite Relaxations in Inference Problems
Chair: Yong Sheng Soh
Cluster: Conic and Semidefinite Optimization

Talk 1: On the exactness of SDP relaxation for quadratic assignment problem
Speaker: Shuyang Ling
Abstract: Quadratic assignment problem (QAP) is a fundamental problem in combinatorial optimization and finds numerous applications in operation research, computer vision, and pattern recognition. However, it is a very well-known NP-hard problem to find the global minimizer to the QAP. In this work, we study the semidefinite relaxation (SDR) of the QAP and investigate when the SDR recovers the global minimizer. In particular, we consider the two input matrices satisfy a simple signal-plus-noise model, and show that when the noise is sufficiently smaller than the signal, then the SDR is exact, i.e., it recovers the global minimizer to the QAP. It is worth noting that this sufficient condition is purely algebraic and does not depend on any statistical assumption of the input data. We apply our bound to several statistical models such as correlated Gaussian Wigner model. Despite the sub-optimality in theory under those models, empirical studies show the remarkable performance of the SDR. Our work could be the first step towards a deeper understanding of the SDR exactness for the QAP.

Talk 2: Exactness Conditions for Semidefinite Relaxations of the Quadratic Assignment Problem
Speaker: Junyu Chen
Abstract: The Quadratic Assignment Problem (QAP) is an important discrete optimization instance that encompasses many well-known combinatorial optimization problems, and has applications in a wide range of areas such as logistics and computer vision. The QAP, unfortunately, is NP-hard to solve. To address this difficulty, a number of semidefinite relaxations of the QAP have been developed. These techniques are known to be powerful in that they compute globally optimal solutions in many instances, and are often deployed as sub-routines within enumerative procedures for solving QAPs. In this paper, we investigate the strength of these semidefinite relaxations. Our main result is a deterministic set of conditions on the input matrices -- specified via linear inequalities -- under which these semidefinite relaxations are exact. Our result is simple to state, in the hope that it serves as a foundation for subsequent studies on semidefinite relaxations of the QAP as well as other related combinatorial optimization problems. As a corollary, we show that the semidefinite relaxation is exact under a perturbation model whereby the input matrices differ by a suitably small perturbation term. One technical difficulty we encounter is that the set of dual feasible solutions is not closed. To circumvent these difficulties, our analysis constructs a sequence of dual feasible solutions whose objective value approaches the primal objective, but never attains it.

Talk 3: Inference of planted subgraphs in random graphs
Speaker: Cheng Mao
Abstract: Suppose that we are given a graph G obtained from randomly planting a template graph in an Erdős–Rényi graph. Can we detect the presence and recover the location of the planted subgraph in G? This problem is well understood in the case where the template graph is a clique or a dense subgraph but has received less attention otherwise. We consider several settings where the planted subgraph is the power of a Hamiltonian cycle or a random geometric graph which arises in multiple applications. The difficulty of the detention or recovery problem primarily lies in the combinatorial nature of the associated optimization problem. To tackle the problem, we study computationally efficient methods such that subgraph counting and spectral methods which leverage the higher triangle density of the planted subgraph.

Speakers
YS

Yong Sheng Soh

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SL

Shuyang Ling

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JC

Junyu Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CM

Cheng Mao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5T: Optimization for Approximation, Estimation, and Control
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Optimization for Approximation, Estimation, and Control
Chair: Martin Andersen
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Fast and Certifiable Nonconvex Optimal Control with Sparse Moment-SOS Relaxations
Speaker: Heng Yang
Abstract: Direct methods for optimal control, also known as trajectory optimization, is a workhorse for optimization-based control in robotics and beyond. Nonlinear programming with engineered initializations has been the de-facto approach for trajectory optimization, which however, can suffer from undesired local optimality. In this talk, I will first show that, using the machinery of sparse moment and sums-of-squares (SOS) relaxations, many nonconvex trajectory optimization problems can be solved to certifiable global optimality. That is, globally optimal solutions of the original nonconvex problems can be computed by solving convex semidefinite programs (SDPs) together with optimality certificates. I will then present a specialized SDP solver implemented in CUDA (C++) and runs in GPUs that exploits the structures of the problems to solve the convex SDPs at a scale far beyond existing solvers.

Talk 2: Optimal diagonal preconditioner and how to find it
Speaker: Zhaonan Qu
Abstract: Preconditioning has long been a staple technique in optimization, often applied to reduce the condition number of a matrix and speed up the convergence of algorithms. Although there are many popular preconditioning techniques in practice, most lack guarantees on reductions in condition number. Moreover, the degree to which we can improve over existing heuristic preconditioners remains an important practical question. In this talk, we discuss the problem of optimal diagonal preconditioning that achieves maximal reduction in the condition number of any full-rank matrix by scaling its rows and/or columns. We reformulate the problem as a quasi-convex optimization problem and design interior point method to solve it with O(log(1/ϵ)) iteration complexity. Next, we specialize to one-sided optimal diagonal preconditioning problems, and demonstrate that they can be formulated as standard dual SDP problems. Based on the SDP formulation, several computational techniques are applicable, and can greatly accelerate finding a good diagonal preconditioner with theoretical guarantees. Our experiments suggest that optimal diagonal preconditioners can significantly improve upon existing heuristic-based diagonal preconditioners at reducing condition numbers and speeding up iterative methods.

Talk 3: Power System State Estimation by Phase Synchronization and Eigenvectors
Speaker: Iven Guzel
Abstract: To estimate accurate voltage phasors from inaccurate voltage magnitude and complex power measurements, the standard approach is to iteratively refine a good initial guess using the Gauss-Newton method. But the nonconvexity of the estimation makes the Gauss-Newton method sensitive to its initial guess, so human intervention is needed to detect convergence to plausible but ultimately spurious estimates. This paper makes a novel connection between the angle estimation subproblem and phase synchronization to yield two key benefits: (1) an exceptionally high quality initial guess over the angles, known as a spectral initialization; (2) a correctness guarantee for the estimated angles, known as a global optimality certificate. These are formulated as sparse eigenvalue-eigenvector problems, which we efficiently compute in time comparable to a few Gauss-Newton iterations. Our experiments on the complete set of Polish, PEGASE, and RTE models show, where voltage magnitudes are already reasonably accurate, that spectral initialization provides an almost-perfect single-shot estimation of angles from moderately noisy bus power measurements (i.e. pairs of PQ measurements), whose correctness becomes guaranteed after a single Gauss--Newton iteration. For less accurate voltage magnitudes, the performance of the method degrades gracefully; even with moderate voltage magnitude errors, the estimated voltage angles remain surprisingly accurate.

Speakers
avatar for Heng Yang

Heng Yang

Assistant Professor, Harvard University
Assistant Professor at Harvard University working on polynomial optimization and semidefinite programming.
ZQ

Zhaonan Qu

Postdoc, Columbia University
My research interests are at the intersection of econometrics, operations research, and machine learning, with a focus on causal inference, optimization, choice modeling, and networks. I leverage novel connections between these topics to investigate foundational and policy-relevant... Read More →
IG

Iven Guzel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5U: Quantum Computing and Optimization
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Quantum Computing and Optimization
Chair: Reuben Tate
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Advances in Warm-Started QAOA
Speaker: Reuben Tate
Abstract: Over a decade ago, the Quantum Approximate Optimization Algorithm (QAOA) was developed by Farhi et al. for solving certain problems in combinatorial optimization; the algorithm is a variational algorithm that involves the tuning of continuous circuit parameters. Five years ago, Tate et al. and Egger et al. independently developed a warm-start variant that involves using classically-obtained information to construct a warm-started initial quantum state. In this talk, we will go over recent developments in regards to Warm-Started QAOA from both a theoretical, empirical, and algorithmic-design perspective.

Talk 2: Investigating Alternative Metrics of Performance for QAOA
Speaker: Sean Feeney
Abstract: In modern optimization tasks, combinatorial optimization problems (COPs) are paramount in fields such as logistics, power systems, and circuit design. Quantum Approximate Optimization Algorithm (QAOA) has emerged as a promising quantum-classical variational approach for tackling these challenges, yet it often offers limited worst-case approximation guarantees. This work focuses on a recently introduced variant, Warm-Start QAOA, which leverages classical solutions as biased initial states in hopes of boosting the odds of finding high-quality solutions. We conduct an extensive numerical study of Warm-Start QAOA on 3-regular Max-Cut instances. Compared to theoretical lower bounds established by Tate et al. for single-round QAOA, our empirical results consistently indicate better performance across a range of tilt angles. However, this apparent advantage does not extend beyond the classical warm-start solution itself when the QAOA parameters are optimized solely by maximizing the expected objective value. This outcome highlights a pressing need to look beyond standard expectation-value metrics, as they may not capture the subtle elements required for surpassing strong classical baselines in practical settings. To address this gap, we introduce the Better Solution Probability (BSP) metric, which shifts the optimization target from maximizing expectation value performance to maximizing the probability of exceeding a given classical warm-start cut. Our simulations show that BSP-optimized Warm-Start QAOA frequently locates better solutions at non-trivial tilt angles, positions that outperform both the purely classical approach at zero tilt and standard QAOA at 90°, and does so with a non-vanishing probability. Notably, the BSP objective remains feasible to optimize in real-world conditions because it does not rely on a priori knowledge of the optimal solution. By exploring objective functions beyond expectation value, this study offers new insights into how hybrid quantum-classical methods can be enhanced for complex optimization tasks, paving the way for more robust algorithm design and a clearer path toward quantum advantage.

Talk 3: Quantum Hamiltonian Descent Algorithms for Nonlinear Optimization
Speaker: Yufan Zheng
Abstract: Nonlinear optimization is a vibrant field of research with wide-ranging applications in engineering and science. However, classical algorithms often struggle with local minima, limiting their effectiveness in tackling nonconvex problems. In this talk, we explore how quantum dynamics can be exploited to develop novel quantum optimization algorithms. Specifically, we introduce Quantum Hamiltonian Descent (QHD), a quantum optimization algorithm inspired by the connection between accelerated gradient descent and Hamiltonian mechanics. We will discuss the theoretical properties of QHD, including its global convergence in nonlinear and nonconvex optimization, along with strong empirical and theoretical evidence of its advantage over classical algorithms. Additionally, we will present an open-source implementation of the algorithm (named QHDOPT) and demonstrate its real-world applications.

Speakers
RT

Reuben Tate

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SF

Sean Feeney

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5V: Newton-ish and Higher-Order Methods
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Newton-ish and Higher-Order Methods
Chair: Nick Tsipinakis
Cluster: nan

Talk 1: Multilevel Regularized Newton Methods with Fast Convergence Rates
Speaker: Nick Tsipinakis
Abstract: We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton sub-problem using second-order information from a coarse (low dimensional) sub-problem. The new regularized multilevel methods provably converge from any initialization point and enjoy faster convergence rates than Gradient Descent. In particular, for arbitrary functions with Lipschitz continuous Hessians, we show that their convergence rate interpolates between the rate of Gradient Descent and that of the cubic Newton method. If, additionally, the objective function is assumed to be convex, then the proposed method converges with the fast $\mathcal{O}(k^{-2})$ rate. Hence, since the updates are generated using a coarse model in low dimensions, the theoretical results of this paper significantly speed-up the convergence of Newton-type or preconditioned gradient methods in practical applications. Preliminary numerical results suggest that the proposed multilevel algorithms are significantly faster than current state-of-the-art methods. [1] K. Mishchenko, Regularized Newton method with global convergence, SIAM Journal on Optimization, 33 (2023), pp. 1440–1462. [2] N. Doikov and Y. Nesterov, Gradient regularization of Newton method with Bregman dis-tances, Mathematical programming, 204 (2024), pp. 1–25. [3] N. Tsipinakis and P. Parpas, A multilevel method for self-concordant minimization, Journal of Optimization Theory and Applications, (2024), pp. 1–51.

Talk 2: First-ish Order Methods: Hessian-aware Scalings of Gradient Descent
Speaker: Oscar Smee
Abstract: Gradient descent is the primary workhorse for optimizing large-scale problems in machine learning. However, its performance is highly sensitive to the choice of the learning rate. A key limitation of gradient descent is its lack of natural scaling, which often necessitates expensive line searches or heuristic tuning to determine an appropriate step size. In this paper, we address this limitation by incorporating Hessian information to scale the gradient direction. By accounting for the curvature of the function along the gradient, our adaptive, Hessian-aware scaling method ensures a local unit step size guarantee, even in nonconvex settings. Near a local minimum that satisfies the second-order sufficient conditions, our approach achieves linear convergence with a unit step size. We show that our method converges globally under a significantly weaker version of the standard Lipschitz gradient smoothness assumption. Even when Hessian information is inexact, the local unit step size guarantee and global convergence properties remain valid under mild conditions. Finally, we validate our theoretical results empirically on a range of convex and nonconvex machine learning tasks, showcasing the effectiveness of the approach. Preprint: https://arxiv.org/abs/2502.03701

Speakers
NT

Nick Tsipinakis

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
OS

Oscar Smee

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5W: Systems of Quadratic Equations/Inequalities
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Systems of Quadratic Equations/Inequalities
Chair: Ruey-Lin Sheu
Cluster: nan

Talk 1: On the implementation issue of the non-homogeneous strict Finsler and the non-homogeneous Calabi theorems
Speaker: Min-Chi Wang
Abstract: Let f(x) and g(x) be two real quadratic functions defined on ℝⁿ. The existence of solutions to the system of (in)equality constraints [f(x) = 0, g(x) = 0] and [f(x) = 0, g(x) ≤ 0] has rarely been studied in the literature. Recently, new results have emerged, called the non-homogeneous strict Finsler Lemma and the non-homogeneous Calabi Theorem, which provide necessary and sufficient conditions for the solvability of the above two systems of (in)equality constraints. This talk aims to simplify the conditions to facilitate easier implementation. Numerical results show that our proposed method is efficient in computation.

Talk 2: A QP1QC approach for deciding whether or not two quadratic surfaces intersect
Speaker: Ting-Tsen Lin
Abstract: Given two $n$-variate quadratic functions $f(x)=x^TAx+2a^Tx+a_0, g(x)=x^TBx+2b^Tx+b_0$, we are interested in knowing whether or not the two hypersurfaces $\{x\in \mathbb{R}^n: f(x)=0\}$ and $\{x\in \mathbb{R}^n: g(x)=0\}$ intersect with each other. There are two aspects to look at this problem. In one respect, the famous Finsler-Calabi theorem (1936-1964) asserts that, if $n\ge3$ and $f,g$ are quadratic forms, $f=g=0$ has no common solution other than the trivial one $x=0$ if and only if there exists a positive definite matrix pencil $\alpha A+\beta B\succ0.$ The result is in general not true for non-homogeneous quadratic functions. On the other hand, Levin (c. late 1970) tried to directly solve the intersection curve of $\{x\in \mathbb{R}^n: f(x)=0\}$ and $\{x\in \mathbb{R}^n: g(x)=0\},$ but it turned out to be way too ambitious. In this paper, we show that, by incorporating with the information about the unboundedness and the un-attainability of several (at most 4) quadratic programming problems with one single quadratic constraint (QP1QC), the answer as to whether or not $f(x)=x^TAx+2a^Tx+a_0=0$ and $g(x)=x^TBx+2b^Tx+b_0=0$ intersect can be successfully determined. References: E. Calabi, Linear systems of real quadratic forms, Proceedings of the American Mathematical Society, 15 (1964), pp. 844–846. P. Finsler, Über das Vorkommen definiter und semidefiniter Formen in Scharen quadratischer Formen, Commentarii Mathematici Helvetici, 9 (1936), pp. 188–192. J. Z. Levin, A parametric algorithm for drawing pictures of solid objects composed of quadric surfaces, Communications of the ACM, 19 (1976), pp. 555–563. J. Z. Levin, Mathematical models for determining the intersections of quadric surfaces, Computer Graphics and Image Processing, 11 (1979), pp. 73–87.

Talk 3: Last two pieces of puzzle for un-solvability of a system of two quadratic (in)equalities
Speaker: Ruey-Lin Sheu
Abstract: Given two quadratic functions f(x) = x^{T}Ax+2a^{T}x+a_0 and g(x) = x^{T}Bx+ 2b^{T }x+b_0, each associated with either the strict inequality (< 0); non-strict inequality (≤ 0); or the equality (= 0), it is a fundamental question to ask whether or not the joint system {x ∈ R^n: f(x) ⋆ 0} and {x ∈ R^n: g(x)#0}, where ⋆ and # can be any of {
Speakers
MW

Min-Chi Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TL

Ting-Tsen Lin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RS

Ruey-Lin Sheu

Professor, National Cheng Kung University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5X: Modeling, Solvers, and Quantum
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Session: Modeling, Solvers, and Quantum
Chair: Giacomo Nannicini
Cluster: nan

Talk 1: Making Some Smooth Problems with Derivative Jumps Easier to Express
Speaker: David Gay
Abstract: Some optimization problems involve an objective or constraints with derivative jumps. The jumps may be due to absolute values or the minimum or maximum of two or more expressions or to the piecewise linearization of a nonlinear function. There are multiple ways to deal with such situations. For example, Griewank and Walther propose using regularization supplied by overloading the relevant expressions. Of interest in this talk is using binary variables to decide which pieces are currently relevant. In the AMPL modeling language, this is implicit when one uses the piecewise-linear notation. This talk will present examples and discuss plans to automatically convert abs(...), min(...), and max}(...) to piecewise-linear forms, enabling efficiently solving by various solvers.

Talk 2: The quantum central path method
Speaker: Giacomo Nannicini
Abstract: Abstract: We propose a new quantum algorithm for solving linear optimization problems by quantum-mechanical simulation of the central path. While interior point methods follow the central path with an iterative algorithm that works with successive linearizations of the perturbed KKT conditions, we perform a single simulation working directly with the nonlinear complementarity equations. Our approach is inspired by the Newton barrier flow studied in the 80s by several authors, and goes beyond those studies by giving a full description of a quantum algorithm that simulates the corresponding dynamical system. The algorithm has a worst case complexity with favorable scaling in the problem dimension compared to state-of-the-art methods, but worse scaling in the precision and the size of the solution. We hope that with further advancements, this method could pave the way for an end-to-end quantum speedup for linear optimization.

Talk 3: Modeling and algorithmic framework for complex optimization problems and equilibrium problems
Speaker: Olivier Huber
Abstract: Algebraic modeling languages (AML) have shaped the problem structure considered in numerical optimization, and indirectly have an impact on the instances coming from applications and the ones considered by solvers. However, problems not fitting the classical AML format are abundantly found in applications. For instance, nonsmoothness commonly arises in (multistage) stochastic programming problems with coherent risk measures. Bilevel or multilevel models feature multiple optimization problems, and so do Nash equilibrium problems. A given application can feature a combination of the above challenge. Consider the decentralized electricity market, where the weather affect the production level of some producers. This leads to a Nash equilibrium problem with multistage risk-averse agents. To capture these challenging but structured models, we propose a modeling framework based on a directed acyclic graph (DAG). The nodes are of two type: the first one subsumes classical optimization problems and variational inequalities, while the second one indicates a Nash behavior among its children nodes. The directed arcs capture the interactions between optimization problems. This modeling approach is implemented via annotation and extends an AML modeling power. Model transformations are defined over the DAG structure to convert part or all of the model into a form amenable to computations by existing solvers. Many instances captured by this modeling paradigm fall outside of the scope for robust solvers, an algorithmic framework to solve these instances, for instance by decomposition or approximation, is presented. An implementation of these ideas is present in ReSHOP, a reformulation solver for hierarchical optimization problems.

Speakers
DG

David Gay

Name: Dr. David M GayTitle: chief scientistAffiliation: AMPL OptimizationBio:Math major at the University of Michigan with senior year (1970-71) at the Albert-Ludwigs-Universitaet, Freiburg im Breisgau, Germany. Grad student at Cornell 1971-74, PhD in Computer Science (1975). Asst... Read More →
OH

Olivier Huber

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 5Y
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Tuesday July 22, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

2:30pm PDT

Coffee & Snack Break (Provided)
Tuesday July 22, 2025 2:30pm - 3:00pm PDT
Tuesday July 22, 2025 2:30pm - 3:00pm PDT
TBA

3:00pm PDT

Parallel Semi-Plenary Talk 2A
Tuesday July 22, 2025 3:00pm - 4:00pm PDT
Speakers
JD

John Duchi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

3:00pm PDT

Parallel Semi-Plenary Talk 2B
Tuesday July 22, 2025 3:00pm - 4:00pm PDT
Speakers
AW

Angelika Wiegele

Angelika Wiegele is Professor at the Mathematics Department at Alpen-Adria-Universität Klagenfurt and is currently a member of the Global Faculty of the University of Cologne. She received her Ph.D. in Mathematics at the Alpen-Adria-Universität Klagenfurt in 2006 and was a researcher... Read More →
Tuesday July 22, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:00pm PDT

Break
Tuesday July 22, 2025 4:00pm - 4:15pm PDT
Tuesday July 22, 2025 4:00pm - 4:15pm PDT
TBA

4:15pm PDT

Parallel Sessions 6A: Adaptive Stochastic Gradient Methods
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Adaptive Stochastic Gradient Methods
Chair: Lin Xiao
Cluster: Nonlinear Optimization

Talk 1: The Road Less Scheduled
Speaker: Aaron Defazio
Abstract: Schedule-Free learning algorithms allow for the training of models in an any-time fashion, without compromising on speed, memory or final test metrics. I will dive into the details of how Schedule-Free learning works and show how it provides further quality-of-life improvements to practitioners, and provide details of our winning entry to the AlgoPerf algorithmic efficiency optimization challenge that used Schedule-Free AdamW.

Talk 2: Analyzing AdaGrad Under Anisotropic Smoothness Assumptions
Speaker: Yuxing Liu
Abstract: Adaptive gradient methods have demonstrated remarkable success for training large-scale deep neural networks. However, the theoretical understanding of these methods, particularly in the large batch size regime (which is commonly used in practice), remains limited. In this talk, we aim to address this gap by introducing a generalized anisotropic smoothness assumption that better reflects the behavior of modern neural network training. Our theoretical analysis reveals that AdaGrad achieves provably faster convergence compared to standard gradient methods, even when large batch sizes are employed. These results provide valuable theoretical insights into the practical efficacy of adaptive gradient methods.

Talk 3: A Novel Approach to Loss Landscape Characterization without Over-Parametrization
Speaker: Antonio Orvieto
Abstract: Modern machine learning heavily depends on the effectiveness of optimization techniques. While deep learning models have achieved remarkable empirical results in training, their theoretical underpinnings remain somewhat elusive. Ensuring the convergence of optimization methods requires imposing specific structures on the objective function, which often do not hold in practice. One prominent example is the widely recognized Polyak-Lojasiewicz (PL) inequality, which has garnered considerable attention in recent years. However, validating such assumptions for deep neural networks entails substantial and often impractical levels of over-parametrization. In order to address this limitation, we propose a novel class of functions that can characterize the loss landscape of modern deep models without requiring extensive over-parametrization and can also include saddle points. Crucially, we prove that gradient-based optimizers possess theoretical guarantees of convergence under this assumption. Finally, we validate the soundness of our assumption through both theoretical analysis and empirical experimentation across a diverse range of deep learning models.

Speakers
LX

Lin Xiao

Lin Xiao is a Research Scientist at Facebook AI Research (FAIR) in Seattle, Washington. He received BE from Beijing University of Aeronautics and Astronautics (Beihang University) and PhD from Stanford University, and was a postdoctoral fellow in the Center for the Mathematics of... Read More →
AD

Aaron Defazio

Research Scientist, Meta Platforms, Inc.
Aaron Defazio is a Research Scientist at Meta on the Fundamental AI Research Team, specializing in the field of optimization algorithms for machine learning. Aaron holds a PhD in Computer Science from ANU (Australian National University) and has a rich background in research, having... Read More →
YL

Yuxing Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AO

Antonio Orvieto

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6B: Machine Learning Algorithms
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Machine Learning Algorithms
Chair: Mark Schmidt
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Leveraging Variable Sparsity to Refine Pareto Stationarity in Multi-Objective Optimization
Speaker: Yaoliang Yu
Abstract: Gradient-based multi-objective optimization (MOO) is essential in modern machine learning, with applications in e.g., multi-task learning, federated learning, algorithmic fairness and reinforcement learning. In this work, we first reveal some limitations of Pareto stationarity, a widely accepted first-order condition for Pareto optimality, in the presence of sparse function-variable structures. Next, to account for such sparsity, we propose a novel solution concept termed Refined Pareto Stationarity (RPS), which we prove is always sandwiched between Pareto optimality and Pareto stationarity. We give an efficient partitioning algorithm to automatically mine the function-variable dependency and substantially trim non-optimal Pareto stationary solutions. Then, we show that gradient-based descent algorithms in MOO can be enhanced with our refined partitioning. In particular, we propose Multiple Gradient Descent Algorithm with Refined Partition (RP-MGDA) as an example method that converges to RPS, while still enjoying a similar per-step complexity and convergence rate. Lastly, we validate our approach through experiments on both synthetic examples and realistic application scenarios where distinct function-variable dependency structures appear. Our results highlight the importance of exploiting function-variable structure in gradient-based MOO and provide a seamless enhancement to existing approaches.

Talk 2: High-dimensional Optimization with Applications to Compute-Optimal Neural Scaling Laws
Speaker: Courtney Paquette
Abstract: Given the massive scale of modern ML models, we now only get a single shot to train them effectively. This restricts our ability to test multiple architectures and hyper-parameter configurations. Instead, we need to understand how these models scale, allowing us to experiment with smaller problems and then apply those insights to larger-scale models. In this talk, I will present a framework for analyzing scaling laws in stochastic learning algorithms using a power-law random features model, leveraging high-dimensional probability and random matrix theory. I will then use this scaling law to address the compute-optimal question: How should we choose model size and hyper-parameters to achieve the best possible performance in the most compute-efficient manner?

Talk 3: A Robustness Metric for Distribution Shifts
Speaker: John Duchi
Abstract: We revisit the stability of optimizers in statistical estimation and stochastic optimization problems, but instead of providing guarantees on the stability of the minimizers themselves, we investigate what shifts to the underlying data-generating process perturb solutions the most. To do so, we develop some new mathematical tools for stability analyses, with guarantees beyond typical differentiable problems. We also make connections with statistical hypothesis testing and discovery, showing how these new results provide certificates of validity---or potential invalidity---of statistical estimate.

Speakers
MS

Mark Schmidt

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YY

Yaoliang Yu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CP

Courtney Paquette

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JD

John Duchi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6C: AI Meets Optimization (Part 1)
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: AI Meets Optimization (Part 1)
Chair: Wotao Yin
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: TBD
Speaker: Bartolomeo Stellato
Abstract: TBD

Talk 2: Differentiating through Solutions to Optimization Problems in Decision-Focused Learning
Speaker: Howard Heaton
Abstract: Many real-world problems can be framed as optimization problems, for which well-established algorithms exist. However, these problems often involve key parameters that are not directly observed. Instead, we typically have access to data that is correlated with these parameters, though the relationships are complex and difficult to describe explicitly. This challenge motivates the integration of machine learning with optimization: using machine learning to predict the hidden parameters and optimization to solve the resultant problem. This integration is known as decision-focused learning. In this talk, I will introduce decision-focused learning, with a particular focus on differentiating through solutions to optimization problems and recent advances in effectively scaling these computations.

Talk 3: TBD
Speaker: Ferdinando Fioretto
Abstract: TBD

Speakers
BS

Bartolomeo Stellato

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FF

Ferdinando Fioretto

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6D: Special Session in Honor of Suvrajeet Sen: Nonsmooth Methods in Stochastic Programming
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Special Session in Honor of Suvrajeet Sen: Nonsmooth Methods in Stochastic Programming
Chair: Junyi Liu
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Improving Dimension Dependence in Zeroth-Order Schemes via Exponentially Shifted Gaussian Smoothing
Speaker: Uday Shanbhag
Abstract: Smoothing-enabled zeroth-order (ZO) methods for nonsmooth convex stochastic optimization have assumed increasing relevance. A shortcoming of such schemes is the dimension dependence in the complexity guarantees, a concern that impedes truly large-scale implementations. We develop a novel exponentially- shifted Gaussian smoothing (esGS) gradient estimator by leveraging a simple change-of-variable argument. The moment bounds of the (esGS) estimator are characterized by a muted dependence on dimension. When the (esGS) estimator is incorporated within a ZO framework, the resulting iteration complexity bounds are reduced to O(n\epsilon^{-2}) from O(n^2 \epsilon^{-2}), the latter being the best available for the existing two-point estimator with Gaussian smoothing. This is joint work with Mingrui Wang and Prakash Chakraborty.

Talk 2: Decomposition-based algorithm for infinite horizon stochastic programs on finite-state Markov chains
Speaker: Shuotao Diao
Abstract: Infinite horizon stochastic programs on finite-state Markov chains can be expressed as a semi-contractive model. In this work, we provide a unifying view of the conditions ensuring the existence of fixed points of the infinite horizon stochastic programs. We further develop a decomposition-based method to solve the infinite horizon stochastic programs with convergence guarantee.

Talk 3: Adaptive Sampling-based Nonconvex and Nonsmooth approaches for Stochastic Programs with Implicitly Decision-dependent Uncertainty
Speaker: Junyi Liu
Abstract: We consider a class of stochastic programming problems where the implicitly decision-dependent random variable follows a nonparametric regression model with heteroskedastic error. We develop an adaptive sampling-based algorithm that integrates the simulation scheme and statistical estimates to construct sampling-based surrogate functions in a way that the simulation process is guided by the algorithmic procedure. We establish the nonasymptotic convergence analysis in terms of $(\epsilon, \delta)$-nearly stationarity in expectation under variable proximal parameters and batch sizes that leads to superior convergence rate. Furthermore, we show that the proposed adaptive simulation scheme embedded in the sampling-based algorithm leads to better error control of sampling-based surrogate functions and thus enhance the stability and efficiency of the sampling-based algorithm, which are further evidenced by numerical results.

Speakers
US

Uday Shanbhag

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SD

Shuotao Diao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Junyi Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6E: Recent advances in algorithms for large-scale optimization (II)
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Recent advances in algorithms for large-scale optimization (II)
Chair: Xudong Li
Cluster: Computational Software

Talk 1: Alternating minimization for distributionally robust principal component pursuit
Speaker: Xudong Li
Abstract: Recently, the square root principal component pursuit (SRPCP) model has garnered significant research interest. It is shown in the literature that the SRPCP model guarantees robust matrix recovery with a universal, constant penalty parameter. While its statistical advantages are well-studied, the computational aspects remain unexplored. In this talk, we focus on developing efficient algorithms for solving the SRPCP problem. Specifically, we propose a tuning-free alternating minimization (AM) algorithm, where each iteration involves subproblems with semi-closed updating rules. Additionally, we introduce techniques based on the variational formulation of the nuclear norm and BM decomposition to further accelerate the AM method. Extensive numerical experiments confirm the efficiency and robustness of our algorithms.

Talk 2: A new dual semismooth Newton method for polyhedral projections
Speaker: Chao Ding
Abstract: We propose a dual semismooth Newton method for computing the orthogonal projection onto a given polyhedron, a fundamental optimization problem that serves as a critical building block for numerous important applications, e.g., financial risk management, statistics, and machine learning. Classical semismooth Newton methods typically depend on subgradient regularity assumptions for achieving local superlinear or quadratic convergence. Our approach, however, marks a significant breakthrough by demonstrating that it is always possible to identify a point where the existence of a nonsingular generalized Jacobian is guaranteed, regardless of any regularity conditions. Furthermore, we explain this phenomenon and its relationship with the weak strict Robinson constraint qualification (W-SRCQ) from the perspective of variational analysis. Building on this theoretical advancement, we develop an inexact semismooth Newton method with superlinear convergence for solving the polyhedral projection problem.

Talk 3: A Proximal DC Algorithm for Sample Average Approximation of Chance Constrained Programming
Speaker: Rujun Jiang
Abstract: Chance constrained programming (CCP) refers to a type of optimization problem with uncertain constraints that are satisfied with at least a prescribed probability level. In this work, we study the sample average approximation (SAA) method for chance constraints, which is an important approach to CCP in the data-driven setting where only a sample of multiple realizations of the random vector in the constraints is available. The SAA method approximates the underlying distribution with an empirical distribution over the available sample. Assuming that the functions in the chance constraints are all convex, we reformulate the SAA of chance constraints into a difference-of-convex (DC) form. Additionally, by assuming the objective function is also a DC function, we obtain a DC constrained DC program. To solve this reformulation, we propose a proximal DC algorithm and show that the subproblems of the algorithm are suitable for off-the-shelf solvers in some scenarios. Moreover, we not only prove the subsequential and sequential convergence of the proposed algorithm but also derive the iteration complexity for finding an approximate Karush-Kuhn-Tucker point. To support and complement our theoretical development, we show via numerical experiments that our proposed approach is competitive with a host of existing approaches.

Speakers
XL

Xudong Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CD

Chao Ding

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Rujun Jiang

Rujun Jiang

Rujun Jiang is currently an associate professor at School of Data Science, Fudan University. He received his Bachelor degree in Mathematics in 2012 from University of Science and Technology of China. He then obtained his PhD degree in 2016 in The Chinese University of Hong Kong under... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6F: Optimization for improving privacy and alignment for LLMs
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Optimization for improving privacy and alignment for LLMs
Chair: Mingyi Hong
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: multi-step preference optimization via two-player markov games
Speaker: Volkan Cevher
Abstract: Reinforcement Learning from Human Feedback (RLHF) has been highly success- ful in aligning large language models with human preferences. While prevalent methods like DPO have demonstrated strong performance, they frame interactions with the language model as a bandit problem, which limits their applicability in real-world scenarios where multi-turn conversations are common. Additionally, DPO relies on the Bradley-Terry model assumption, which does not adequately capture the non-transitive nature of human preferences. In this paper, we address these challenges by modeling the alignment problem as a two-player constant-sum Markov game, where each player seeks to maximize their winning rate against the other across all steps of the conversation. Our approach Multi-step Preference Optimization (MPO) is built upon the natural actor-critic framework (Peters & Schaal, 2008). We further develop MPO based on the optimistic online gradient descent algorithm (Rakhlin & Sridharan, 2013; Joulani et al., 2017). Theoretically, we provide a rigorous analysis for both algorithms on convergence and show that 0MPO requires O(ϵ−1) policy updates to converge to an ϵ-approximate Nash equi- librium. We also validate the effectiveness of our method through experiments on the multi-turn conversations dataset in MT-bench-101.

Talk 2: Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration via Bilevel Optimization Improves LLM Alignment
Speaker: Mingyi Hong
Abstract: Aligning human preference and value is an important requirement for contemporary foundation models. State-of-the-art techniques such as Reinforcement Learning from Human Feedback (RLHF) often consist of two stages: 1) supervised fine-tuning (SFT), where the model is fine-tuned by learning from human demonstration data; 2) Preference learning, where preference data is used to learn a reward model, which is in turn used by a reinforcement learning (RL) step to fine-tune the model. Such reward model serves as a proxy to human preference, and it is critical to guide the RL step towards improving the model quality. In this work, we argue that the SFT stage significantly benefits from learning a reward model as well. Instead of using the human demonstration data directly via supervised learning, we propose to leverage an Inverse Reinforcement Learning (IRL) and bilevel optimization technique to simultaneously build an reward model and a policy model. This approach leads to new SFT algorithms that are not only efficient to implement, but are robust to the presence of low-quality supervised learning data. Moreover, we discover a connection between the proposed IRL based approach, and a recent line of works called Self-Play Fine-tune (SPIN). Theoretically, we show that the proposed algorithms converge to the stationary solutions of the IRL problem. Empirically, we align 1B and 7B models using proposed methods and evaluate them on a reward benchmark model and the HuggingFace Open LLM Leaderboard. The proposed methods show significant performance improvement over existing SFT approaches. Our results indicate that it is beneficial to leverage reward learning throughout the entire alignment process.

Talk 3: Pre-training Differentially Private Models with Limited Public Data
Speaker: Xinwei Zhang
Abstract: The superior performance of large foundation models relies on the use of massive amounts of high-quality data, which often contain sensitive, private, and copyrighted material that requires formal protection. While differential privacy (DP) is a prominent method to gauge the degree of security provided to the models, its application is commonly limited to the model fine-tuning stage due to the performance degradation when DP is applied during the pre-training stage. Consequently, DP is yet incapable of protecting a substantial portion of the data used during the initial pre-training process. In this work, we provide a theoretical understanding of the efficacy of DP training by analyzing the improvement of per-iteration loss through the lens of the Hessian matrix for large neural networks. We make a key observation that DP optimizers' performance degradation can be significantly mitigated by the use of limited public data, which leads to a novel DP continual pre-training strategy. Empirically, using only 10\% of public data, our strategy can achieve DP accuracy of 41.5% on ImageNet-21k (with =8), as well as non-DP accuracy of 55.7% and 60.0% on downstream tasks Places365 and iNaturalist-2021, respectively, on par with state-of-the-art standard pre-training and substantially outperforming existing DP pre-trained models.

Speakers
VC

Volkan Cevher

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MH

Mingyi Hong

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XZ

Xinwei Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6G: Advances in Data-driven Optimization
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Advances in Data-driven Optimization
Chair: Rui Gao & Daniel Kuhn
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: A Class of Interpretable and Decomposable Multi-period Convex Risk Measures
Speaker: Luhao Zhang
Abstract: Multi-period risk measures evaluate the risk of a stochastic process by assigning it a scalar value. A desirable property of these measures is dynamic decomposition, which allows the risk evaluation to be expressed as a dynamic program. However, many widely used risk measures, such as Conditional Value-at-Risk, do not possess this property. In this work, we introduce a novel class of multi-period convex risk measures that do admit dynamic decomposition. Our proposed risk measure evaluates the worst-case expectation of a random outcome across all possible stochastic processes, penalized by their deviations from a nominal process in terms of both the likelihood ratio and the outcome. We show that this risk measure can be reformulated as a dynamic program, where, at each time period, it assesses the worst-case expectation of future costs, adjusting by reweighting and relocating the conditional nominal distribution. This recursive structure enables more efficient computation and clearer interpretation of risk over multiple periods.

Talk 2: An MILP-Based Solution Scheme for Factored and Robust Factored Markov Decision Processes
Speaker: Man-Chung Yue
Abstract: Factored Markov decision processes (MDPs) are a prominent paradigm within the artificial intelligence community for modeling and solving large-scale MDPs whose rewards and dynamics decompose into smaller, loosely interacting components. Through the use of dynamic Bayesian networks and context-specific independence, factored MDPs can achieve an exponential reduction in the state space of an MDP and thus scale to problem sizes that are beyond the reach of classical MDP algorithms. However, factored MDPs are typically solved using custom-designed algorithms that can require meticulous implementations and considerable fine-tuning. In this paper, we propose a mathematical programming approach to solving factored MDPs. In contrast to existing solution schemes, our approach leverages off-the-shelf solvers, which allows for a streamlined implementation and maintenance; it effectively capitalizes on the factored structure present in both state and action spaces; and it readily extends to the largely unexplored class of robust factored MDPs, whose transition kernels are only known to reside in a pre-specified ambiguity set. Our numerical experiments demonstrate the potential of our approach.

Talk 3: A Deep Learning Approach to Multistage Stochastic Programming
Speaker: Rui Gao
Abstract: Multistage stochastic programming problems are challenging due to the curse of dimensionality. We introduce a practical algorithm for solving multistage stochastic programming in high dimensions, leveraging neural networks to parameterize the policy. The proposed algorithm demonstrates effectiveness in terms of both accuracy and speed across a variety of problems.

Speakers
LZ

Luhao Zhang

Name: Dr. Luhao ZhangTitle: Assistant ProfessorAffiliation: Deparment of Applied Mathematics and Statistics, Johns Hopkins University
RG

Rui Gao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6H: Semidefinite programming and applications
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Semidefinite programming and applications
Chair: Georgina Hall
Cluster: Conic and Semidefinite Optimization

Talk 1: A Matrix Generalization of the Goemans-Williamson Algorithm for Orthogonality Constrained Optimization Problems
Speaker: Ryan Cory-Wright
Abstract: A central research question in optimization concerns developing algorithms that solve non-convex problems to provable near optimality in a practically tractable amount of time. One popular two-step methodology called ``relax-and-round'' successfully addresses many well-behaved non-convex problems. In 1995, Goemans and Williamson proposed a polynomial-time relax-and-round algorithm for approximately solving Max-Cut problems by rounding their semidefinite relaxations. In this work, we propose a matrix generalization of their approach which applies to orthogonal matrices where U^\top U=I, as opposed to binary variables where z^2=1, and study the quality of this approach in both theory and practice. Our approach has applications in low-rank matrix completion and sparse PCA with multiple PC problems, among others.

Talk 2: Generalized Ellipsoids
Speaker: Cemil Dibek
Abstract: Ellipsoids are fundamental in applied and computational mathematics, featuring in optimization, control, convex geometry, and statistics due to their geometric and computational properties. In this talk, we introduce a new family of symmetric convex bodies called generalized ellipsoids of degree d (GE-ds), with ellipsoids corresponding to the case of d=0. Generalized ellipsoids (GEs) retain many geometric, algebraic, and algorithmic properties of ellipsoids, while also being tractable to search for and optimize over. We show that one can search for GEs of a given degree by solving a semidefinite program whose size grows only linearly with dimension, and that every GE has a semidefinite representation whose size depends linearly on both its dimension and degree. In terms of expressiveness, we demonstrate that every symmetric full-dimensional polytope and every intersection of co-centered ellipsoids can be represented exactly as a GE-d. Using this result, we show that every symmetric convex body can be approximated arbitrarily well by a GE-d. We also present applications of GEs in areas such as portfolio optimization, stability analysis of switched linear systems, and robust optimization.

Talk 3: TBD
Speaker: Georgina Hall
Abstract: TBD

Speakers
RC

Ryan Cory-Wright

Assistant Professor of Analytics and Operations, Imperial College London
I am an Assistant Professor in the Analytics and Operations Group at Imperial College Business School (ICBS) since July 2023, affiliated with the Imperial-X initiative on interdisciplinary AI/ML.My research focuses on optimization, machine learning, statistics, and their application in business analytics. I am particularly interested in broadening the scope of optimization to address practical problems that current methods cannot solve... Read More →
CD

Cemil Dibek

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GH

Georgina Hall

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6I: Advances in Conic Optimization
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Advances in Conic Optimization
Chair: Timotej Hrga
Cluster: Conic and Semidefinite Optimization

Talk 1: Completely positive reformulations for sparse quadratic optimization
Speaker: Bo Peng
Abstract: In this talk, we explore the completely positive reformulation of quadratic optimization problems involving the $\ell_0$ quasinorm, which is well-known for promoting solution sparsity. Compared to existing approaches in the literature, we propose novel, more compact relaxations that are proven to be exact under milder conditions. Furthermore, we demonstrate the exactness of a decomposed completely positive relaxation by leveraging inherent sparse patterns of the model. A numerical study is conducted to compare double nonnegative relaxations derived from these reformulations. Extensive numerical experiments illustrate the quality of the resulting bounds while maintaining tractability and scalability.

Talk 2: Connectivity via convexity: Bounds on the edge expansion in graphs
Speaker: Timotej Hrga
Abstract: Convexification techniques have gained increasing interest over the past decades. In this work, we apply a recently developed convexification technique for fractional programs by He, Liu and Tawarmalani (2024) to the problem of determining the edge expansion of a graph. Computing the edge expansion of a graph is a well-known, difficult combinatorial problem that seeks to partition the graph into two sets such that a fractional objective function is minimized. We give a formulation of the edge expansion as a completely positive pro- gram and propose a relaxation as a doubly non-negative program, further strengthened by cutting planes. Additionally, we develop an augmented Lagrangian algorithm to solve the doubly non-negative program, obtaining lower bounds on the edge expansion. Numerical results confirm that this relaxation yields strong bounds and is computationally efficient, even for graphs with several hundred vertices.

Talk 3: Facial Reduction in BiqCrunch
Speaker: Ian Sugrue
Abstract: BiqCrunch is a binary quadratic solver that uses a semidefinite bounding procedure with a branch-and-bound framework to get exact solutions to combinatorial optimization problems. Since its latest official update, BiqCrunch has undergone further development, most notably the inclusion of the dimensionality reduction process known as facial reduction. We will discuss the details of how facial reduction is implemented, the numerical results that support this inclusion, as well as future development plans for BiqCrunch.

Speakers
BP

Bo Peng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TH

Timotej Hrga

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6J: Continuous and discreate optimization solving real problems
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Continuous and discreate optimization solving real problems
Chair: Julio C. Góez
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Active constraint indicators in interior point algorithms for conic optimization
Speaker: Alexander Guldemond
Abstract: In conic optimization, determining which constraints are active at a given solution can be crucial information for the user. In this talk, we discuss an approach used by Mosek to predict active conic constraints within the framework of interior point algorithms. It is well known that in the case of the nonnegative orthant, active sets can be predicted by comparing $dx_i / x_i$ to $ds_i / s_i$ for each primal dual variable pair. We extend this technique to generic conic constraints, including non-symmetric cones. At each iteration, we leverage information from both the current iterate and the computed search direction to identify likely active conic constraints. We will present theoretical insights, computational experiments, and discuss the implications for large-scale optimization problems.

Talk 2: A Mixed-Integer Conic Program for the Multi-Pursuer Moving-Target Traveling Salesmen Problem
Speaker: Allen George Philip
Abstract: The Moving-Target Traveling Salesman Problem (MT-TSP) seeks to find a shortest path for a pursuer, that starts at a depot, visits a set of moving targets, and returns to the depot. This problem is motivated by various practical applications such as monitoring and surveillance, intercepting hostile UAVs, and motion planning for industrial robots. We consider the Multi-Pursuer Moving-Target Traveling Salesman Problem (MP-MT-TSP) which generalizes the MT-TSP to include multiple pursuers. We introduce a new Mixed-Integer Conic Program (MICP) for MP-MT-TSP where targets move along piecewise-linear paths. We compare our formulation with the current state-of-the-art MICP for the MP-MT-TSP, and present experimental results demonstrating significant improvements over the state-of-theart in both runtime and optimality ga

Talk 3: Sparse Polynomial Optimization for Water Networks
Speaker: Olga Heijmans-Kuryatnikova
Abstract: In this work we exploit sparsity in polynomial optimization for the valve setting problem in water networks. The valve setting problem consists of many small subproblems involving few variables and monomials and connected with sparse linking constraints. The problem exhibits correlative sparsity (subsets of variables appearing separately in constraints), term sparsity (few monomials present in the problem), and ideal sparsity (size reduction due to equality constraints). We suggest a new simple SDP relaxation that uses all types of sparsity to reach a trade-off between the bound quality and running times. We compare it with the existing sparse SDP relaxations and report numerical results on four water networks ranging in size from 4 to about 2000 nodes. The approach extends to electricity and gas network problems.

Speakers
JC

Julio C Góez

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AG

Alexander Guldemond

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AG

Allen George Philip

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
OH

Olga Heijmans-Kuryatnikova

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6K: Applications To Signal Processing
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Applications To Signal Processing
Chair: Robert Bassett
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Stochastic Optimal Search with Signals
Speaker: Jefferson Huang
Abstract: We consider the problem of optimally searching for a target located at one of the nodes of a network. On each time step, the searcher receives a (possibly unreliable) signal indicating where the target is on the network. Starting with the case of searching on a line, we explore the structure of search policies that optimally make use of the signals. 

Talk 2: Sparse signal reconstruction for over-dispersed low-photon count imaging
Speaker: Roummel Marcia
Abstract: Low-photon count imaging is typically modeled by Poisson statistics. This discrete probability distribution model assumes that the mean and variance of a signal are equal. In the presence of greater variability in a dataset than what is expected, the negative binomial distribution is a suitable over-dispersed alternative to the Poisson distribution. In this work, we present an optimization framework for reconstructing sparse signals in these over-dispersed low-count settings.

Talk 3: Decentralized Sensor Network Localization via Customized Proximal Splitting
Speaker: Peter Barkley
Abstract: We apply recent advances in the design of custom proximal splitting algorithms to build a decentralized algorithm for solving the node-based SDP relaxation of the sensor network localization problem using noisy distance data. We implement the algorithm using MPI. We then explore the performance of various algorithm design options, and compare the decentralized algorithm with other decentralized approaches, including a graph-based decentralized ADMM.

Speakers
RB

Robert Bassett

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JH

Jefferson Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RM

Roummel Marcia

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Peter Barkley

Peter Barkley

US Navy
I am a naval officer and a PhD student in Operations Research at the Naval Postgraduate School. My research interests include decentralized optimization, scheduling, and machine learning. In previous roles in the Navy, I’ve flown the P-3C Orion as a flight instructor and mission... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6L: Alternative and Hybrid Algorithms in Quantum Computing for Optimization and Applications
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Alternative and Hybrid Algorithms in Quantum Computing for Optimization and Applications
Chair: Xiu Yang
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Unleashed from Constrained Optimization: Quantum Computing for Quantum Chemistry Employing Generator Coordinate Method
Speaker: Bo Peng
Abstract: Hybrid quantum-classical approaches offer potential solutions to quantum chemistry problems, yet they also introduce challenges. These challenges include addressing the barren plateau and ensuring the accuracy of the ansatze, which often manifest as constrained optimization problems. In this work, we explore the interconnection between constrained optimization and generalized eigenvalue problems through a unique class of Givens rotations. These rotations frequently serve as disentangled unitary coupled cluster building blocks constituting the ansatze in variational quantum eigensolver (VQE) and adaptive derivative-assembled pseudo-Trotter VQE (ADAPT-VQE) simulations. Herein, we employ Givens rotations to construct non-orthogonal, overcomplete many-body generating functions, projecting the system Hamiltonian into a working subspace. The resulting generalized eigenvalue problem is proven to generate rigorous lower bounds to the VQE/ADAPT-VQE energies, effectively circumventing the barren plateau issue and the heuristic nature of numerical minimizers in standard VQE processes. For practical applications, we further propose an adaptive scheme for the robust construction of many-body basis sets using these Givens rotations, emphasizing a linear expansion that balances accuracy and efficiency. The effective Hamiltonian generated by our approach would also facilitate the computation of excited states and evolution, laying the groundwork for more sophisticated quantum simulations in chemistry.

Talk 2: Quantum DeepONet: Neural operators accelerated by quantum computing
Speaker: Lu Lu
Abstract: In the realm of mathematics, engineering, and science, constructing models that reflect real- world phenomena requires solving partial differential equations (PDEs) with different param- eters. Recent advancements in DeepONet, which learn mappings between infinite-dimensional function spaces, promise efficient evaluations of PDE solutions for new parameter sets in a single forward pass. However, classical DeepONet entails quadratic time complexity concerning input dimensions during evaluation. Given the progress in quantum algorithms and hardware, we propose utilizing quantum computing to accelerate DeepONet evaluations, resulting in time complexity that is linear in input dimensions. Our approach integrates unary encoding and orthogonal quantum layers to facilitate this process. We benchmark our Quantum DeepONet using a variety of equations, including the first-order linear ordinary differential equation, advection equation, and Burgers' equation, demonstrating the method’s efficacy in both ideal and noisy conditions. Furthermore, we show that our quantum DeepONet can also be informed by physics, minimizing its reliance on extensive data collection. We expect Quantum DeepONet to be particularly advantageous in applications in outer loop problems which require to explore parameter space and solving the corresponding PDEs, such as forward uncertainty propagation and optimal experimental design.

Talk 3: Koopman Linearization for Optimization in Quantum Computing
Speaker: Xiu Yang
Abstract: Nonlinearity presents a significant challenge in developing quantum algorithms involving differential equations, prompting the exploration of various linearization techniques, including the well-known Carleman Linearization. Instead, this paper introduces the Koopman Spectral Linearization method tailored for nonlinear autonomous ordinary differential equations. This innovative linearization approach harnesses the interpolation methods and the Koopman Operator Theory to yield a lifted linear system. It promises to serve as an alternative approach that can be employed in scenarios where Carleman Linearization is traditionally applied. Numerical experiments demonstrate the effectiveness of this linearization approach for several commonly used nonlinear ordinary differential equations. Hence, it enables a special design of gradient-descent type of method based on the technique called Schrodingerization that is used to solve linear differential equations on quantum computers.

Speakers
BP

Bo Peng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LL

Lu Lu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XY

Xiu Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6M: Data-Driven Decision and Control
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Data-Driven Decision and Control
Chair: Guannan Qu
Cluster: Optimization For Data Science

Talk 1: Robustness in data-driven control: Requirement? An opportunity
Speaker: Laixi Shi
Abstract: Reinforcement learning (RL), which strives to learn desirable sequential decisions based on trial-and-error interactions with an unknown environment, has achieved remarkable success recently in a variety of domains including games and large language model alignment. While standard RL has been heavily investigated recently, a policy learned in an ideal, nominal environment might fail catastrophically when the deployed environment is subject to small changes in task objectives or adversarial perturbations, especially in high-stake applications such as robotics and clinical trials. This talk concerns the central issues of sample efficiency and model robustness in reinforcement learning (RL) to reduce the sim-to-real gap in practice. We adopt the framework of distributionally robust Markov decision processes (RMDPs), aimed at learning a policy that optimizes the worst-case performance when the deployed environment falls within a prescribed uncertainty set around the nominal MDP. Despite recent efforts, the sample complexity of RMDPs remained mostly unsettled regardless of the uncertainty set in use. It was unclear if distributional robustness bears any statistical consequences when benchmarked against standard RL. Somewhat surprisingly, our results uncover that RMDPs are not necessarily easier or harder to learn than standard MDPs. The statistical consequence incurred by the robustness requirement depends heavily on the size and shape of the uncertainty set. In addition, we break down the sample barrier of robust RL in offline setting by providing the first provable near-optimal algorithm for offline robust RL that can learn under simultaneous model uncertainty and limited historical datasets.

Talk 2: Learning and Control in Countable State Spaces
Speaker: Rayadurgam Srikant
Abstract: We will consider policy optimization methods in reinforcement learning where the state space is countably infinite. The motivation arises from control problems in communication networks and matching markets. Specifically, we consider the popular Natural Policy Gradient (NPG) algorithm, which has been studied in the past only under the assumptions that the cost is bounded and the state space is finite, neither of which holds for the aforementioned control problems. Assuming a Lyapunov drift condition, which is naturally satisfied in some cases and can be satisfied in other cases at a small cost in performance, we design a state-dependent step-size rule which dramatically improves the performance of NPG for our intended applications. In addition to experimentally verifying the performance improvement, we also theoretically show that the iteration complexity of NPG can be made independent of the size of the state space. The key analytical tool we use is the connection between NPG stepsizes and the solution to Poisson’s equation. In particular, we provide policy-independent bounds on the solution to Poisson’s equation, which are then used to guide the choice of NPG stepsizes.

Talk 3: Distributionally Robust Control via Optimal Transport
Speaker: Liviu Aolaritei
Abstract: In this talk I will challenge the standard uncertainty models, i.e., robust (norm-bounded) and stochastic (one fixed distribution, e.g., Gaussian), and propose to model uncertainty in dynamical systems via Optimal Transport (OT) ambiguity sets. I will then show that OT ambiguity sets are analytically tractable: they propagate easily and intuitively through linear and nonlinear (possibly corrupted by noise) maps, and the result of the propagation is again an OT ambiguity set or can be tightly upper bounded by one. In the context of dynamical systems, this allows to consider multiple sources of uncertainty (e.g., initial condition, additive noise, multiplicative noise) and to capture in closed-form, via an OT ambiguity set, the resulting uncertainty in the state at any future time. The resulting OT ambiguity sets are also computationally tractable, and can be directly employed in various distributionally robust control formulations that can optimally trade between safety and performance.

Speakers
LS

Laixi Shi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RS

Rayadurgam Srikant

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LA

Liviu Aolaritei

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6N: The Role of Constraints in the Computational Complexity and Solvability of Optimization Problems
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: The Role of Constraints in the Computational Complexity and Solvability of Optimization Problems
Chair: Tatjana Chavdarova
Cluster: Fixed Points and Variational Inequalities

Talk 1: On the Role of Constraints in the Complexity of Min-Max Optimization
Speaker: Gabriele Farina
Abstract: We investigate the role of constraints in the computational complexity of min-max optimization. The work of Daskalakis, Skoulakis, and Zampetakis [2021] was the first to study min-max optimization through the lens of computational complexity, showing that min-max problems with nonconvex-nonconcave objectives are PPAD-hard. However, their proof hinges on the presence of joint constraints between the maximizing and minimizing players. The main goal of this paper is to understand the role of these constraints in min-max optimization. The first contribution of this paper is a fundamentally new proof of their main result, which improves it in multiple directions: it holds for degree 2 polynomials, it is essentially tight in the parameters, and it is much simpler than previous approaches, clearly highlighting the role of constraints in the hardness of the problem. Second, we show that with general constraints (i.e., the min player and max player have different constraints), even convex-concave min-max optimization becomes PPAD-hard. Along the way, we also provide PPAD-membership of a general problem related to quasi-variational inequalities, which has applications beyond our problem.

Talk 2: A First Order Primal-Dual Method for Solving Constrained Variational Inequalities
Speaker: Tatjana Chavdarova
Abstract: We introduce an interior-point method to solve constrained variational inequality (cVI) problems. By combining primal-dual and interior-point techniques, we develop a family of first-order methods called the ADMM-based interior-point method for constrained VIs (ACVI). This approach enables the solution of cVIs with complex constraints. We establish convergence guarantees for ACVI under the assumption that the operator is monotone (not necessarily L-Lipschitz) and that its subproblems can be solved exactly. Specifically, we achieve a convergence rate of O(1/sqrt(K)) for the gap function of the last iterate, matching the known lower bound. To address cases where subproblems cannot be solved analytically, we propose an inexact version of ACVI, which leverages a warm-starting technique for the subproblems, taking advantage of the fact that these subproblems change only slightly between iterations. When the operator is monotone, L-Lipschitz, and the errors in solving the subproblems decrease at appropriate rates, we demonstrate that the same convergence rate holds for the gap function. To the best of our knowledge, this is the first first-order interior-point method for general cVI problems with a global convergence guarantee.

Talk 3: The Computational Complexity of Finding Stationary Points in Non-convex Optimization
Speaker: Manolis Zampetakis
Abstract: Finding approximate stationary points, i.e., points where the gradient is approximately zero, of non-convex but smooth objective functions over unrestricted d-dimensional domains is one of the most fundamental problems in classical non-convex optimization. Nevertheless, the computational and query complexity of this problem are still not well understood when the dimension d of the problem is independent of the approximation error. In this paper, we show the following computational and query complexity results: 1. The problem of finding approximate stationary points over unrestricted domains is PLS-complete. 2. For d = 2, we provide a zero-order algorithm at a lower bound that shows that this algorithm achieves the optimal query complexity in both the constrained and the unconstrained setting. 3. Combining our results with recent results in complexity theory we show that finding approximate KKT points in constrained optimization is reducible to finding approximate stationary points in unconstrained optimization but the converse is impossible.

Speakers
GF

Gabriele Farina

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TC

Tatjana Chavdarova

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MZ

Manolis Zampetakis

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6O: First-order methods for nonsmooth and constrained optimization - II
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: First-order methods for nonsmooth and constrained optimization - II
Chair: Zhe (Jimmy) Zhang
Cluster: Nonlinear Optimization

Talk 1: A Single-Loop Spider-Type Stochastic Subgradient Method for Nonconvex Nonsmooth Expectation Constrained Optimization
Speaker: Wei Liu
Abstract: Many real-world problems involve complex nonconvex functional constraints and large datasets, necessitating efficient stochastic methods for solving stochastic optimization problems. A majority of existing works assume no constraints or easy-to-project constraints. In this paper, we consider nonconvex stochastic optimization problems with nonconvex expectation constraints. We construct an unconstrained exact penalty model with a new penalized function for the expectation constraints that share the same stationary points as the original problem. To solve this problem, we present a single-loop spider-type stochastic subgradient first-order method, which utilizes the derivatives of f and g and the function value of g at each iteration. Under certain regularity conditions (weaker than Slater-type constraint qualification in existing works), we establish an oracle complexity result of O(ϵ−4) to reach a point that is ϵ\epsilonϵ-close to an ϵ\epsilonϵ-KKT point of the original problem in expectation, matching the lower bound for such tasks. As important applications, we apply our method to two fairness-constrained problems, demonstrating that it is at least 9 times faster than state-of-the-art algorithms, including switching subgradient methods and inexact proximal point methods.

Talk 2: Monotone Variational Inequality problem under Generalized Conditions
Speaker: Digvijay Boob
Abstract: We consider the well-known variational inequality (VI) problem on monotone operators under a novel Lipschitz type condition and possibly unbounded feasible set. This new class of problems cover various class of problems, including convex function constrained problem, convex semi-infinite constrained problem, general robust optimization problem, and function-constrained variational inequalities among others - covering wide class of problems as a specific case. We show that when the problem is generalized smooth, our method converges at the rate of O(1/K) in terms of Minty-gap criterion appropriately defined for unbounded sets. For strongly monotone generalized smooth problems, we show linear convergence. For generalized nonsmooth and stochastic VI problems, we show convergence at the rate of O(1/\sqrt{K}). To our best knowledge, this is the first time such problems are addressed, especially in the context of variational inequality problems.

Talk 3: Stochastic Block-Wise Iterative Methods for Training Feasibility-Based Neural Architectures
Speaker: Manish Krishan
Abstract: We present a unifying framework for finitely convergent stochastic block-wise iterative methods, tailored for training set-feasibility-based neural network architectures. We apply this disciplined approach to selecting both the architecture and its corresponding training algorithm for classification and regression tasks.

Speakers
avatar for Wei Liu

Wei Liu

research scholar, Rensselaer Polytechnic Institute
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DB

Digvijay Boob

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MK

Manish Krishan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6P: Recent Advances in PDE-Constrained Optimizatin
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Recent Advances in PDE-Constrained Optimizatin
Chair: Michael Ulbrich
Cluster: PDE-constrained Optimization

Talk 1: Probabilistic state constraints for optimal control problems under uncertainty
Speaker: Caroline Geiersbach
Abstract: In this talk, we discuss optimal control problems subject to random state constraints, where we distinguish between the chance-constrained case and the almost sure formulation. We highlight some of the difficulties in the infinite-dimensional setting, which is of interest in physics-based models where a control belonging to a Banach space acts on a system described by a partial differential equation (PDE) with random inputs or parameters. We study the setting in which the obtained state should be bounded uniformly over the physical domain with high probability, or even probability one. We apply our results to a model with a random elliptic PDE, where the randomness is induced by the right-hand side. For the chance-constrained setting, this structure allows us to obtain an explicit representation for the Clarke subdifferential of the probability function using the spherical radial decomposition of Gaussian random vectors. This representation is used for the numerical solution in a discretize-then-optimize approach. For the almost sure setting, we use a Moreau-Yosida regularization and solve a sequence of regularized problems in an optimize-then-discretize approach. The solutions are compared, providing insights for the development of further algorithms.

Talk 2: Image Registration in Non-Reflexive Banach Spaces
Speaker: Johannes Haubner
Abstract: We consider image registration as an optimal control problem using an optical flow formulation, i.e., we aim to solve an optimization problem that is governed by a linear hyperbolic transport equation. In order to be able to transform the image of a square into the image of a circle and ensure bi-Lipschitz continuity of the transformation, we aim for $W^{1,\infty}$-regularity of the velocity that parametrizes the transformation. This leads to an optimization problem in the non-reflexive Banach space $W^{1,\infty}$. We introduce relaxations of the optimization problem involving smoothed maximum and minimum functions and the Orlicz space $L_{\exp}$. To derive well-posedness results for the relaxed optimization problem, we revisit and establish new existence and uniqueness results for the linear hyperbolic transport equations. We present limit considerations and discuss differentiability.

Talk 3: Numerical methods for shape optimal shpe design of fluid-structure interaction problems
Speaker: Michael Ulbrich
Abstract: We consider the method of mappings for performing shape optimization for unsteady fluid–structure interaction (FSI) problems. The main focus is on the numerical implementation. Our modelling of the optimization problem is guided by several theoretical aspects, such as regularity requirements on the transformations and connections to differential geometric aspects. We discretize the problem in a way such that exact discrete gradients can be computed. This in combination with the way we represent shapes allows for an efficent application of general purpose optimization solvers. Our numerical tests focus on problems derived from an FSI benchmark to validate our implementation, which builds on FEniCS, dolfin-adjoint, and IPOPT. The method is used to optimize parts of the outer boundary and the interface.

Speakers
CG

Caroline Geiersbach

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JH

Johannes Haubner

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MU

Michael Ulbrich

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6Q: Optimization on Riemannian manifolds and stratified sets (2/2)
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Optimization on Riemannian manifolds and stratified sets (2/2)
Chair: Guillaume Olikier
Cluster: Optimization on Manifolds

Talk 1: A space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints
Speaker: Bin Gao
Abstract: Imposing additional constraints on low-rank optimization has garnered growing interest recently. However, the geometry of coupled constraints restricts the well-developed low-rank structure and makes the problem nonsmooth. In this paper, we propose a space-decoupling framework for optimization problems on bounded-rank matrices with orthogonally invariant constraints. The "space-decoupling" is reflected in several ways. Firstly, we show that the tangent cone of coupled constraints is the intersection of the tangent cones of each constraint. Secondly, we decouple the intertwined bounded-rank and orthogonally invariant constraints into two spaces, resulting in optimization on a smooth manifold. Thirdly, we claim that implementing Riemannian algorithms is painless as long as the geometry of additional constraint is known a prior. In the end, we unveil the equivalence between the original problem and the reformulated problem. The numerical experiments validate the effectiveness and efficiency of the proposed framework.

Talk 2: A Riemannian rank-adaptive method for tensor completion in the tensor-train format
Speaker: Charlotte Vermeylen
Abstract: Riemannian rank-adaptive methods (RRAMs) are state-of-the-art methods that aim to optimize a continuously differentiable function on a low-rank variety, a problem appearing notably in low-rank matrix or tensor completion. The RRAMs that are developed for the set of bounded rank matrices iteratively optimize over smooth fixed-rank manifolds, starting from a low initial rank. They increase the rank by performing a line search along a descent direction selected in the tangent cone to the variety. This direction can be the projection of the negative gradient onto the tangent cone but does not need to; for instance, the RRAM developed by Gao and Absil uses the projection of the negative gradient onto the part of the tangent cone that is normal to the tangent space. Additionally, they decrease the rank based on a truncated SVD when the algorithm converges to an element of a lower-rank set. This is possible because the manifold is not closed. We aim to generalize these RRAMs to the tensor-train (TT) format. In this format, only the RRAM by Steinlechner is known to us from the literature. This method is developed for high-dimensional tensor completion and has a random rank update mechanism in the sense that each TT-rank is increased subsequently by one by adding a small random term in the TTD of the current best approximation such that the cost function does not increase much. No rank reduction step is included which makes the algorithm prone to overfitting. We improve this RRAM by including a method to increase the TT-rank based on an approximate projection of the negative gradient onto the tangent cone to the variety. The tangent cone is the set of all tangent vectors to the variety. The approximate projection ensures that the search direction is sufficiently gradient related. Furthermore, a method is proposed to determine how much the rank should be increased for the low-rank tensor completion problem (LRTCP). Lastly, a method to decrease the rank is proposed, which is necessary when the iterate comes close to a lower-rank set. This is possible because, as for the manifold of fixed-rank matrices, the manifold of fixed-rank TTDs is not closed.

Talk 3: Low-rank optimization on Tucker tensor varieties
Speaker: Renfeng Peng
Abstract: In the realm of tensor optimization, the low-rank Tucker decomposition is crucial for reducing the number of parameters and saving storage. In this talk, we delve into the geometry and optimization methods for Tucker tensor varieties---the set of tensors with bounded Tucker rank---which is notably more intricate than the well-explored matrix varieties. We give an explicit parametrization of the tangent cone of Tucker tensor varieties and leverage its geometry to develop provable gradient-related line-search methods for optimization on Tucker tensor varieties. In practice, low-rank tensor optimization suffers from the difficulty of choosing a reliable rank parameter. To this end, we incorporate the established geometry and propose a Tucker rank-adaptive method that aims to identify an appropriate rank. Numerical experiments on tensor completion reveal that the proposed methods are in favor of recovering performance over other state-of-the-art methods. This is joint work with Bin Gao (AMSS) and Ya-xiang Yuan (AMSS).

Speakers
GO

Guillaume Olikier

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BG

Bin Gao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CV

Charlotte Vermeylen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Renfeng Peng

Renfeng Peng

Ph. D. student, Academy of Mathematics and Systems Science, Chinese Academy of Sciences
Name: Renfeng PengTitle: Low-rank optimization on Tucker tensor varietiesAffiliation: Academy of Mathematics and Systems Science, Chinese Academy of SciencesBio:Renfeng Peng is a Ph.D. student of the Optimization Group at Academy of Mathematics and Systems Science (AMSS), Chinese... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6R: Recent Adavances in Interval Arithmetic based Methods
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Recent Adavances in Interval Arithmetic based Methods
Chair: Frederic Messine
Cluster: Global Optimization

Talk 1: Boosting the performance of rigorous global optimization
Speaker: Victor Reyes Rodriguez
Abstract: In this talk, I will present some boosting methods to improve the performance of rigorous global optimization.

Talk 2: Node selection through upper bounding local search methods in branch & bound solvers for NCOPs
Speaker: Ignacio Araya
Abstract: In this talk, I will present how to improve the convergence of branch & bound solvers for NCOPs by making node selection through upper bounding local search methods.

Talk 3: Interval Branch and Bound Code with PDE-constraints
Speaker: Frédéric Messine
Abstract: In this article, I will show how, in a global optimization solver whose bound calculations are performed by interval arithmetic, we can insert constraints based on numerical simulations. The aim of this extension is to provide exact solutions to concrete complex problems: an example will be given for the design of electric motors.

Speakers
VR

Victor Reyes Rodriguez

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
IA

Ignacio Araya

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FM

Frédéric Messine

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6S: Multi-agent learning in games and markets
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Multi-agent learning in games and markets
Chair: Manxi Wu
Cluster: Multi-agent Optimization and Games

Talk 1: Learning Optimal Stable Matches in Decentralized Markets with Unknown Preferences
Speaker: Bryce Ferguson
Abstract: Matching algorithms have demonstrated great success in several practical applications, but they often require centralized coordination and plentiful information. In many modern online marketplaces, agents must independently seek out and match with another using little to no information. For these kinds of settings, can we design decentralized, limited-information matching algorithms that preserve the desirable properties of standard centralized techniques? In this work, we constructively answer this question in the affirmative. We model a two-sided matching market as a game consisting of two disjoint sets of agents, referred to as proposers and acceptors, each of whom seeks to match with their most preferable partner on the opposite side of the market. However, each proposer has no knowledge of their own preferences, so they must learn their preferences while forming matches in the market. We present a simple online learning rule that guarantees a strong notion of probabilistic convergence to the welfare-maximizing equilibrium of the game, referred to as the proposer-optimal stable match. To the best of our knowledge, this represents the first completely decoupled, communication-free algorithm that guarantees probabilistic convergence to an optimal stable match, irrespective of the structure of the matching market.

Talk 2: Stochastic Online Fisher Markets: Static Pricing Limits and Adaptive Enhancements
Speaker: Devansh Jalota
Abstract: Fisher markets are one of the most fundamental models for resource allocation. However, the problem of computing equilibrium prices in Fisher markets typically relies on complete knowledge of users' budgets and utility functions and requires transactions to happen in a static market where all users are present simultaneously. Motivated by these practical considerations, we study an online variant of Fisher markets, wherein users with privately known utility and budget parameters, drawn i.i.d. from a distribution, arrive sequentially. In this setting, we first study the limitations of static pricing algorithms, which set uniform prices for all users, along two performance metrics: (i) regret, i.e., the optimality gap in the objective of the Eisenberg-Gale program between an online algorithm and an oracle with complete information, and (ii) capacity violations, i.e., the over-consumption of goods relative to their capacities. Given the limitations of static pricing, we design adaptive posted-pricing algorithms, one with knowledge of the distribution of users' budget and utility parameters and another that adjusts prices solely based on past observations of user consumption, i.e., revealed preference feedback, with improved performance guarantees. Finally, we present numerical experiments to compare our revealed preference algorithm's performance to several benchmarks.

Talk 3: Decentralized learning in Markov potential games and beyond
Speaker: Manxi Wu
Abstract: Infinite-horizon stochastic games provide a versatile framework for studying the repeated interaction among multiple strategic agents in dynamic environments. However, computing equilibria in such games is highly complex, and the long-run outcomes of decentralized learning algorithms in multi-agent settings remain poorly understood. The first part of this talk introduces a multi-agent reinforcement learning dynamics tailored for independent and decentralized settings, where players lack knowledge of the game model and cannot coordinate. The proposed dynamics guarantee convergence to a stationary Nash equilibrium in Markov potential games, demonstrating the effectiveness of simple learning dynamics even with limited information. In the second part of the talk, we extend the learning framework to encompass Markov near potential games, offering flexibility to incorporate a wide range of practically-relevant multi-agent interaction settings. We present efficient algorithms for approximating the stationary Nash equilibrium and substantiate their effectiveness through regret analysis and numerical experiments.

Speakers
BF

Bryce Ferguson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DJ

Devansh Jalota

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MW

Manxi Wu

Assistant Professor, University of California, Berkeley
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6T: Randomized optimization algorithms 1/2
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Randomized optimization algorithms 1/2
Chair: Laurent Condat
Cluster: Nonlinear Optimization

Talk 1: Variance reduction for stochastic proximal algorithms
Speaker: Cheik Traoré
Abstract: In the context of finite sums minimization, variance reduction techniques are widely used to improve the performance of state-of-the-art stochastic gradient methods. Their practical impact is clear, as well as their theoretical properties. Stochastic proximal point algorithms have been studied as an alternative to stochastic gradient algorithms since they are more stable with respect to the choice of the stepsize but their variance reduced versions are not as studied as the gradient ones. In this work, we propose the first unified study of variance reduction techniques for stochastic proximal point algorithms. We introduce a generic stochastic proximal algorithm that can be specified to give the proximal version of SVRG, SAGA, and some of their variants for smooth and convex functions. We provide several convergence results for the iterates and the objective function values. In addition, under the Polyak-Łojasiewicz (PL) condition, we obtain linear convergence rates for the iterates and the function values. Our numerical experiments demonstrate the advantages of the proximal variance reduction methods over their gradient counterparts, especially about the stability with respect to the choice of the stepsize for difficult problems.

Talk 2: Taming Nonconvex Stochastic Mirror Descent with General Bregman Divergence
Speaker: Ilyas Fatkhullin
Abstract: This paper revisits the convergence of Stochastic Mirror Descent (SMD) in the contemporary nonconvex optimization setting. Existing results for batch-free nonconvex SMD restrict the choice of the distance generating function (DGF) to be differentiable with Lipschitz continuous gradients, thereby excluding important setups such as Shannon entropy. In this work, we present a new convergence analysis of nonconvex SMD supporting general DGF, that overcomes the above limitations and relies solely on the standard assumptions. Moreover, our convergence is established with respect to the Bregman Forward-Backward envelope, which is a stronger measure than the commonly used squared norm of gradient mapping. We further extend our results to guarantee high probability convergence under sub-Gaussian noise and global convergence under the generalized Bregman Proximal Polyak-{Ł}ojasiewicz condition. Additionally, we illustrate the advantages of our improved SMD theory in various nonconvex machine learning tasks by harnessing nonsmooth DGFs. Notably, in the context of nonconvex differentially private (DP) learning, our theory yields a simple algorithm with a (nearly) dimension-independent utility bound. For the problem of training linear neural networks, we develop provably convergent stochastic algorithms.

Talk 3: Adaptive Bregman-Kaczmarz: An approach to solve linear inverse problems with independent noise exactly
Speaker: Lionel Tondji
Abstract: We consider the block Bregman–Kaczmarz method for finite dimensional linear inverse problems. The block Bregman–Kaczmarz method uses blocks of the linear system and performs iterative steps with these blocks only. We assume a noise model that we call independent noise, i.e. each time the method performs a step for some block, one obtains a noisy sample of the respective part of the right-hand side which is contaminated with new noise that is independent of all previous steps of the method. One can view these noise models as making a fresh noisy measurement of the respective block each time it is used. In this framework, we are able to show that a well-chosen adaptive stepsize of the block Bregman–Kaczmarz method is able to converge to the exact solution of the linear inverse problem. The plain form of this adaptive stepsize relies on unknown quantities (like the Bregman distance to the solution), but we show a way how these quantities can be estimated purely from given data. We illustrate the finding in numerical experiments and confirm that these heuristic estimates lead to effective stepsizes.

Speakers
avatar for Laurent Condat

Laurent Condat

Senior Research Scientist, King Abdullah University of Science and Technology (KAUST)
Laurent Condat received a PhD in applied mathematics in 2006 from Grenoble Institute of Technology, Grenoble, France. After a postdoc in the Helmholtz Zentrum Muenchen, Munich, Germany, he was hired in 2008 as a permanent researcher by the French National Center for Scientific Research... Read More →
CT

Cheik Traoré

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
IF

Ilyas Fatkhullin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LT

Lionel Tondji

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6U: Applications of derivative-free optimization
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Applications of derivative-free optimization
Chair: Juliane Müller
Cluster: Derivative-free Optimization

Talk 1: Surrogate model guided search for optimizing metabolic models
Speaker: Juliane Müller
Abstract: The development and scaleup of novel biofuels require accurate prediction of microbial performance. While there are very sophisticated metabolic and expression (ME) models that simulate cellular metabolism and expression, their accuracy depends on a variety of parameters that must be tuned such that the model agrees with observation data. This yields a continuous optimization problem in which the quality of parameter sets must be assessed by running the compute intensive ME models. In this talk, we will present the challenge associated with this application and propose derivative-free surrogate models guided optimization approaches to tackle it.

Talk 2: A multilevel stochastic regularized first-order method with application to training
Speaker: Filippo Marini
Abstract: We present a new multilevel stochastic framework for the solution of optimization problems. The proposed approach uses random regularized first-order models that exploit an available hierarchical description of the problem, being either in the classical variable space or in the function space, meaning that different levels of accuracy for the objective function are available. We present the converge analysis of the method and show its numerical behavior on the solution of finite-sum minimization problems arising in binary classification problems.

Talk 3: Surrogate-based evolutionary optimization extended to categorical and dependent variables
Speaker: Charlotte Beauthier
Abstract: The objective of this work is the development of methods combining numerical simulation tools and advanced optimization algorithms, which play a crucial role in exploring new conceptual designs. Nowadays the exploitation of multi-disciplinary optimization using high-fidelity simulation models is common to many engineering design problems. A globally effective approach to optimization problems based on computationally expensive analysis lies in the exploitation of surrogate models. They act as cheap-to-evaluate alternatives to the original high-fidelity models reducing the computational cost, while still providing improved designs. Furthermore, in practice, various industrial design problems have continuous, discrete, and categorical variables. From an engineering point of view, the specific case of categorical variables is of great practical interest by their ability to represent the choice of a material, the type of engine architecture, the shape of a cross-section for a beam profile, etc. The contributions of this talk are focused on the management of mixed variables, in particular the categorical ones, in a surrogate-based evolutionary algorithm where dependency can also be defined between variables. More precisely, the dependency considered here is when the definition domain of a variable can be linked to another variable’s value. In order to deal with mixed and dependent variables, different methods are proposed to refine the notion of distance between mixed variables, using the notion of affinity, for instance, and to adapt the genetic operators in the evolutionary algorithm accordingly. These novel developments have been implemented in Minamo, Cenaero’s in-house design space exploration and multi-disciplinary optimization platform. Numerical results will be presented on specific test problems coming from structural and mechanical design frameworks to study the impact of the proposed strategies.

Speakers
JM

Juliane Müller

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6V: Trust-Region Methods
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Trust-Region Methods
Chair: Mohamed Laghdaf Habiboullah
Cluster: nan

Talk 1: Complexity of trust-region methods in the presence of unbounded Hessian approximations
Speaker: Mohamed Laghdaf Habiboullah
Abstract: We extend traditional complexity analyses of trust-region methods for unconstrained, possibly nonconvex, optimization. Whereas most complexity analyses assume uniform boundedness of the model Hessians, we work with potentially unbounded model Hessians. Boundedness is not guaranteed in practical implementations, in particular ones based on quasi-Newton updates such as PSB, BFGS and SR1. Our analysis is conducted for a family of trust-region methods that includes most known methods as special cases. We examine two regimes of Hessian growth: one bounded by a power of the number of successful iterations, and one bounded by a power of the number of iterations. This allows us to formalize and confirm the profound intuition of Powell [IMA J. Numer. Ana. 30(1):289-301,2010], who studied convergence under a special case of our assumptions, but whose proof contained complexity arguments. Specifically, for \(0 \leq p < 1\), we establish sharp \(O(\epsilon^{-2/(1-p)})\) evaluation complexity to find an \(\epsilon\)-stationary point when model Hessians are \(O(k^p)\), where \(k\) is the iteration counter. For \(p = 1\), which is the case studied by Powell, we establish a sharp \(O(\exp(c\epsilon^{-2}))\) evaluation complexity for a certain constant \(c > 0\). This is as Powell suspected and is far worse than other bounds surmised elsewhere in the literature. We establish similar bounds when model Hessians are \(O(|\mathcal{S}_k|^p)\), where \(|\mathcal{S}_k|\) is the number of iterations where the step was accepted, up to iteration \(k\). To the best of our knowledge, ours is the first work to provide complexity bounds when model Hessians grow linearly with \(|\mathcal{S}_k|\) or at most linearly with \(k\), which covers multiple quasi-Newton approximations. Link: https://arxiv.org/abs/2408.06243

Talk 2: Advancing Trust-Region Methods: Gradient-Based Radius Updates and Stochastic Optimization
Speaker: Fabian Bastin
Abstract: We consider the framework of nonlinear, unconstrained, smooth optimization, where the trust-region approach has been established as one of the most prominent methods over the last decades. However, significant investigation has been devoted to updating the trust-region radius in recent years, leading to various schemes that exploit the structure of the problem under consideration. In particular, a more intricate link between the trust-region radius and the norm of the gradient at the current iterate has enabled the derivation of upper bounds on the number of iterations required to obtain an ϵ-optimal solution while preserving the main traditional convergence properties. Such results have also been extended to the context of stochastic programming with adaptive sampling to ensure convergence guarantees, with the key ingredient being the ability to achieve a sufficient decrease with high probability. We revisit the progress made regarding the relationship between the trust-region radius update and the gradient norm to establish that, even under these schemes, the trust region ultimately becomes inactive, even when using an adaptive sampling approach in stochastic optimization. This ensures that, under mild conditions, minimizing the model inside the trust region can ultimately yield an approximate quasi-Newton direction. We illustrate these ideas using a trust-region method with adaptive sampling and truncated conjugate gradient on a set of benchmark problems.

Talk 3: Beyond Newton Interpolation: p-Factor Polynomials for Function Approximation in Optimization
Speaker: Olga Brezhneva
Abstract: We introduce a new interpolation method for nonlinear functions over an interval, focusing on cases where the approximated function is non-regular at a solution. In such cases, classical interpolation methods, such as Newton’s interpolation polynomial, do not necessarily provide the required accuracy for approximating solutions to the equation $f(x)=0$. In contrast, our method, based on p-factor interpolation polynomials, ensures the necessary accuracy, leading to improved function approximations in solving nonlinear equations. The presented results are based on the theory of p-regularity. Our approach provides new insights into function approximation techniques in nonlinear optimization. Specifically, it has potential applications in derivative-free optimization, where function evaluations are costly or noisy, as well as in trust-region methods and polynomial-based approaches for solving nonlinear problems.

Speakers
FB

Fabian Bastin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
OB

Olga Brezhneva

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6W: Stochastic Optimization
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Session: Stochastic Optimization
Chair: Honghao Zhang
Cluster: nan

Talk 1: Multi-cut stochastic approximation methods for solving stochastic convex composite optimization
Speaker: Honghao Zhang
Abstract: The development of a multi-cut stochastic approximation (SA) method for solving stochastic convex composite optimization (SCCO) problems has remained an open challenge. The difficulty arises from the fact that the stochastic multi-cut model, constructed as the pointwise maximum of individual stochastic linearizations, provides a biased estimate of the objective function, with the error being uncontrollable. This paper introduces multi-cut SA methods for solving SCCO problems, achieving near-optimal convergence rates. The cutting-plane models used in these methods are the pointwise maxima of appropriately chosen one-cut models. To the best of our knowledge, these are the first multi-cut SA methods specifically designed for SCCO problems.

Talk 2: Data-driven Policies for Two-Stage Stochastic Linear Programs
Speaker: Chhavi Sharma
Abstract: A stochastic program typically involves several parameters including deterministic first-stage parameter, and stochastic second-stage elements that serve as input data. These programs are usually re-solved whenever there is a change in any of the input parameters. For example, a stochastic dispatch problem is solved multiple times a day due to fluctuations in electricity prices, demand, and renewable energy availability. However, in practical situations, quick decision-making is crucial, and solving a stochastic program from scratch for every change in input data can be computationally costly. This work addresses this challenge for two-stage stochastic linear programs (2-SLPs) with varying first-stage right-hand sides. We employ data-driven approaches to first construct a novel piecewise linear difference of max-affine policy (PLDC) for deterministic linear programs. This is achieved by leveraging optimal basis matrices from previous solutions and the piecewise linear nature of the optimal solution trajectory. This PLDC policy retains optimal solutions for previously encountered parameters and is expected to provide good-quality solutions for new parameters. Our developed policy applies directly to the extensive form of 2-SLP. When algorithms such as the L-shaped method are applied to solve 2-SLP, we construct the policy using local outer approximations of the recourse function and optimal basis matrices from previous solves. We assess the performance of our policy through both numerical and theoretical analyses. Our numerical experiments show small feasibility and relative optimal gap at solutions returned by the policy.

Talk 3: Statistical Robustness Analysis of In-CVaR Based Regression
Speaker: Yulei You
Abstract: Based on the interval conditional value-at-risk (In-CVaR) proposed in Liu & Pang (2022), this paper investigates the robustness of In-CVaR based regression under data contamination and perturbation. To quantify robustness under data contamination, we introduce the concept of “distributional breakdown point (BP) value”. Our results provide upper and lower bounds for the distributional BP value, which can be widely applied to classic regression tasks, including linear regression, piecewise affine regression, and feedforward neural network regression with homogeneous activation functions. Furthermore, we demonstrate that under data perturbation, the In-CVaR based estimator is qualitatively robust against optimization if and only if the largest portion of the loss is trimmed. Overall, this research complements the robust analysis of In-CVaR and shows that In-CVaR outperforms conditional value-at-risk and sample average approximation in terms of robustness for regression tasks. This talk is based on a joint work with Prof. Junyi Liu at Tsinghua University.

Speakers
HZ

Honghao Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CS

Chhavi Sharma

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YY

Yulei You

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6X
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 6Y
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Tuesday July 22, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

5:30pm PDT

Break or End of day
Tuesday July 22, 2025 5:30pm - 5:45pm PDT
Tuesday July 22, 2025 5:30pm - 5:45pm PDT
TBA

5:45pm PDT

Poster Session
Tuesday July 22, 2025 5:45pm - 7:15pm PDT
Poster Session

Poster 1: Weighted Data Valuation for Statistical Learning via DC Programming Methods
Presenter: Hassan Nojavan
Abstract: We propose a new formulation of empirical risk minimization that accounts for the weights of data points. We reformulate the problem as difference-of-convex (DC) and bi-convex programs and apply suitable algorithms, including the DC algorithm and Alternate Convex Search (ACS). The proposed methods are applied to regression settings for outlier detection and the top N recommender system problem for data valuation. Our numerical experiments demonstrate that the proposed approaches consistently deliver high-quality solutions, outperforming existing methods used in practice, while effectively identifying data anomalies.

Poster 2: On the Infeasibility of Convex QCQPs
Presenter: Matias Villagra
Abstract: This poster presents preliminary research on fundamental questions regarding the computational complexity of infeasible convex quadratically constrained quadratic programs (QCQPs) within the Turing model of computation.Given an infeasible system $\Sigma$ of convex quadratic inequalities $\{ f_{i}(x) \leq 0, \forall i \in [m] \}$, we say that $\Sigma$ is an Irreducible Inconsistent Subsystem (IIS) if after removing any constraint $f_{j}(x) \leq 0$ from $\Sigma$, the subsystem becomes feasible. Our goal is to understand whether, given an IIS $\Sigma$, we can exhibit a polynomial-sized certificate, on the length of $\Sigma$, which proves that $\Sigma$ is infeasible.A natural way to address this question is to understand the complexity of the minimum infeasibility (MINF) for $\Sigma$, which can be defined as
\begin{equation*}
s^* := \inf \left\{ s \in \R_{+} : f_{i}(x) \leq s, \forall i \in [m] \right\}.
\end{equation*}
The so-called fundamental theorem for convex QCQPs (Terlaky 1985, Luo and Zhang 1999) tells us that if a convex QCQP is bounded below, then the infimum is always attained. But, is MINF well defined under the Turing model of computation? In other words, is the size of $s^*$ always bounded by a polynomial on the bit length of $\Sigma$? We present partial results for this question, along with alternative methods for certifying the infeasibility of $\Sigma$.
This is joint work with Daniel Bienstock.

Poster 3: Enhancing Convergence of Decentralized Gradient Tracking under the KL Property
Presenter: Xiaokai Chen
Abstract: We study decentralized multiagent optimization over networks, modeled as undirected graphs. The optimization problem consists of minimizing a nonconvex smooth function plus a convex extended-value function, which enforces constraints or extra structure on the solution (e.g., sparsity, low-rank). We further assume that the objective function satisfies the Kurdyka-Łojasiewicz (KL) property, with given exponent θ∈[0,1). The KL property is satisfied by several (nonconvex) functions of practical interest, e.g., arising from machine learning applications; in the centralized setting, it permits to achieve strong convergence guarantees. Here we establish convergence of the same type for the notorious decentralized gradient-tracking-based algorithm SONATA. Specifically, (i) when θ∈(0,1/2], the sequence generated by SONATA converges to a stationary solution of the problem at R-linear rate;(ii)when θ∈(1/2,1), sublinear rate is certified; and finally (iii) when θ=0, the iterates will either converge in a finite number of steps or converges at R-linear rate. This matches the convergence behavior of centralized proximal-gradient algorithms except when θ=0. Numerical results validate our theoretical findings.

Poster 4: The Nonconvex Riemannian Proximal Gradient Method
Presenter: Paula J. John
Abstract: We consider a class of nonconvex optimization problems over a Riemannian manifold, where the objective is a sum of a smooth and a possibly nonsmooth function. Our work introduces a new approach to Riemannian adaptations of the proximal gradient method. The algorithm has a straightforward implementation and does not require any computation in the embedding space or solving of subproblems on the tangent space. This is achieved by first performing a gradient step and then applying a proximal operator directly on the manifold. We present numerical examples showing that this method finds applications in different Riemannian optimization problems. This is joint work with Ronny Bergmann, Hajg Jasa, and Max Pfeffer.

Poster 5: A Stochastic Approach to the Subset Selection Problem via Mirror Descent
Presenter: Dan Greenstein
Abstract: The subset selection problem is fundamental in machine learning and other fields of computer science.
We introduce a stochastic formulation for the minimum cost subset selection problem in a black box setting, in which only the subset metric value is available.
Subsequently, we can handle two-stage schemes, with an outer subset-selection component and an inner subset cost evaluation component. We propose formulating the subset selection problem in a stochastic manner by choosing subsets at random from a distribution whose parameters are learned. Two stochastic formulations are proposed.
The first explicitly restricts the subset's cardinality, and the second yields the desired cardinality in expectation.
The distribution is parameterized by a decision variable, which we optimize using Stochastic Mirror Descent.
Our choice of distributions yields constructive closed-form unbiased stochastic gradient formulas and convergence guarantees, including a rate with favorable dependency on the problem parameters.
Empirical evaluation of selecting a subset of layers in transfer learning complements our theoretical findings and demonstrates the potential benefits of our approach.

Poster 6: Directional Smoothness and Gradient Methods: Convergence and Adaptivity
Presenter: Aaron Mishkin
Abstract: We develop new sub-optimality bounds for gradient descent (GD) that depend on the conditioning of the objective along the path of optimization rather than on global, worst-case constants. Key to our proofs is directional smoothness, a measure of gradient variation that we use to develop upper-bounds on the objective. Minimizing these upper-bounds requires solving implicit equations to obtain a sequence of strongly adapted step-sizes; we show that these equations are straightforward to solve for convex quadratics and lead to new guarantees for two classical step-sizes. For general functions, we prove that the Polyak step-size and normalized GD obtain fast, path-dependent rates despite using no knowledge of the directional smoothness. Experiments on logistic regression show our convergence guarantees are tighter than the classical theory based on L-smoothness.

Poster 7: The Convex Riemannian Proximal Gradient Method
Presenter: Hajg Jasa
Abstract: We consider a class of (strongly) geodesically convex optimization problems on Hadamard manifolds, where the objective function splits into the sum of a smooth and a possibly nonsmooth function. We introduce an intrinsic convex Riemannian proximal gradient (CRPG) method that employs the manifold proximal map for the nonsmooth step, without operating in the embedding or in a tangent space. We establish a sublinear convergence rate for convex problems and a linear convergence rate for strongly convex problems, and derive fundamental prox-grad inequalities that generalize the Euclidean case. Our numerical experiments on hyperbolic spaces and manifolds of symmetric positive definite matrices demonstrate substantial computational advantages over existing methods. This is joint work with Ronny Bergmann, Paula J. John, and Max Pfeffer.

Poster 8: DCatalyst: A
Speakers
XC

Xiaokai Chen

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PJ

Paula J. John

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DG

Dan Greenstein

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AM

Aaron Mishkin

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HJ

Hajg Jasa

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TC

Tianyu Cao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HG

Henry Graven

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZZ

Zhiyuan Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JB

Jialu Bao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CC

Can Chen

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CH

Chengpiao Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SH

Smajil Halilovic

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JW

Jie Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
EN

Edward Nguyen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Tuesday July 22, 2025 5:45pm - 7:15pm PDT
Olin Hall of Engineering (OHE) Patio 3650 McClintock Ave, Los Angeles, CA 90089
 
Wednesday, July 23
 

8:30am PDT

Auditorium Opens Doors for seating
Wednesday July 23, 2025 8:30am - 9:00am PDT
Wednesday July 23, 2025 8:30am - 9:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

9:00am PDT

Plenary 3
Wednesday July 23, 2025 9:00am - 10:00am PDT
Speakers
CS

Claudia Sagastizábal

After finishing her undergraduate math studies in Argentina, Claudia moved to Paris where she obtained her PhD and habilitation degrees. Personal reasons caused Claudia to reverse direction over the Atlantic Ocean; she now lives in Rio de Janeiro. Claudia has participated in industrial... Read More →
Wednesday July 23, 2025 9:00am - 10:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

10:00am PDT

Coffee & Snack Break (Provided)
Wednesday July 23, 2025 10:00am - 10:30am PDT
Wednesday July 23, 2025 10:00am - 10:30am PDT
TBA

10:30am PDT

Parallel Sessions 7A: Optimization Methods for Next-Generation Wireless Communication Networks (I)
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Optimization Methods for Next-Generation Wireless Communication Networks (I)
Chair: Ya-Feng Liu
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Exploiting Statistical Hardness for Private Wireless Localization
Speaker: Urbashi Mitra
Abstract: Securing signals from unintended eavesdroppers has become an increasingly important problem in next generation (5G or 6G) wireless communications with the emergence of the Internet-of-Things and Machine-to-Machine communications. Herein, we examine learning problems in signal processing that are inherently hard without key side information. In particular, we exploit necessary resolution limits for classical compressed sensing/sparse approximation problems. To limit an eavesdropper's capabilities, we create an environment for the eavesdropper wherein the appropriate structured statistics algorithm would provably fail. The intended receiver overcomes this ill-posed problem by leveraging a very modest amount of secret side information shared between the intended transmitter and receiver. Two instantiations of private localization are considered, both independent of channel state information and both based on the design of a novel precoder at the transmitter of the device to be localized. In the first case, spurious, virtual multipath is introduced so that the eavesdropper perceives a much more rich channel than the intended user. In the second case, the channel perceived by the eavesdropper has multipath that appears to have been moved from the original, resulting in the eavesdropper learning a spoofed location. Parameter design is enabled through the development of bounds on the estimation error for the eavesdropper. We pose optimization problems whose solutions yield parameter designs for maximal security. Proper parameter design can result in a significant increase (orders of magnitude) in the localization error for the eavesdropper. Theoretical guarantees are provided for all problems considered. All proposed algorithms are validated via numerical results, and it is seen that the eavesdropper’s capabilities are severely degraded.

Talk 2: Reconfigurable Intelligent Surface-aided Wideband Communications
Speaker: Chandra Murthy
Abstract: In this talk, we discuss the beam-split effect in wideband communications aided by a large passive reconfigurable intelligent surface (RIS) equipped with configurable phase shifters. The beam-split is caused by the spatial delay spread across the large aperture of the RIS coupled with its phased array architecture, and results in different frequency components of the wideband signal getting beamformed to different directions. As a result, only a few subcarriers of an orthogonal frequency division multiplexing (OFDM)-based transmission get correctly reflected by the RIS towards a desired user. In turn, this severely degrades the achievable data rate. We present two approaches to alleviate the beam-split effect. The first is a distributed RIS approach where the size of the individual RISs is chosen to ensure that the effect of beam-split at each RIS remains within a tolerance level. The second approach exploits beam-split instead of controlling it, by opportunistically allotting non-overlapping sub-bands of the wide bandwidth to different users for a given RIS phase configuration, thereby leveraging multi-user diversity. The former case provides near-optimal RIS benefits for a given user but requires careful RIS placement to control the temporal delay spread introduced by the multiple RISs. The latter case provides near-optimal benefits in terms of the network throughput but requires many users in the system. We contrast the two approaches via numerical simulations. This is joint work with Yashvanth L.

Talk 3: A Connection Between Schur Complement and Quadratic Transform for Fractional Programming
Speaker: Kaiming Shen
Abstract: Fractional programming (FP) is an invaluable tool for communications, signal processing, and machine learning, because many practical optimization problems are fractionally structured, e.g., the signal-to-interference-plus-noise ratio maximization for wireless transmission, the normalized cut maximization for graph clustering, the Cram\'{e}r-Rao bound minimization for radar signal processing. The state-of-the-art method for FP, called quadratic transform, significantly extends the classical Dinkelbach’s algorithm in that it is capable of handling multiple-ratio and matrix-ratio optimizations. This talk shows that the quadratic transform has an intimate connection to the Schur complement technique in characterizing the positive semidefiniteness of square matrices. Although the quadratic transform and the Schur complement seem to be completely distinct from each other both in their motivation and mathematical form, it turns out that the quadratic transform can be derived from Schur complement under certain conditions related to a minimax theorem. We show an application of this connection between Schur complement and FP in a wireless integrated sensing and communications problem.

Speakers
YL

Ya-Feng Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
UM

Urbashi Mitra

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Chandra Murthy

Chandra Murthy

Name: Chandra R. MurthyTitle: ProfessorAffiliation:Indian Institute of ScienceChandra R. Murthy received the B. Tech degree in Electrical Engineering from the Indian Institute of Technology, Madras, Chennai, India in 1998 and the M.S. degree in Electrical and Computer Engineering... Read More →
KS

Kaiming Shen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7B: Relaxations for non-convex optimization
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Relaxations for non-convex optimization
Chair: Andres Gomez
Cluster: Global Optimization

Talk 1: Convex Reformulations and Approximation Bounds for Low-Rank Semidefinite Programs
Speaker: Soroosh Shafiee
Abstract: Low-rank optimization has found numerous applications in finance, machine learning, and statistics. We develop convex relaxations for these problems in lifted spaces, leveraging perspective functions and majorization operators. These relaxations are shown to be provably stronger than existing approaches, such as those based on the nuclear norm. Additionally, we demonstrate that low-rank optimization problems in low-dimensional spaces typically exhibit a small duality gap, emphasizing the effectiveness and tightness of the relaxation

Talk 2: Multi-period mixed-integer quadratic programming
Speaker: Jisun Lee
Abstract: In this talk, we consider multi-period convex quadratic optimization problems with indicator variables, where the state linearly evolves from its current state subject to control inputs. This problem class has important applications in hybrid control and statistical learning. We give a compact convex hull description in an extended space with linear and conic quadratic inequalities for the uncapacited case. We also propose a polynomial-time algorithm. Computational experiments with data from neuron activation inference and hybrid-electric vehicle power management indicate promises and challenges.

Talk 3: TBD
Speaker: Chen Chen
Abstract: TBD

Speakers
AG

Andres Gomez

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SS

Soroosh Shafiee

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jisun Lee

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CC

Chen Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7C: Duality in Optimization for Data Science
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Duality in Optimization for Data Science
Chair: Ahmet Alacaoglu
Cluster: Optimization For Data Science

Talk 1: Addressing Misspecification in Contextual Optimization
Speaker: Jiawei Zhang
Abstract: We study a linear contextual optimization problem where a decision maker has access to historical data and contextual features to learn a cost prediction model aimed at minimizing decision error. We adopt the predict-then-optimize framework for this analysis. Given that perfect model alignment with reality is often unrealistic in practice, we focus on scenarios where the chosen hypothesis set is misspecified. In this context, it remains unclear whether current contextual optimization approaches can effectively address such model misspecification. In this paper, we present a novel integrated learning and optimization approach designed to tackle model misspecification in contextual optimization. This approach offers theoretical generalizability, tractability, and optimality guarantees, along with strong practical performance. Our method involves minimizing a tractable surrogate loss that aligns with the performance value from cost vector predictions, regardless of whether the model misspecified or not, and can be optimized in reasonable time. To our knowledge, no previous work has provided an approach with such guarantees in the context of model misspecification.

Talk 2: Density Estimation from Moments
Speaker: Michael Friedlander
Abstract: We present a maximum entropy method for estimating probability densities from a limited set of moment measurements, with applications to x-ray Thomson scattering in high-energy physics. A stable dual formulation using indirect linear algebra operations yields robust density estimates.

Talk 3: A Dual-Certificate Analysis for Neural Network Optimization Problems
Speaker: Rahul Parhi
Abstract: We consider the problem of optimizing neural networks with common regularization schemes such as weight decay (which corresponds to the standard Tikhonov regularization). In the case of shallow neural networks, it turns out that this non-convex optimization problem can be lifted to a convex optimization problem posed over a a space of Radon measures. This enables us to bring tools from convex analysis to study properties of solutions to these problems. Via a novel dual-certificate analysis of the lifted problem for multivariate and vector-valued neural networks, we prove that solutions to the original non-convex problem are always unique. These unique neural network solutions also have widths (number of neurons) bounded by the number of training data squared, regardless of the level of overparameterization. This result recovers recent observations in the literature that were made only in the univariate case. Furthermore, this result also sheds light on the "critical level" of overparameterization necessary for neural networks.

Speakers
AA

Ahmet Alacaoglu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JZ

Jiawei Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MF

Michael Friedlander

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RP

Rahul Parhi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7D: Service or Scientific Contribution? Promoting and Recognizing Best Review Practices
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Service or Scientific Contribution? Promoting and Recognizing Best Review Practices
Chair: Katya Scheinberg
Cluster: nan

Talk 1: Service or Scientific Contribution? Promoting and Recognizing Best Review Practices
Speaker: Katya Scheinberg
Abstract: TBD

Talk 2: Service or Scientific Contribution? Promoting and Recognizing Best Review Practices
Speaker: Frank E. Curtis
Abstract: TBD

Speakers
KS

Katya Scheinberg

Professor, Georgia Institute of Technology
Name:Katya Scheinberg Title: Coca-Cola Foundation Chair and ProfessorAffiliation: H. Milton Stewart School of Industrial and Systems Engineering  Georgia Institute of Technology, Atlanta, GABio:Katya Scheinberg is a Coca-Cola Foundation Chair and Professor in the H. Milton Stewart... Read More →
avatar for Frank E. Curtis

Frank E. Curtis

Professor, Lehigh University
Name: Frank E. Curtis, Ph.D.Title: ProfessorAffiliation: Lehigh UniversityBio: Please see my website.Fun Fact: My wife and I have been together for 20 years and she's never seen me without a beard.
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7E: Robust optimization for sequential decision-making
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Robust optimization for sequential decision-making
Chair: Julien Grand-Clément
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Statistical Learning of Distributionally Robust Stochastic Control in Continuous State Spaces
Speaker: Shengbo Wang
Abstract: We explore the control of stochastic systems with potentially continuous state and action spaces, characterized by the state dynamics $X_{t+1} = f(X_t, A_t, W_t)$. Here, $X$, $A$, and $W$ represent the state, action, and exogenous random noise processes, respectively, with $f$ denoting a known function that describes state transitions. Traditionally, the noise process $\set{W_t, t \geq 0}$ is assumed to be independent and identically distributed, with a distribution that is either fully known or can be consistently estimated. However, the occurrence of distributional shifts, typical in engineering settings, necessitates the consideration of the robustness of the policy. This paper introduces a distributionally robust stochastic control paradigm that accommodates possibly adaptive adversarial perturbation to the noise distribution within a prescribed ambiguity set. We examine two adversary models: current-action-aware and current-action-unaware, leading to different dynamic programming equations. Furthermore, we characterize the optimal finite sample minimax rates for achieving uniform learning of the robust value function across continuum states under both adversary types, considering ambiguity sets defined by $f_k$-divergence and Wasserstein distance. Finally, we demonstrate the applicability of our framework across various real-world settings.

Talk 2: Robust Regret Minimization in Bayesian Offline Bandits
Speaker: Marek Petrik
Abstract: We study how to make decisions that minimize Bayesian regret in offline linear bandits. Prior work suggests that one must take actions with maximum lower confidence bound (LCB) on their reward. We argue that the reliance on LCB is inherently flawed in this setting and propose a new algorithm that directly minimizes upper bounds on the Bayesian regret using efficient conic optimization solvers. Our bounds build heavily on new connections to monetary risk measures. Proving a matching lower bound, we show that our upper bounds are tight, and by minimizing them we are guaranteed to outperform the LCB approach. Our numerical results on synthetic domains confirm that our approach is superior to LCB.

Talk 3: On the interplay between average and discounted optimality in robust Markov decision processes
Speaker: Julien Grand-Clément
Abstract: We introduce the Blackwell discount factor for Markov Decision Processes (MDPs). Classical objectives for MDPs include discounted, average, and Blackwell optimality. Many existing approaches to computing average-optimal policies solve for discount-optimal policies with a discount factor close to 1, but they only work under strong or hard-to-verify assumptions on the MDP structure such as unichain or ergodicity. We are the first to highlight the shortcomings of the classical definition of Blackwell optimality, which does not lead to simple algorithms for computing Blackwell-optimal policies and overlooks the pathological behaviors of optimal policies as regards the discount factors. To resolve this issue, in this paper, we show that when the discount factor is larger than the Blackwell discount factor, all discount-optimal policies become Blackwell-and average-optimal, and we derive a general upper bound on the Blackwell discount factor. Our upper bound on the Blackwell discount factor, parametrized by the bit-size of the rewards and transition probabilities of the MDP instance, provides the first reduction from average and Blackwell optimality to discounted optimality, without any assumptions, along with new polynomial-time algorithms. Our work brings new ideas from polynomials and algebraic numbers to the analysis of MDPs. Our results also apply to robust MDPs, enabling the first algorithms to compute robust Blackwell-optimal policies.

Speakers
SW

Shengbo Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MP

Marek Petrik

Name: Marek PetrikTitle: Associate ProfessorAffiliation: University of New HampshireBio:Marek Petrik is an associate professor of Computer Science at the University of New Hampshire. Until 2016, he was a research staff member at the Mathematical Sciences Department of IBM’s T. J... Read More →
JG

Julien Grand-Clément

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7F: GPU-Accelerated Mathematical Programming (Part II)
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: GPU-Accelerated Mathematical Programming (Part II)
Chair: Haihao Lu
Cluster: Computational Software

Talk 1: Recovering sparse DFT on missing signals via interior point method on GPU
Speaker: Alexis Montoison
Abstract: We present a method for recovering a sparse Discrete Fourier Transform (DFT) of a signal that is noisy and potentially incomplete (i.e., containing missing values). The problem is formulated as a penalized least-squares optimization based on the Inverse Discrete Fourier Transform (IDFT) with an $l_1$-penalty term. By transforming the $l_1$-norm into elastic constraints, we make the problem suitable for an interior point method (IPM) approach. Although Krylov methods are not typically used to solve KKT systems arising in IPM due to the ill-conditioning of these systems, we derive a tailored preconditioner to address this issue. Thanks to this dedicated preconditioner and the fact that FFT and IFFT act as linear operators without requiring the explicit materialization of the underlying matrices, KKT systems can be solved efficiently at large scales in a matrix-free manner. Numerical results from a Julia implementation leveraging Krylov.jl, MadNLP.jl, and GPU-based FFT toolkits such as cuFFT and rocFFT demonstrate the scalability of our approach on problems with millions of variables.

Talk 2: MPAX: Mathematical Programming in JAX
Speaker: Zedong Peng
Abstract: We introduce MPAX (Mathematical Programming in JAX), a versatile and efficient toolbox for integrating mathematical programming into machine learning workflows. MPAX implemented the state-of-the-art first-order methods, restarted average primal-dual hybrid gradient and reflected restarted Halpern primal-dual hybrid gradient, to solve linear programming, quadratic programming and optimal transport problems in JAX. It provides native support for hardware accelerations along with features like batch solving, auto-differentiation, and device parallelism. Extensive numerical experiments demonstrate the advantages of MPAX over existing solvers. The solver is available at https://github.com/MIT-Lu-Lab/MPAX.

Talk 3: Accelerating Low-Rank Factorization-Based Semidefinite Programming Algorithms on GPU
Speaker: Qiushi Han
Abstract: In this paper, we address a long-standing challenge: how to achieve both efficiency and scalability in solving semidefinite programming problems. We propose acceleration techniques for a wide range of low-rank factorization-based first-order methods using GPUs, making the computation much more efficient and scalable. To illustrate the idea and effectiveness of our approach, we use the low-rank factorization-based SDP solver, LoRADS, as an example, which involves both the classic Burer-Monterio method and a novel splitting scheme with a starting logarithmic rank. Our numerical results demonstrate that the accelerated GPU version of LoRADS, cuLoRADS, can solve huge-scale semidefinite programming problems with remarkable efficiency. By effectively leveraging GPU computational power, cuLoRADS exhibits outstanding performance. Specifically, it can solve a set of MaxCut problems with 10^7 ×10^7 matrix variables in 10 seconds to 1 minute each on an NVIDIA H100 GPU with 80GB memory, whereas previous solvers demonstrated the capability of handling problems of this scale, required at least dozens of hours per problem on CPUs. Additionally, cuLoRADS shows exceptional scalability by solving 1) a MaxCut problem with a 170 million × 170 million matrix variable and 2) a Matrix Completion problem with a 20 million × 20 million matrix variable and approximately 200 million constraints, both in a matter of minutes.

Speakers
HL

Haihao Lu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AM

Alexis Montoison

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZP

Zedong Peng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7G: Statistical and Computational Aspects of Multi-Agent Games
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Statistical and Computational Aspects of Multi-Agent Games
Chair: Zhuoran Yang
Cluster: Multi-agent Optimization and Games

Talk 1: Partially Observable Multi-Agent Reinforcement Learning with Information Sharing
Speaker: Kaiqing Zhang
Abstract: We study provable multi-agent reinforcement learning (RL) in the general framework of partially observable stochastic games (POSGs). To circumvent the known hardness results and the use of computationally intractable oracles, we advocate leveraging the potential information-sharing among agents, a common practice in empirical multi-agent RL, and a standard model for multiagent control systems with communications. We first establish several computational complexity results to justify the necessity of information-sharing, as well as the observability assumption that has enabled quasi-efficient single-agent RL with partial observations, for efficiently solving POSGs. Inspired by the inefficiency of planning in the ground-truth model, we then propose to further approximate the shared common information to construct an approximate model of the POSG, in which planning an approximate equilibrium (in terms of solving the original POSG) can be quasi-efficient, i.e., of quasi-polynomial-time, under the aforementioned assumptions. Furthermore, we develop a partially observable multi-agent RL algorithm that is both statistically and computationally quasi-efficient. Finally, beyond equilibrium learning, we extend our algorithmic framework to finding the team-optimal solution in cooperative POSGs, i.e., decentralized partially observable Markov decision processes, a much more challenging goal. We establish concrete computational and sample complexities under several common structural assumptions of the model. We hope our study could open up the possibilities of leveraging and even designing different information structures, a well-studied notion in control theory, for developing both sample- and computation-efficient partially observable multi-agent RL.

Talk 2: Faster Rates for No-Regret Learning in General Games via Cautious Optimism
Speaker: Ashkan Soleymani
Abstract: Uncoupled learning dynamics for multiagent interactions (“games”) define iterative update rules that each agent can apply repeatedly to improve their strategy. A celebrated result establishes that for several choices of learning dynamics, global notions of equilibrium in the system will arise. This connection runs deep and is far from trivial: equilibrium emerges even despite formally chaotic behavior. Today, learning dynamics are the most scalable technique for equilibrium computation in large-scale, general games.

Talk 3: Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithms
Speaker: Weiqiang Zheng
Abstract: Self-play via online learning is one of the premier ways to solve large-scale two-player zero-sum games, both in theory and practice. Particularly popular algorithms include optimistic multiplicative weights update (OMWU) and optimistic gradient-descent-ascent (OGDA). While both algorithms enjoy $O(1/T)$ ergodic convergence to Nash equilibrium in two-player zero-sum games, OMWU offers several advantages including logarithmic dependence on the size of the payoff matrix and $\Tilde{O}(1/T)$ convergence to coarse correlated equilibria even in general-sum games. However, in terms of last-iterate convergence in two-player zero-sum games, an increasingly popular topic in this area, OGDA guarantees that the duality gap shrinks at a rate of $(1/\sqrt{T})$, while the best existing last-iterate convergence for OMWU depends on some game-dependent constant that could be arbitrarily large. This begs the question: **is this potentially slow last-iterate convergence an inherent disadvantage of OMWU, or is the current analysis too loose?** Somewhat surprisingly, we show that the former is true. More generally, we prove that a broad class of algorithms that do not forget the past quickly all suffer the same issue: for any arbitrarily small $\delta>0$, there exists a $2\times 2$ matrix game such that the algorithm admits a constant duality gap even after $1/\delta$ rounds. This class of algorithms includes OMWU and other standard optimistic follow-the-regularized-leader algorithms.

Speakers
KZ

Kaiqing Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Weiqiang Zheng

Weiqiang Zheng

PhD candidate, Yale University
Weiqiang Zheng is a fourth-year computer science PhD student at Yale University, advised by Prof. Yang Cai. He received his bachelor's degree from Turing Class, Peking University. His has a broad interest in algorithmic game theory, online learning, and optimization. His recent research... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7H: Recent advances in optimization and statistical estimation on manifolds
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Recent advances in optimization and statistical estimation on manifolds
Chair: Krishna Balasubramanian
Cluster: Optimization on Manifolds

Talk 1: Riemannian Coordinate Descent Algorithms on Matrix Manifolds
Speaker: Yunrui Guan
Abstract: Many machine learning applications are naturally formulated as optimization problems on Riemannian manifolds. The main idea behind Riemannian optimization is to maintain the feasibility of the variables while moving along a descent direction on the manifold. This results in updating all the variables at every iteration. In this work, we provide a general framework for developing computationally efficient coordinate descent (CD) algorithms on matrix manifolds that allows updating only a few variables at every iteration while adhering to the manifold constraint. In particular, we propose CD algorithms for various manifolds such as Stiefel, Grassmann, (generalized) hyperbolic, symplectic, and symmetric positive (semi)definite. While the cost per iteration of the proposed CD algorithms is low, we further develop a more efficient variant via a first-order approximation of the objective function. We analyze their convergence and complexity, and empirically illustrate their efficacy in several applications.

Talk 2: Online covariance estimation for stochastic gradient descent under non-smoothness
Speaker: Abhishek Roy
Abstract: We investigate the online overlapping batch-means covariance estimator for Stochastic Gradient Descent (SGD) under non-smoothness and establish convergence rates. Our analysis overcomes significant challenges that arise due to non-smoothness, leading to the introduction of additional error terms and handling manifold structures in the solution path. Moreover, we establish the convergence rate for the first four moments of the $\ell_2$ norm of the error of SGD dynamics under non-smoothness which holds potential interest as an independent result. Numerical simulations are provided to illustrate the practical performance of the proposed methodology.

Talk 3: Momentum Stiefel optimizer, with applications to suitably-orthogonal attention, and optimal transport
Speaker: Molei Tao
Abstract: The problem of optimization on Stiefel manifold, i.e., minimizing functions of (not necessarily square) matrices that satisfy orthogonality constraints, has been extensively studied. Yet, a new approach is proposed based on, for the first time, an interplay between thoughtfully designed continuous and discrete dynamics. It leads to a gradient-based optimizer with intrinsically added momentum. This method exactly preserves the manifold structure but does not require additional operation to keep momentum in the changing (co)tangent space, and thus has low computational cost and pleasant accuracy. Its generalization to adaptive learning rates is also demonstrated. Notable performances are observed in practical tasks. For instance, we found that placing orthogonal constraints on attention heads of trained-from-scratch Vision Transformer (Dosovitskiy et al., 2020) could markedly improve its performance, when our optimizer is used, and it is better that each head is made orthogonal within itself but not necessarily to other heads. This optimizer also makes the useful notion of Projection Robust Wasserstein Distance (Paty and Cuturi, 2019; Lin et al., 2020) for high-dim. optimal transport even more effective.

Speakers
AR

Abhishek Roy

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MT

Molei Tao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7I: Variational Analysis: Theory and Applications IV
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Variational Analysis: Theory and Applications IV
Chair: Walaa Moursi
Cluster: Nonsmooth Optimization

Talk 1: TBA
Speaker: Jonathan Eckstein
Abstract: TBA

Talk 2: TBA
Speaker: Walaa Moursi
Abstract: TBA

Talk 3: TBA
Speaker: Shambhavi Singh
Abstract: TBA

Speakers
JE

Jonathan Eckstein

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WM

Walaa Moursi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SS

Shambhavi Singh

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7J: Discrete Structures in Nonlinear Optimization
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Discrete Structures in Nonlinear Optimization
Chair: Qimeng / Kim Yu
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Lower bounds for binary polynomial optimization via signed certificates
Speaker: Liding Xu
Abstract: We consider the binary polynomial optimization (BPO) problem of minimizing a polynomial $f$ over the binary hypercube. The Lasserre and Sherali-Adams hierarchies are well-known successive convex relaxations of BPO. These relaxations are based on certificates for binary non-negativity of polynomials. Developing sparse-aware non-negativity certificates is vital for the scalability of such hierarchical-style-like relaxations, since the sizes of relaxations can blow up combinatorially w.r.t. the level of the relaxation. In this talk, we present a novel approach to constructing binary non-negativity certificates for polynomials and a new class of relaxations. We start with a subclass of binary polynomials with a specific signed support pattern called \emph{nonlinearly negatively signed (NNS) polynomials}. It is easy to identify them, as their nonlinear monomials all have negative coefficients (their linear parts can have arbitrary signs). We show that, for the set of NNS polynomials with $m$ monomials of degree $d$, their binary non-negativity can be checked in a $\bO(m^2d)$ time via minimum cut algorithms, and we construct a linear programming representation for this set through the min-cut-max-flow duality. We categorize binary polynomials based on their signed support patterns. For an arbitrary $n$-variate polynomial, we can decompose it as the sum of an NNS polynomial and a positively signed (PS) polynomial (that only contains nonlinear monomials with positive coefficients). Then, we develop parameterized linear programming representations of binary non-negative polynomials. This allows constructing binary no n-negative signed certificates with adjustable signed support patterns and representation complexities. Because the size of the LP reformulation may be huge, we further propose a refined signed support decomposition to decompose this polynomial as binary non-negativity certificates with simpler signed support patterns and lower representation complexities. This method yields new hierarchies of linear programming relaxations for binary polynomial optimization. The hierarchies of LP relaxations for the BPO that converge in at most $\ceil{\log(n)}$ steps, and each level of the relaxations can be solved in a time doubly exponential in its level number $i \le \ceil{\log(n)}$ (hence exponential in $n$). Moreover, since our decomposition only depends on the support of $f$, the new hierarchies are sparsity-preserving. We also provide preliminary computational experiments on comparing the proposed relaxations with SDP relaxations for max cut problems. This is a joint work with Leo Liberti.

Talk 2: A Parametric Approach for Solving Convex Quadratic Optimization with Indicators Over Trees
Speaker: Aaresh Bhathena
Abstract: We investigate convex quadratic optimization problems involving $n$ indicator variables, each associated with a continuous variable, particularly focusing on scenarios where the matrix $Q$ defining the quadratic term is positive definite and its sparsity pattern corresponds to the adjacency matrix of a tree graph. We introduce a graph-based dynamic programming algorithm that solves this problem in time and memory complexity of $\mathcal{O}(n^2)$. Central to our algorithm is a precise parametric characterization of the cost function across various nodes of the graph corresponding to distinct variables. Our computational experiments conducted on both synthetic and real-world datasets demonstrate the superior performance of our proposed algorithm compared to existing algorithms and state-of-the-art mixed-integer optimization solvers. An important application of our algorithm is in the real-time inference of Gaussian hidden Markov models from data affected by outlier noise. Using a real on-body accelerometer dataset, we solve instances of this problem with over 30,000 variables in under a minute, and its online variant within milliseconds on a standard computer.

Talk 3: The Augmented Factorization Bound for Maximum-Entropy Sampling
Speaker: Yongchun Li
Abstract: The maximum-entropy sampling problem (MESP) aims to select the most informative principal submatrix of a pre- specified size from a given covariance matrix. This paper proposes an augmented factorization bound for MESP based on concave relaxation. By leveraging majorization and Schur-concavity theory, we demonstrate that this new bound dominates the classic factorization bound of Nikolov (2015) and a recent upper bound proposed by Li et al. (2024). Furthermore, we provide theoretical guarantees that quantify how much our proposed bound improves the two existing ones and establish sufficient conditions for when the improvement is strictly attained. These results allow us to refine the celebrated approximation bounds for the two approximation algorithms of MESP. Besides, motivated by the strength of this new bound, we develop a variable fixing logic for MESP from a primal perspective. Finally, our numerical experiments demonstrate that our proposed bound achieves smaller integrality gaps and fixes more variables than the tightest bounds in the MESP literature on most benchmark instances, with the improvement being particularly significant when the condition number of the covariance matrix is small.

Speakers
LX

Liding Xu

Postdoc researcher
Name: Dr. Liding XuTitle: Postdoc researcherAffiliation: IOL group at Zuse-Institut BerlinBio:
AB

Aaresh Bhathena

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Yongchun Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7K: Stability and Robustness in Statistical Learning
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Stability and Robustness in Statistical Learning
Chair: Louis Chen
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: A Stability Principle for Learning under Non-Stationarity
Speaker: Kaizheng Wang
Abstract: We develop a versatile framework for statistical learning in non-stationary environments. In each time period, our approach applies a stability principle to select a look-back window that maximizes the utilization of historical data while keeping the cumulative bias within an acceptable range relative to the stochastic error. Our theory showcases the adaptability of this approach to unknown non-stationarity. The regret bound is minimax optimal up to logarithmic factors when the population losses are strongly convex, or Lipschitz only. At the heart of our analysis lie two novel components: a measure of similarity between functions and a segmentation technique for dividing the non-stationary data sequence into quasi-stationary pieces. The talk is based on joint work with Chengpiao Huang.

Talk 2: On the Adversarial Robustness of Benjamini Hochberg
Speaker: Louis Chen
Abstract: The Benjamini-Hochberg (BH) procedure is widely used to control the false detection rate (FDR) in multiple testing. Applications of this control abound in drug discovery, forensics, anomaly detection, and, in particular, machine learning, ranging from nonparametric outlier detection to out-of-distribution detection and one-class classification methods. Considering this control's place in critical safety/security contexts, we investigate its adversarial robustness. More precisely, we study under what conditions BH does and does not exhibit adversarial robustness, we present a class of simple and easily implementable adversarial test-perturbation algorithms, and we perform computational experiments. With our algorithms, we demonstrate that there are conditions under which BH's control can be significantly broken with relatively few (even just one) test score perturbation(s), and provide non-asymptotic guarantees on the expected adversarial-adjustment to FDR. Our technical analysis involves a combinatorial reframing of the BH procedure as a ``balls into bins'' process, and drawing a connection to generalized ballot problems to facilitate an information-theoretic approach for deriving non-asymptotic lower bounds.

Talk 3: Estimating the Direction-of-Arrival of a Signal Under Impulsive Noise
Speaker: Robert Bassett
Abstract: We consider the problem of estimating a signal subspace in the presence of interference that contaminates some proportion of the received observations. Our emphasis is on detecting the contaminated observations so that the signal subspace can be estimated with the contaminated observations discarded. We employ a signal model which explicitly includes an interference term that is distinct from environmental noise. To detect when the interference term is nonzero, we estimate the interference term using an optimization problem with a sparsity-inducing group SLOPE penalty which accounts for simultaneous sparsity across all channels of the multichannel signal. We propose an iterative algorithm which efficiently computes the observations estimated to contain interference. Theoretical support for the accuracy of our interference estimator is provided by bounding its false discovery rate. Finally, we demonstrate the empirical performance of our contributions in a number of simulated experiments.

Speakers
KW

Kaizheng Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LC

Louis Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RB

Robert Bassett

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7L: optimization with variational inequality constraints
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: optimization with variational inequality constraints
Chair: Jin Zhang
Cluster: Fixed Points and Variational Inequalities

Talk 1: A Smoothing Implicit Gradient Algorithm for Optimization with Parametric Variational Inequality Constraints
Speaker: Jin Zhang
Abstract: We propose a smoothing implicit gradient algorithm for optimization with Parametric Variational Inequality (PVI) constraints on a moving polyhedron. We approximate the nonsmooth PVI constraints by smoothing constraints and establish relation between the smoothing problem and the original problem regarding their global solutions and first-order stationary points. Moreover, we introduce a penalty function and auxiliary variables to solve the smoothing problem by the implicit gradient algorithm with an updating scheme of smoothing parameter. We prove that the algorithm converges to a stationary point of the original problem. Numerical experiments are conducted to validate the efficiency of the proposed algorithm.

Talk 2: On resolution of large-scale optimization
Speaker: Xue Xie
Abstract: In this talk, I will discuss the resolution of optimal transport and general nonconvex optimization with expected-valued objective functions in large-scale. For the OT problem, we introduced the random block coordinate descent (RBCD) methods to directly solve the linear programming (LP) problem motivated by optimal transport (OT). Our approach restricts the potentially large-scale LP to small LP subproblems constructed via randomly chosen working sets. We equip the vanilla version of RBCD with almost sure convergence and a linear convergence rate. To further improve the efficiency, we explore the special structure of constraints in OT and refine the random working set selection. Preliminary numerical experiments demonstrate that the accelerated RBCD compares well with other solvers and offers the advantage of saving memory. The second problem is complicated by nonconvex expected-valued objective functions. Such a formulation can capture deep learning, policy optimization, autoencoder training, etc. It is known in literature that the sample complexity of stochastic first-order methods depend linearly on the problem dimension, which is undesirable for solving large-scale problems. To address this issue, we propose dimension insensitive stochastic first-order methods. Our algorithm allow for non-Euclidean and nonsmooth distance functions as the proximal terms when taking the stochastic gradient step. State-of-art sample complexity guarantees in terms of the dimension are shown.

Talk 3: Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions
Speaker: Wei Yao
Abstract: Constrained bilevel optimization tackles nested structures present in constrained learning tasks like constrained meta-learning, adversarial learning, and distributed bilevel optimization. However, existing bilevel optimization methods mostly are typically restricted to specific constraint settings, such as linear lower-level constraints. In this work, we overcome this limitation and develop a new single-loop, Hessian-free constrained bilevel algorithm capable of handling more general lower-level constraints. We achieve this by employing a doubly regularized gap function tailored to the constrained lower-level problem, transforming constrained bilevel optimization into an equivalent single-level optimization problem with a single smooth constraint. We rigorously establish the non-asymptotic convergence analysis of the proposed algorithm under the convexity of lower-level problem, avoiding the need for strong convexity assumptions on the lower-level objective or coupling convexity assumptions on lower-level constraints found in existing literature. Additionally, the generality of our method allows for its extension to bilevel optimization with minimax lower-level problem. We evaluate the effectiveness and efficiency of our algorithm on various synthetic problems, typical hyperparameter learning tasks, and generative adversarial network.

Speakers
JZ

Jin Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XX

Xue Xie

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WY

Wei Yao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7M: Nonsmooth PDE Constrained Optimization: Algorithms, Analysis and Applications Part 1
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Nonsmooth PDE Constrained Optimization: Algorithms, Analysis and Applications Part 1
Chair: Denis Ridzal
Cluster: PDE-constrained Optimization

Talk 1: Digital Twins and Optimization Under Uncertainty 
Speaker: Harbir Antil
Abstract: This talk begins by studying the role of risk measures, such as Conditional Value at Risk (CVaR), in identifying weaknesses in Structural Digital Twins. CVaR is shown to outperform classical expectation (risk-neutral setting) for such problems. Nevertheless, this framework assumes a knowledge of the underlying distribution. To overcome such a requirement, we introduce the notion of Rockafellian relaxation which can handle realistic distributional ambiguities. Both, risk-neutral and risk-averse formulations are discussed. Applications to real life digital twins of bridges, dams, and wind turbines are considered. Time permitting, both the static and dynamic problems arising in civil and mechanical engineering will be presented.

Talk 2: Infinite-horizon optimal control of operator equations with random inputs
Speaker: Olena Melnikov
Abstract: We investigate infinite-horizon discounted optimal control problems governed by operator equations with random inputs. Our framework includes parameterized evolution equations, such as those arising from ordinary and partial differential equations. The objective function is risk-neutral, aiming to optimize the expected discounted cost over an infinite time horizon. We establish the existence of optimal solutions. Furthermore, we discuss the convergence of sample-based approximations, demonstrating their effectiveness in approximating the true problem.

Talk 3: Nonuniform derivative-based random weight initialization for neural network optimization
Speaker: Konstantin Pieper
Abstract: Neural networks can alleviate the curse of dimensionality by detecting subspaces in the input data corresponding to large output variability. In order to exploit this, the nonlinear input weights of the network have to align with these directions during network training. As a step on the way to guess these patterns before nonlinear optimization-based neural network regression, we propose nonuniform data-driven parameter distributions for weight initialization. These parameter distributions are developed in the context of non-parametric regression models based on shallow neural networks and employ derivative data of the function to be approximated. We use recent results on the harmonic analysis and sparse representation of fully trained (optimal) neural networks to obtain densities that concentrate in appropriate regions of the input weight space. Then, we suggest simplifications of these exact densities based on approximate derivative data in the input points that allow for very efficient sampling. This leads to performance of random feature models close to optimal networks in several scenarios and compares favorably to conventional uniform random feature models.

Speakers
DR

Denis Ridzal

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HA

Harbir Antil

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
OM

Olena Melnikov

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KP

Konstantin Pieper

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7N: Derivative-free stochastic optimization methods II
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Derivative-free stochastic optimization methods II
Chair: Matt Menickelly
Cluster: Derivative-free Optimization

Talk 1: Augmenting subspace optimization methods with linear bandits
Speaker: Matt Menickelly
Abstract: In recent years, there has been a growing interest in developing iterative optimization methods that focus on finding descent restricted to affine subspaces containing an incumbent. Each iteration typically consists of choosing a subspace according to some possibly randomized technique, and then building and minimizing a local model employing derivatives of the function projected onto the chosen subspace. In model-based derivative-free optimization, where gradient approximations essentially require a finite difference (i.e., a number of function evaluations linear in problem dimension), these methods suggest serious opportunities for practical gains in efficiency, since the number of function evaluations necessary to obtain a projected gradient approximation instead scales with the chosen subspace dimension. Motivated by practicality, we consider a simple augmentation of such a generic subspace method. In particular, we consider a sequential optimization framework where actions consist of one-dimensional linear subspaces and rewards consist of projected gradient measurements made along corresponding affine subspaces. This sequential optimization problem can be analyzed through the lens of dynamic regret. We modify an existing upper confidence bound (UCB) linear bandit method to achieve sublinear dynamic regret. We demonstrate the efficacy of employing this UCB method alongside a sketched version of the derivative-free optimization method, POUNDers.

Talk 2: Adaptive sampling trust region optimization with randomized subpaces
Speaker: Sara Shashaani
Abstract: The globally convergent ASTRO-DF algorithm was established to make stochastic derivative-free optimization efficient with careful sampling strategy of generating only sufficiently many random outputs in every function evaluation. Despite theoretical guarantees for sample complexity, this algorithm still suffers curse of dimensionality. The new variant of this algorithm uses randomized subspaces in combination with adaptive sampling to address this shortcoming. This variant, therefore, has the ability to be run for high-dimensional derivative-free problems while maintaining optimal sample-efficiency.

Talk 3: Zeroth-order Riemannian averaging stochastic approximation algorithms
Speaker: Krishnakumar Balasubramanian
Abstract: We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds. We show that \texttt{Zo-RASA} achieves optimal sample complexities for generating -approximation first-order stationary solutions using only one-sample or constant-order batches in each iteration. Our approach employs Riemannian moving-average stochastic gradient estimators, and a novel Riemannian-Lyapunov analysis technique for convergence analysis. We improve the algorithm's practicality by using retractions and vector transport, instead of exponential mappings and parallel transports, thereby reducing per-iteration complexity. Additionally, we introduce a novel geometric condition, satisfied by manifolds with bounded second fundamental form, which enables new error bounds for approximating parallel transport with vector transport.

Speakers
avatar for Matt Menickelly

Matt Menickelly

Hi! I'm Matt Menickelly, and I'm a computational mathematician at Argonne National Laboratory, part of the US Department of Energy's network of national laboratories. I broadly characterize my research as being in "mathematical optimization", and I particularly focus on expensive... Read More →
SS

Sara Shashaani

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KB

Krishnakumar Balasubramanian

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7O: Nonconvex optimization with applications in high-dimensional statistics
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Nonconvex optimization with applications in high-dimensional statistics
Chair: Andrew McRae
Cluster: Nonlinear Optimization

Talk 1: Near optimal sample complexity for the matrix and tensor normal model via geodesic convexity
Speaker: Akshay Ramachandran
Abstract: The matrix normal model, the family of Gaussian matrix-variate distributions whose covariance matrix is the Kronecker product of two lower dimensional factors, is frequently used to model matrix-variate data. The tensor normal model generalizes this family to Kronecker products of three or more factors. We study the estimation of the Kronecker factors of the covariance matrix in the matrix and tensor models. We show optimal nonasymptotic bounds for the error achieved by the maximum likelihood estimator (MLE) in several natural metrics. In contrast to existing bounds, our results do not rely on the factors being well-conditioned or sparse. For the matrix normal model, all our bounds are minimax optimal up to logarithmic factors, and for the tensor normal model our bound for the largest factor and overall covariance matrix are minimax optimal up to constant factors provided there are enough samples for any estimator to obtain constant Frobenius error. In the same regimes as our sample complexity bounds, we show that an iterative procedure to compute the MLE known as the flip-flop algorithm converges linearly with high probability. Our main tool is geodesic strong convexity in the geometry on positive-definite matrices induced by the Fisher information metric. This strong convexity is determined by the expansion of certain random quantum channels.

Talk 2: Nonconvex landscapes for phase retrieval
Speaker: Andrew McRae
Abstract: In phase retrieval, we want to recover a signal from the magnitudes of linear measurements. The nonlinearity of the absolute value makes a simple least-squares estimator a nonconvex optimization problem. Nevertheless, empirically, nonconvex methods work well, and there is some limited theory to explain this when the measurements are made randomly. I will present some general deterministic results on when a common smooth but quartic nonconvex formulation has a benign landscape: that is, the only local optimum is, in fact, the true signal we want to recover. This recovers existing landscape results with a simpler and more general analysis. Furthermore, even when this result fails, I show that we can often overcome this and still have a benign landscape by (slightly) relaxing the problem. This brings the benefits of the convex semidefinite PhaseLift relaxation while maintaining a far smaller optimization problem.

Talk 3: Gradient descent with adaptive stepsize converges (nearly) linearly under fourth-order growth
Speaker: Liwei Jiang
Abstract: A prevalent belief among optimization specialists is that linear convergence of gradient descent is contingent on the function growing quadratically away from its minimizers. In this work, we argue that this belief is inaccurate. We show that gradient descent with an adaptive stepsize converges at a local (nearly) linear rate on any smooth function that merely exhibits fourth-order growth away from its minimizer. The adaptive stepsize we propose arises from an intriguing decomposition theorem: any such function admits a smooth manifold around the optimal solution—which we call the ravine—so that the function grows at least quadratically away from the ravine and has constant order growth along it. The ravine allows one to interlace many short gradient steps with a single long Polyak gradient step, which together ensure rapid convergence to the minimizer. We illustrate the theory and algorithm on the problems of matrix sensing and factorization and learning a single neuron in the overparameterized regime.

Speakers
AR

Akshay Ramachandran

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AM

Andrew McRae

EPFL
Name: Andrew D. McRaeTitle: Postdoctoral researcherAffiliation: Institute of Mathematics, EPFL
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7P: Advances in Nonlinear Optimization Methods and Applications
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Advances in Nonlinear Optimization Methods and Applications
Chair: Wenqi Zhu
Cluster: Nonlinear Optimization

Talk 1: A simple and reliable adaptive trust-region method
Speaker: Fadi Hamad
Abstract: We present an adaptive trust region method for unconstrained optimization that allows inexact solutions to the trust-region subproblems. Our method is a simple variant of the classical trust-region method of \citet{sorensen1982newton}. The method achieves the best possible convergence bound up to an additive log factor, for finding an $\epsilon$-approximate stationary point, i.e., $O( \Delta_f L^{1/2} \epsilon^{-3/2}) + \tilde{O}(1)$ iterations where $L$ is the Lipschitz constant of the Hessian, $\Delta_f$ is the optimality gap, and $\epsilon$ is the termination tolerance for the gradient norm. This improves over existing trust-region methods in terms of its $L$ dependence. We compare our performance with state-of-the-art trust-region (TRU) and cubic regularization (ARC) methods from the GALHAD library on the CUTEst benchmark set on problems with more than 100 variables. We use fewer function evaluations, gradient evaluations and Hessian evaluations than these methods. For instance, the shifted geometric mean of function evaluations improve over TRU and ARC by $1.4$ and $2$ times respectively. Compared to the conference version of this paper \cite{hamad2022consistently}, our revised method includes practical enhancements and a refined subproblems termination criteria. These modifications dramatically improved performance including an order of magnitude reduction in the shifted geometric mean of wall-clock times.}

Talk 2: An Infeasible Primal-Dual Interior-Point Method for Nonconvex Conic Optimization
Speaker: Chuwen Zhang
Abstract: We propose an infeasible primal-dual interior-point method (iPDIPM) for linear-constrained nonconvex conic optimization problems. Existing nonconvex interior-point methods are often pure "primal" and rely on a two-stage strategy that first finds a feasible start. Our approach utilizes the regularized augmented systems that align closely with more practical interior-point solvers such as IPOPT, KNITRO, and MadNLP. In iPDIPM, primal and dual residuals are carefully balanced by controls of regularization and perturbed primal-dual residuals. In spirit, this method is an extension of "the surface of analytic centers" in Mizuno-Todd-Ye (1995) for linear programming. We leverage a Lyapunov-type analysis for self-concordant barriers and show iPDIPM has an $O(\mu^{-3/2})$ worst-case complexity to find $\mu$-approximate KKT points, which matches the pure primal methods with strictly feasible initial points. Extensions to inequality-constrained problems are also discussed.

Talk 3: Quasi-Newton methods for solving general nonlinear equations
Speaker: Chengchang Liu
Abstract: This talk focuses on quasi-Newton methods for solving general nonlinear equations. Unlike existing theory for quasi-Newton methods, which mostly focuses on the case such that the number of nonlinear equations $n$ and the dimension of the variable $d$ are identical, we provide analysis on the case that they can can be different. For $n\geq d$, we establish a fast local linear rate, which can be independent of the condition number. For $n\leq d$, we establish local superlinear rates, which match the results for strongly convex optimization. We will further show how to boost these results by incorporating the block-type quasi-Newton updates.

Speakers
WZ

Wenqi Zhu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FH

Fadi Hamad

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CZ

Chuwen Zhang

The University of Chicago
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CL

Chengchang Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7Q: Quantum algorithms and optimization
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Quantum algorithms and optimization
Chair: Sander Gribling
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Quantum interior point methods
Speaker: Sander Gribling
Abstract: TBD

Talk 2: Quantum algorithms for volume estimation
Speaker: Arjan Cornelissen
Abstract: TBD

Talk 3: Quantum speedups for stochastic optimization
Speaker: Chenyi Zhang
Abstract: TBD

Speakers
SG

Sander Gribling

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AC

Arjan Cornelissen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CZ

Chenyi Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7R: Algebraic Methods in Optimization (Part 2)
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Algebraic Methods in Optimization (Part 2)
Chair: Kevin Shu
Cluster: Conic and Semidefinite Optimization

Talk 1: Hidden Convexity via Algebraic Topology
Speaker: Kevin Shu
Abstract: Hidden convexity is a general term used to describe situations in which a seemingly nonconvex optimization problem can be reformulated as a convex optimization problem. For this talk, we will focus on examples in which the image of a nonconvex set (which we think of as the domain of some optimization problem) under a (possibly) nonlinear map is convex, and we will highlight the connections between this question and some basic notions in algebraic topology. Specific examples will be various projections of the group of rotation matrices that are convex, and applications to orientation finding will be given.

Talk 2: Spectral methods for polynomial optimization
Speaker: Elvira Moreno Ferreira
Abstract: We propose a hierarchy of tractable relaxations based on spectral methods for polynomial optimization problems (POPs). Concretely, our hierarchy of spectral relaxations yields a monotonic sequence of bounds for the optimal value of a POP, each of which is computed as the minimum eigenvalue of a matrix obtained from the problem data. Because spectral methods are less computationally demanding than semidefinite programs, which underpin state-of-the-art techniques based on SOS certificates of non-negativity, our hierarchy provides an attractive alternative for obtaining bounds on large-scale problems. We identify the algebraic structure underlying a POP that makes it amenable to spectral relaxations, and we demonstrate the efficiency and applicability of our framework with numerical examples. 

Talk 3: The landscape analysis problem of non-convex reformulations
Speaker: Chenyang Yuan
Abstract: We consider the problem of determining whether non-convex parametrizations of convex optimization problems have no spurious first- and second-order critical points. We show that this problem is equivalent to determining whether an infinite family of convex programs has a non-zero solution. When these optimization problems have polynomial structure, we use algebraic tools to develop strategies for partitioning and searching this infinite family, as well as finding counterexamples. We also discuss situations when strong duality fails to hold and how to remedy this with extended dual formulations.

Speakers
KS

Kevin Shu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
EM

Elvira Moreno Ferreira

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CY

Chenyang Yuan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7S: Any-dimensional symmetric optimization
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Any-dimensional symmetric optimization
Chair: Eitan Levin
Cluster: Conic and Semidefinite Optimization

Talk 1: Any-dimensional polynomial optimization
Speaker: Eitan Levin
Abstract: Optimization problems in many applications arise as sequences indexed by dimension. Relaxations producing bounds for such problems should then similarly be defined for all relevant dimensions, and bounds should be produced on their limiting optimal values. For example, in combinatorics convex relaxations for graph parameters need to be defined for graphs of all sizes, and inequalities between graph densities need to be certified for graphs of increasing size. In (quantum) information theory, relaxations should be defined for distributions on any number of (qu)bits. And in signal processing, regularizers should be defined for signals of any length. We present a framework to systematically study optimization problems which are defined for any dimension, fit tractable relaxations for them from data which are defined for all problem sizes, and derive lower bounds on their asymptotic values. We do so by observing that these problems are described in a “free” way which makes it obvious how to instantiate them in any dimension. We explain how such free descriptions arise from the phenomenon of representation stability, and study their properties by considering the relations between problems across dimensions and their symmetries. We present two consequences of our framework. First, we show that sequences of semidefinite relaxations of many polynomial problems stabilize in size, allowing us to derive bounds on their asymptotic optimal values. Second, we derive families of functions defined for all dimensions described by finitely-many parameters, allowing us to fit such functions to data in a few small dimensions and instantiate the fitted function in any other dimension.

Talk 2: Optimality Conditions for Extremal Combinatorics
Speaker: Daniel Brosch
Abstract: Given a sequence of graphs of increasing size $\mathcal{G} = (G_1,G_2,G_3,\ldots)$, we define the density of a finite graph $H$ in $\mathcal{G}$ as the limit of the subgraph densities of $H$ in the finite graphs $G_i$. By considering functions which assign $\mathcal{G}$ a real linear combination of subgraph densities, we can formulate a wide variety of problems of (asymptotic) extremal graph theory in the form $\min_{\mathcal{G}} \{f(\mathcal{G})\mid g(\mathcal{G})\geq 0, h(\mathcal{G})= 0\}$. We define derivations of graph density functions, study the tangent cone of such problems, and prove optimality conditions for the constrained and unconstrained case. Finally, we consider the consequences for the geometry of the space of graph limits, as well as open conjectures such as Sidorenko's conjecture and the Caccetta-Häggkvist conjecture.

Talk 3: Semidefinite bounds for covering codes
Speaker: Sven Polak
Abstract: Let $K_q(n,r)$ denote the minimum size of a $q$-ary \emph{covering code} of word length $n$ and covering radius $r$. In other words, $K_q(n,r)$ is the minimum size of a set of $q$-ary codewords of length $n$ such that the Hamming balls of radius $r$ around the codewords cover the Hamming space $\{0,\ldots,q-1\}^n$. The special case $K_3(n,1)$ is often referred to as the \emph{football pool problem}, as it is equivalent to finding a set of forecasts on $n$ football matches that is guaranteed to contain a forecast with at most one wrong outcome. In this work, we build and expand upon the work of Gijswijt (2005), who introduced a semidefinite programming lower bound on $K_q(n,r)$ via matrix cuts. We develop techniques that strengthen this bound, by introducing new semidefinite constraints inspired by Lasserre's hierarchy for 0-1 programs and symmetry reduction methods, and a more powerful objective function. The techniques lead to sharper lower bounds, setting new records across a broad range of values of $q$, $n$, and $r$. This talk is based on joint work with Dion Gijswijt.

Speakers
EL

Eitan Levin

PhD student, Caltech
I'm a PhD student in applied math at Caltech interested in the interplay between geometry, symmetry, and optimization and its applications to data science.
DB

Daniel Brosch

Postdoc, University of Klagenfurt
Interested in all things SDPs, interactions with symmetries, and representation theory.Recently, I've mostly been working with flag algebras.
SP

Sven Polak

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7T: Algorithms for Optimization and Equilibrium Models in Energy
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Algorithms for Optimization and Equilibrium Models in Energy
Chair: Dominic Flocco
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Using spectral stepsizes and total variation in FWI
Speaker: Paulo Silva
Abstract: Full Wave Inversion is a computational technique that can unveil Earth's subsurface properties by solving large-scale inverse problems that minimize the difference between a wave propagation simulation based on a differential equation and actual observed data. In this talk, we will use the spectral proximal gradient (SPG) method to solve regularized FWI and compare the results to the direct application of L-BFGS-B, widely used by the geophysics community. SPG employs the spectral step introduced by Barzilai and Borwein and popularized by Raydan, with a non-monotone line search to achieve performance similar to quasi-Newton methods. At the same time, as it can cope with proximal terms, it can deal with different regularizations to induce desired properties in the model. Then, we used a total variation with box constraints regularization to recover high-quality models even from poor initial solutions. The number of iterations SPG requires is low, showing it might be useful to solve huge-scale 3D problems.

Talk 2: Modelling unit commitment constraints in a Cournot equilibrium model for a electricity market
Speaker: Mel Devine
Abstract: Electricity market modelling that captures the technical constraints of power system operation, such as start and shut down costs, require the use of binary programming so as to model the “on-off” condition of conventional generation units (Bothwell & Hobbs, 2017). Accurate modelling of power systems with higher levels of renewable generation requires such integer variables (Shortt, et al., 2012) and also high time resolution (Merrick, 2016). These high resolution integer models determine the least-cost schedule for generation technologies, which correlates to a strategy of profit-maximisation in a perfectly competitive market. However, electricity markets are better characterized by an oligopoly, where at least some firms have market power, giving them the ability to influence prices by varying their output and/or prices. Modelling this price-making ability requires equilibrium problems such as Mixed Complementarity Problems (MCPs), Mathematical Programs with Equilibrium Constraints (MPECs) or Equilibrium Problem with Equilibrium Constraints (EPECs). However, these problems cannot model a market where all market participants have integer decision variables. Despite notable exceptions (Gabriel, 2017; Weinhold and Gabriel, 2020), the literature on incorporating integer decision variables into equilibrium models is only emerging. In this work, we develop a Game Theory Optimisation model of a wholesale electricity market where all generators maximise profits through price-making, in addition to having integer decision variables. To solve our model, we employ the Gauss-Seidel diagonalisation algorithm, which is typically associated with EPEC models. The optimisation problem of each generator is solved individually, holding all other generators’ decision variables fixed. The algorithm iteratively solves each generator’s optimisation problem by fixing all other generators’ decisions, until it converges to a point where no generator has an optimal deviation. To improve computational efficiency, we also employ a rolling-horizon algorithm. In this talk, results will be presented on based on real-world data from the Irish power system. Thus we consider 16 generating firms that vary in size from large scale integrated firms with several generating technologies to small stand-alone firms. Furthermore, computational analysis, issues surrounding our methodological approach, and future directions will be discussed.

Talk 3: Exact Penalty Techniques via Difference-of-Convex Function Programming for General Bilinear Programs
Speaker: Dominic Flocco
Abstract: We present a new difference of convex functions algorithm (DCA) for solving general bilinear programs. The approach is based on the reformulation of bilinear constraints as difference of convex (DC) functions, more specifically, the difference of scalar, convex quadratic terms. This reformulation gives rise to a DC program, which is solved via sequential linear approximations of the concave term using standard DCA techniques. We explore variations on the classical DCA approach by considering linear- and Newton-proximal DCA and DC bundle methods. We apply this novel algorithmic framework to a variety of linear complementarity problems (LCP), mathematical programs with equilibrium constraints (MPEC) and bilevel optimization problems that arise in energy and infrastructure models.

Speakers
PS

Paulo Silva

Full Professor, Universidade Estadual de Campinas
Name: Paulo J. S. SilvaTitle: Full ProfessorAffiliation: Universidade Estadual de Campinas
DF

Dominic Flocco

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7U: Randomized optimization algorithms 2/2
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Randomized optimization algorithms 2/2
Chair: Laurent Condat
Cluster: Nonlinear Optimization

Talk 1: Block Coordinate DC Programming
Speaker: Alp Yurtsever
Abstract: We introduce an extension of the Difference of Convex Algorithm (DCA) in the form of a randomized block coordinate approach for problems with a separable structure. For n coordinate blocks and k iterations, our main result proves a non-asymptotic convergence rate of O(n/k) for the proposed method. Furthermore, leveraging the connection between DCA and Expectation Maximization (EM), we propose a block coordinate EM algorithm.

Talk 2: Variable metric proximal stochastic gradient methods with additional sampling
Speaker: Ilaria Trombini
Abstract: Many optimization problems in machine learning can be framed as minimiz- ing the sum of two functions: the first typically represents the expected risk, which is often substituted by the empirical risk in practice, while the second en- codes prior information about the solution. Given that the first term is generally differentiable and the second is convex, proximal gradient methods are particu- larly well-suited for addressing such optimization challenges. This talk presents a new class of proximal stochastic gradient methods, which are characterized by three fundamental components: a variable metric guiding the iterations, a stochastic line search governing the decrease properties, and an incremental mini- batch size technique that utilizes additional sampling. The convergence of the proposed algorithms is established under various assumptions about the function being minimized. Notably, no assumption is made about the Lipschitz continu- ity of the gradient for the differentiable part of the objective function. The talk also explores possible strategies for automatically choosing the parameters of the proposed method. Numerical tests on binary classification tasks demonstrate the effectiveness of this approach when compared to other state-of-the-art proximal stochastic gradient methods.

Talk 3: A Framework for Stochastic Quasi-Newton Type Methods
Speaker: Titus Pinta
Abstract: We develop a stochastic framework to analyze Quasi-Newton type methods where the Hessian approximation is an instance of a random variable. We will show that under a weak assumption on the regularity of these Hessian approximations, the sequence produced by the algorithm has a sufficient decrease property, with high probability. This intermediary result will be help us characterize the probability distribution of the random variable that counts the number of iterations until the algorithm produces a point with gradient norm less than a given epsilon. As a special case, we will see how random additive Gaussian noise impacts the performance of a Quasi-Newton type method. Finally, we provide some numerical results, based on random subspace embedding, showcasing our theory.

Speakers
avatar for Laurent Condat

Laurent Condat

Senior Research Scientist, King Abdullah University of Science and Technology (KAUST)
Laurent Condat received a PhD in applied mathematics in 2006 from Grenoble Institute of Technology, Grenoble, France. After a postdoc in the Helmholtz Zentrum Muenchen, Munich, Germany, he was hired in 2008 as a permanent researcher by the French National Center for Scientific Research... Read More →
AY

Alp Yurtsever

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Ilaria Trombini

Ilaria Trombini

Fellow, University of Ferrara
TP

Titus Pinta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7V: First-order Methods for Riemannian Optimization
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: First-order Methods for Riemannian Optimization
Chair: David Gutman
Cluster: Optimization on Manifolds

Talk 1: Tangent Subspace Descent via Discontinuous Subspace Selections on Fixed-Rank Manifolds
Speaker: David Gutman
Abstract: The tangent subspace descent method (TSD) extends the coordinate descent algorithm to manifold domains. The key insight underlying TSD is to draw an analogy between coordinate blocks in Euclidean space and tangent subspaces of a manifold. The core principle behind ensuring convergence of TSD for smooth functions is the appropriate choice of subspace at each iteration. Previously, it was shown that it is always possible to appropriately pick such subspaces on the broad class of manifolds known as naturally reductive homogeneous spaces. In this talk, we provide the first instances of TSD for manifolds outside of this class. The main idea underlying these new instances is the use of discontinuous subspace selections. As a result of our developments we derive new and efficient methods for large-scale optimization on the fixed-rank and fixed-rank positive semidefinite matrix manifolds.

Talk 2: Retraction-Free Decentralized Non-convex Optimization with Orthogonal Constraints
Speaker: Shahin Shahrampour
Abstract: In this work, we investigate decentralized non-convex optimization with orthogonal constraints. Conventional algorithms for this setting require either manifold retractions or other types of projection to ensure feasibility, both of which involve costly linear algebra operations (e.g., SVD or matrix inversion). On the other hand, infeasible methods are able to provide similar performance with higher computational efficiency. Inspired by this, we propose the first decentralized version of the retraction-free landing algorithm, called Decentralized Retraction-Free Gradient Tracking (DRFGT). We theoretically prove that DRFGT enjoys the ergodic convergence rate of O(1/K), matching the convergence rate of centralized, retraction-based methods. We further establish that under a local Riemannian PŁ condition, DRFGT achieves the much faster linear convergence rate. Numerical experiments demonstrate that DRFGT performs on par with the state-of-the-art retraction-based methods with substantially reduced computational overhead.

Talk 3: Adaptive Low Rank Representation in Reinforcement Learning
Speaker: Chenliang Li
Abstract: In reinforcement learning (RL), there is a trade-off between asymptotic bias (performance gap between the policy identified by RL and the actual optimal policy) and over-fitting (additional suboptimality due to limited data or other source of noise). In this paper, we study these two sources of error in RL with noisy environment dynamics. Our theoretical analysis demonstrates that while a low-rank representation of the value and policy functions may increase asymptotic bias, it reduces the risk of over-fitting. Further, we propose a practical algorithm, named Adaptive Low Rank (ALR) Representation for Reinforcement Learning, which adaptively tunes the rank of the model to better suit the reinforcement learning environment in terms of balancing the asymptotic bias and overfitting. We validate the efficiency of the proposed solution through extensive experiments such as the standard MuJoCo task. Our results show that the algorithm significantly outperforms baseline reinforcement learning solvers, such as Soft Actor Critic (SAC), particularly in noisy environments with limited observations.

Speakers
DG

David Gutman

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SS

Shahin Shahrampour

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CL

Chenliang Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7W: Minimax Optimization Algorithms
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Minimax Optimization Algorithms
Chair: Jaeyeon Kim
Cluster: nan

Talk 1: Stochastic Extragradient with Flip-Flop Shuffling & Anchoring: Provable Improvements
Speaker: Jiseok Chae
Abstract: In minimax optimization, the extragradient (EG) method outperforms gradient descent-ascent in convex-concave (C-C) problems. Yet, stochastic EG (SEG) has seen limited success in C-C problems, especially for unconstrained cases. In this talk, we discuss the convergence properties of shuffling-based SEG in unconstrained finite-sum minimax problems, motivated by recent progress in shuffling-based stochastic methods. We first show that both random reshuffling and flip-flop shuffling sampling schemes, while successful in minimization problems, each alone can suffer divergence in C-C problems. Then, we show how incorporating a simple "anchoring" trick led to our SEG with flip-flop anchoring (SEG-FFA) method, which successfully converges in C-C problems. We will also present upper and lower bounds in the strongly-convex-strongly-concave setting, demonstrating SEG-FFA's provably faster convergence rate compared to other shuffling-based methods. Reference to the publication(s): https://proceedings.neurips.cc/paper_files/paper/2024/hash/df658fe5097f65485ad80b06e6cb30dd-Abstract-Conference.html

Talk 2: Double-step alternating extragradient with timescale separation for finding local minimax points
Speaker: Kyuwon Kim
Abstract: In minimization, gradient descent converges to a local minimum, and almost surely avoids strict saddle points, under mild conditions. In contrast, minimax optimization lacks such comparable theory for finding local minimax (optimal) points. Recently, the two-timescale extragradient (EG) method has shown potential for finding local minimax points, over the two-timescale gradient descent ascent method. However, it is yet not stable enough for finding any degenerate local minimax points that are prevalent in modern over-parameterized setting. We thus propose to incorporate a new double-step alternating update strategy to further improve the stability of the two-timescale EG method, which remedies the aforementioned issue. Under mild conditions, we show that the proposed method converges to local minimax points that all existing two-timescale methods fail to do so.

Talk 3: H-duality: Duality between Minimizing Gradient Norm and Function Value
Speaker: Jaeyeon Kim
Abstract: In convex optimization, first-order optimization methods efficiently minimizing function values have been a central subject study since Nesterov’s seminal work of 1983. More recently, Kim and Fessler’s OGM-G and Lee et al.’s FISTA-G have been introduced as alternatives that focus on efficiently minimizing the gradient magnitude instead. In this talk, we present H-duality [1], a surprising one-to-one correspondence between methods efficiently minimizing function values and methods efficiently minimizing gradient magnitude. The H-dual of a given first-order method is defined as another first-order method whose coefficients are time-reversed. To the best of our knowledge, H-duality is distinct from Lagrange/Fenchel duality and is distinct from any previously known duality or symmetry relations. H-duality is further extended to the mirror descent setup [2] and fixed-point problems [3]. Using H-duality, we derive new acceleration first-order methods by simply considering the H-dual of existing methods. [1] Jaeyeon Kim, Asuman Ozdaglar, Chanwoo Park, and Ernest Ryu. Time-reversed dissipation induces duality between minimizing gradient norm and function value. Advances in Neural Information Processing Systems, 2023. [2] Jaeyeon Kim, Chanwoo Park, Asuman Ozdaglar, Jelena Diakonikolas, and Ernest K Ryu. Mirror duality in convex optimization. arXiv preprint arXiv:2311.17296, 2023. [3] TaeHo Yoon, Jaeyeon Kim, Jaewook J Suh, and Ernest K Ryu. Optimal acceleration for minimax and fixed-point problems is not unique. International Conference of Machine Learning, 2024.

Speakers
JC

Jiseok Chae

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KK

Kyuwon Kim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JK

Jaeyeon Kim

Ph.D. Student, Harvard University
Name: Jaeyeon KimHello, my name is Jaeyeon Kim! I’m a first-year Ph.D. student at Harvard CS.
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7X: Constraint Qualification and Error Bound
Wednesday July 23, 2025 10:30am - 11:45am PDT
Session: Constraint Qualification and Error Bound
Chair: Mariana da Rosa
Cluster: nan

Talk 1: TBD
Speaker: Meisam Razaviyayn
Abstract: TBD

Talk 2: Reparametrization of feasible sets: new constraint qualifications in nonlinear programming and connections to MFCQ
Speaker: Mariana da Rosa
Abstract: It is well known that constant rank-type constraint qualifications (CQs) imply the Mangasarian-Fromovitz CQ (MFCQ) after a suitable local reparametrization of the feasible set. This process involves eliminating redundancies without changing the feasible set locally. Traditionally, such reparametrizations have been used to highlight connections between well-established CQs from the literature. In this talk, we take a different perspective by introducing a type of reparametrization that itself constitutes a CQ. We conduct a thorough study of such reparametrizations, not only in the context of MFCQ but also in relation to arbitrary CQs, and discuss the relationship between these CQs obtained via reparametrizations and the local error bound property. Additionally, we propose a relaxed version of CRSC, called constrained CRSC, which retains the key geometric properties of CRSC, is linked to reparametrizations leading to MFCQ, and also guarantees the local error bound property. Preprint: https://optimization-online.org/?p=28999

Speakers
avatar for Meisam Razaviyayn

Meisam Razaviyayn

Associate Professor, University of Southern California
Bio: Meisam Razaviyayn (https://sites.usc.edu/razaviyayn) is an associate professor in the departments of Industrial and Systems Engineering, Computer Science, Quantitative and Computational Biology, and Electrical Engineering at the University of Southern California. He also serves as the associate director of the USC-Meta Center for... Read More →
MD

Mariana da Rosa

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 7Y
Wednesday July 23, 2025 10:30am - 11:45am PDT
Wednesday July 23, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

11:45am PDT

Lunch 3 (provided)
Wednesday July 23, 2025 11:45am - 1:15pm PDT
Asian Buffet
Wednesday July 23, 2025 11:45am - 1:15pm PDT
USC Founder's / Hutton Park 3551 Trousdale Pkwy, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8A: California to Amalfi Coast via Québec
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: California to Amalfi Coast via Québec
Chair: Leo Liberti
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Fair and Accurate Regression: Relaxations, Heuristics and Exact Methods
Speaker: Anna Deza
Abstract: We consider the problem of fair regression which aims to train a machine learning model subject to fairness constraints or regularizers. While there is a large body of work addressing fair classification, the regression setting has received far less attention. Key to measuring popular fairness metrics is the use of indicators based on the intervals into which predictions fall. This poses a challenge as it makes the training problem non-convex and non-smooth. Most existing approaches avoid these hurdles by using convex proxies, approximations, and heuristics to produce fair models. We instead consider the exact fairness measures at hand and develop mixed-integer non-linear formulations for training fair models. We give a strong reformulation obtained by jointly convexifying the non-linear loss function of the model and indicators associated with each prediction. Solution times relative to the natural big-M formulation are substantially improved via the proposed reformulation. We provide an efficient coordinate descent algorithm to produce locally optimal fair models. Our numerical results show that the models produced by the relaxation alone have competitive statistical performance when compared to the state-of-the-art, achieving better accuracy-fairness trade-offs at a fraction of the time. Coordinate descent further improves upon the relaxed models. These results are consistent across synthetic and real datasets.

Talk 2: A bilevel approach for compensation and routing decisions in last-mile delivery
Speaker: Martina Cerulli
Abstract: In last-mile delivery logistics, peer-to-peer logistic platforms play an important role in connecting senders, customers, and independent carriers to fulfill delivery requests. Since the carriers are not under the platform’s control, the platform has to anticipate their reactions, while deciding how to allocate the delivery operations. In this work, we model this problem using bilevel programming. At the upper level, the platform decides how to assign the orders to the carriers together with the compensation paid for each accepted request; at the lower level, each carrier solves a profitable tour problem to determine which offered requests to accept, based on her own profit maximization. For the resulting bilevel formulation, we propose a single-level reformulation and an alternative formulation where the lower-level routing variables are projected out. A branch-and-cut algorithm is proposed to solve the bilevel models and extensive computational tests are performed to compare the proposed formulations and analyze solution characteristics.

Talk 3: A Polyhedral Study on L-Natural-Convex Minimization and Its Mixed-Integer Extension
Speaker: Kim Yu
Abstract: L-natural-convex functions are a class of nonlinear functions defined over integral domains. Such functions are not necessarily convex, but they display a discrete analogue of convexity. In this work, we explore the polyhedral structure of the epigraph of any L-natural-convex function and provide a class of valid inequalities. We show that these inequalities are sufficient to describe the epigraph convex hull completely, and we give an exact separation algorithm. We further examine a mixed-integer extension of this class of minimization problems and propose strong valid inequalities. We establish the connection between our results and the valid inequalities for some structured mixed-integer sets in the literature.

Speakers
LL

Leo Liberti

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AD

Anna Deza

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MC

Martina Cerulli

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KY

Kim Yu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8B: Manifolds, samples, and learning
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Manifolds, samples, and learning
Chair: Ralf Zimmermann
Cluster: Optimization on Manifolds

Talk 1: Dynamic Subspace Estimation using Grassmannian Geodesics
Speaker: Laura Balzano
Abstract: In this work, we consider recovering a sequence of low-rank matrices from noisy and possibly undersampled measurements, where an underlying subspace varies across samples over time. We propose a Riemannian block majorize-minimization algorithm that constrains the time-varying subspaces as a geodesic along the Grassmann manifold. Our proposed method can faithfully estimate the subspaces at each time point, even when the number of samples at each time point is less than the rank of the subspace. We demonstrate the effectiveness of our algorithm on synthetic data, dynamic fMRI data, video data, and graph data with evolving communities.

Talk 2: Shape-Graph Matching Network (SGM-Net): Registration for Statistical Shape Analysis
Speaker: Shenuan Liang
Abstract: This talk explores the statistical analysis of shapes of data objects called shape graphs, a set of nodes connected by articulated curves with arbitrary shapes. A critical need here is a constrained registration of points (nodes to nodes, edges to edges) across objects. This requires optimization over the permutations, made challenging by differences in nodes (in terms of numbers, locations) and edges (in terms of shapes, placements, and sizes) across objects. We tackle this registration problem using a novel neural-network architecture and formulate it using an unsupervised loss function derived from the elastic shape metric. This architecture results in (1) state-of-the-art performance and (2) an order of magnitude reduction in the computational cost relative to baseline approaches. We demonstrate the effectiveness of the proposed approach using both simulated data and real-world, publicly available 2D and 3D shape graphs.

Talk 3: NP-hardness of Grassmannian optimization
Speaker: Zehua Lai
Abstract: We show that unconstrained quadratic optimization over a Grassmannian is NP-hard. Our results cover all scenarios: (i) when k and n are both allowed to grow; (ii) when k is arbitrary but fixed; (iii) when k is fixed at its lowest possible value 1. We then deduce the NP-hardness of unconstrained cubic optimization over the Stiefel manifold and the orthogonal group. As an addendum we demonstrate the NP-hardness of unconstrained quadratic optimization over the Cartan manifold, i.e., the positive definite cone regarded as a Riemannian manifold, another popular example in manifold optimization. We will also establish the nonexistence of FPTAS in all cases.

Speakers
RZ

Ralf Zimmermann

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LB

Laura Balzano

Name: Professor Laura BalzanoTitle: Associate Professor of Electrical Engineering and Computer ScienceAffiliation: University of MichiganBio:Laura Balzano is an associate professor of Electrical Engineering and Computer Science, and of Statistics by courtesy, at the University of... Read More →
SL

Shenuan Liang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZL

Zehua Lai

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8C: Data-driven Methods for Optimization under Uncertainty
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Data-driven Methods for Optimization under Uncertainty
Chair: Meng Qi
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Beyond Discretization: Learning the Optimal Solution Map
Speaker: Paul Grigas
Abstract: Many applications require minimizing a family of optimization problems, indexed by some hyperparameter vector, to obtain an entire solution map (or solution path). Traditional approaches proceed by discretizing and solving a series of optimization problems. We propose an alternative approach that parametrizes the solution map within a given function class and solves a single stochastic optimization problem to learn the entire solution map. Our method offers substantial complexity improvements over discretization. When using constant-step size SGD, the uniform error of our learned solution map relative to the true map exhibits linear convergence to a constant that diminishes as the expressiveness of the function class increases. In the case of a one-dimensional hyperparameter, we prove stronger results that demonstrate complexity improvements over the best known results for uniform discretization whenever the solution map is sufficiently smooth. Finally, we discuss other extensions including an adaptive variant of our method that sequentially adds basis functions, and we demonstrate strong numerical performance through experiments on imbalanced binary classification and portfolio optimization examples.

Talk 2: Decision-Focused Learning with Directional Gradients
Speaker: Michael Huang
Abstract: We propose a novel family of decision-aware surrogate losses, called Perturbation Gradient (PG) losses, for the predict-then-optimize framework. The key idea is to connect the expected downstream decision loss with the directional derivative of a particular plug-in objective, and then approximate this derivative using zeroth order gradient techniques. Unlike the original decision loss which is typically piecewise constant and discontinuous, our new PG losses is a Lipschitz continuous, difference of concave functions that can be optimized using off-the-shelf gradient-based methods. Most importantly, unlike existing surrogate losses, the approximation error of our PG losses vanishes as the number of samples grows. Hence, optimizing our surrogate loss yields a best-in-class policy asymptotically, even in misspecified settings. This is the first such result in misspecified settings, and we provide numerical evidence confirming our PG losses substantively outperform existing proposals when the underlying model is misspecified.

Talk 3: Norm-Free Exact Regularization and Applications in Data-Driven Optimization
Speaker: Meng Qi
Abstract: This paper revisits the theory of \textit{exact regularization} – where optimal solutions of a regularized convex optimization problem exhibit a phase transition phenomenon and eventually coincide with those of the original unregularized problem (under certain conditions).We examine this phenomenon from a norm-free perspective – instead of adopting norm-related assumptions, our results are established on conditions only involving Bregman divergence and convexity. We proved two key results: (1) a norm-free version of Lipschitz continuity of the regularized optimal solution, and (2) a phase-transition threshold for the exact regularization to hold that depends solely on intrinsic problem parameters. Notably, our norm-free framework generalizes classical norm-dependent conditions, such as strong convexity of the regularization function, and broadens applicability. Our theoretical results have applications in many data-driven optimization problems, for example to integrated prediction-optimization, inverse optimization, and decentralized optimization. Our results for exact regularization potentially lead to faster convergence or tighter error bounds in these settings.

Speakers
PG

Paul Grigas

Associate Professor, UC Berkeley
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MH

Michael Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MQ

Meng Qi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8D: Performance Estimation Problem and Universal Methods
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Performance Estimation Problem and Universal Methods
Chair: Sam Davanloo
Cluster: Nonlinear Optimization

Talk 1: Performance Estimation for Smooth and Strongly Convex Sets
Speaker: Benjamin Grimmer
Abstract: This talk will present extensions of recent computer-assisted design and analysis techniques for first-order optimization over structured functions---known as performance estimation---to apply to structured sets. Namely, we prove ``interpolation theorems'' for smooth and strongly convex sets with Slater points and bounded diameter, showing a wide range of extremal questions amount to structured mathematical programs. Prior function interpolation theorems are recovered as a limit of our set interpolation theory. Our theory provides finite-dimensional formulations of performance estimation problems for algorithms utilizing separating hyperplane oracles, linear optimization oracles, and/or projection oracles of smooth/strongly convex sets. As direct applications of this computer-assisted machinery, we identify the minimax optimal separating hyperplane method and several areas for improvement in the theory of Frank-Wolfe, Alternating Projections, and non-Lipschitz Smooth Optimization.

Talk 2: Universal subgradient and proximal bundle methods for convex and strongly convex hybrid composite optimization
Speaker: Jiaming Liang
Abstract: This work develops two parameter-free methods for solving convex and strongly convex hybrid composite optimization problems, namely, a composite subgradient type method and a proximal bundle type method. Both functional and stationary complexity bounds for the two methods are established in terms of the unknown strong convexity parameter. To the best of our knowledge, the two proposed methods are the first universal methods for solving hybrid strongly convex composite optimization problems that do not rely on any restart scheme nor require the knowledge of the optimal value.

Talk 3: Nonasymptotic Analysis of Accelerated Methods With Inexact Oracle Under Absolute Error Bound
Speaker: Sam Davanloo
Abstract: Performance analysis of first-order algorithms with inexact oracle has gained recent attention due to various emerging applications in which obtaining exact gradients is impossible or computationally expensive. Previous research has demonstrated that the performance of accelerated first-order methods is more sensitive to gradient errors compared with non-accelerated ones. This work investigates the nonasymptotic convergence bound of two accelerated methods with inexact gradients to solve deterministic smooth convex problems. Performance Estimation Problem (PEP) is used as the primary tool to analyze the convergence bounds of the underlying algorithms. By finding an analytical solution to PEP, we derive novel convergence bounds for the Inexact Optimized Gradient Method (OGM) of Kim and Fessler 2016 and the Inexact Fast Gradient Method (FGM) of Devolder et al. 2013 with variable oracle inexactness along iterations. Under the absolute error assumption, we derive bounds in which the accumulated errors are independent of the initial conditions and the trajectory of the sequences generated by the algorithms. Furthermore, we analyze the tradeoff between the convergence rate and accumulated error that guides finding the optimal stepsize. Finally, we determine the optimal strategy to set the gradient inexactness along iterations (if possible), ensuring that the accumulated error remains subordinate to the convergence rate.

Speakers
BG

Benjamin Grimmer

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jiaming Liang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SD

Sam Davanloo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8E: Optimization Methods for Next-Generation Wireless Communication Networks (II)
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Optimization Methods for Next-Generation Wireless Communication Networks (II)
Chair: Wei Yu
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Optimizations and Signalling for Sustainable and Multifunctional Wireless Networks
Speaker: Christos Masouros
Abstract: The future global cellular infrastructure will underpin a variety of applications, such as smart city solutions, urban security, infrastructure monitoring, and smart mobility, among others. These emerging applications require new network functionalities that go beyond traditional communication. Key network KPIs for 6G include Gb/s data rates, cm-level localization, μs-level latency, and Tb/Joule energy efficiency. Additionally, future networks must support the UN's Sustainable Development Goals to ensure sustainability, net-zero emissions, resilience, and inclusivity. This talk will discuss the applications of optimization and signalling design for sustainable and multifunctional wireless networks.

Talk 2: Optimal Joint Fronthaul Compression and Beamforming Design for Networked ISAC Systems
Speaker: Tsung-Hui Chang
Abstract: This study investigates a networked integrated sensing and communication (ISAC) system, where multiple base stations (BSs), connected to a central processor (CP) via capacity-limited fronthaul links, cooperatively serve communication users while simultaneously sensing a target. The primary objective is to minimize the total transmit power while meeting the signal-to-interference-plus-noise ratio (SINR) requirements for communication and sensing under fronthaul capacity constraints, resulting in a joint fronthaul compression and beamforming design (J-FCBD) problem. We demonstrate that the optimal fronthaul compression variables can be determined in closed form alongside the beamformers, a novel finding in this field. Leveraging this insight, we show that the remaining beamforming design problem can be solved globally using the semidefinite relaxation (SDR) technique, albeit with considerable complexity. Furthermore, the tightness of its SDR reveals zero duality gap between the considered problem and its Lagrangian dual. Building on this duality result, we exploit the novel UL-DL duality within the ISAC framework to develop an efficient primal-dual (PD)-based algorithm. The algorithm alternates between solving beamforming with a fixed dual variable via fixed-point iteration and updating dual variable via bisection, ensuring global optimality and achieving high efficiency due to the computationally inexpensive iterations. Numerical results confirm the global optimality, effectiveness, and efficiency of the proposed PD-based algorithm.

Talk 3: Covariance-Based Activity Detection in Cooperative Multi-Cell Massive MIMO: Scaling Law and Efficient Algorithms
Speaker: Ya-Feng Liu
Abstract: In this talk, we focus on the activity detection problem in a multi-cell massive multiple-input multiple-output (MIMO) system, where active devices transmit their signature sequences to multiple base stations (BSs), and the BSs cooperatively detect the active devices based on the received signals. In this talk, we shall present some recent results of the covariance-based approach for activity detection in cooperative multi-cell massive MIMO systems. In particular, we shall present some scaling law results and efficient coordinate descent (CD) algorithms.

Speakers
CM

Christos Masouros

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TC

Tsung-hui Chang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Ya-Feng Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8F: Multiagent Optimization and Games
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Multiagent Optimization and Games
Chair: Songtao Lu
Cluster: Multi-agent Optimization and Games

Talk 1: A Single-Loop Algorithm for Decentralized Bilevel Optimization
Speaker: Shiqian Ma
Abstract: Bilevel optimization has gained significant attention in recent years due to its broad applications in machine learning. In this talk we focus on bilevel optimization in decentralized networks. In particular, we propose a novel single-loop algorithm for solving decentralized bilevel optimization with a strongly convex lower-level problem. Our approach is a fully single-loop method that approximates the hypergradient using only two matrix-vector multiplications per iteration. Importantly, our algorithm does not require any gradient heterogeneity assumption, distinguishing it from existing methods for decentralized bilevel optimization and federated bilevel optimization. Our analysis demonstrates that the proposed algorithm achieves the best-known convergence rate for bilevel optimization algorithms. We also present experimental results on hyperparameter optimization problems using both synthetic and MNIST datasets, which demonstrate the efficiency of our proposed algorithm.

Talk 2: Optimal No-Regret Learning in Repeated First-Price Auctions
Speaker: Zhengyuan Zhou
Abstract: First-price auctions have very recently swept the online advertising industry, replacing second-price auctions as the predominant auction mechanism on many platforms for display ads bidding. This shift has brought forth important challenges for a bidder: how should one bid in a first-price auction, where unlike in second-price auctions, it is no longer optimal to bid one's private value truthfully and hard to know the others' bidding behaviors? In this paper, we take an online learning angle and address the fundamental problem of learning to bid in repeated first-price auctions. We discuss our recent work in leveraging the special structures of the first-price auctions to design minimax optimal no-regret bidding algorithms.

Talk 3: A Primal-Dual Framework for Decentralized Bilevel Optimization
Speaker: Songtao Lu
Abstract: In this talk, I will introduce our recently proposed primal-dual framework for decentralized bilevel optimization. This framework addresses settings where multiple agents collaborate to solve nested optimization problems through neighborhood communication. While most existing approaches rely heavily on gradient tracking to manage data heterogeneity, they often overlook other effective techniques such as EXTRA or Exact Diffusion. Moreover, applying the same decentralized strategy to both the upper- and lower-level problems misses the potential of leveraging distinct methods for each level. Our unified primal-dual algorithm framework fills these gaps, allowing for the integration of various heterogeneity-correction strategies and the application of different decentralized algorithms at each level. I will present convergence results for our proposed algorithms, which achieve state-of-the-art convergence rates across all variants. Our findings also demonstrate that EXTRA and Exact Diffusion are particularly well-suited for decentralized bilevel optimization, and that combining different strategies across the bilevel hierarchy offers significant advantages over relying solely on gradient tracking.

Speakers
SM

Shiqian Ma

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZZ

Zhengyuan Zhou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SL

Songtao Lu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8G: Modern Polynomial Optimization in Data Science I
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Modern Polynomial Optimization in Data Science I
Chair: Xindong Tang
Cluster: Conic and Semidefinite Optimization

Talk 1: Sparse matrix constrained polynomial optimization
Speaker: Xindong Tang
Abstract: We study sparse matrix Moment-SOS relaxation for solving sparse matrix constrained polynomial optimization. We prove a sufficient and necessary condition for the sparse matrix Moment-SOS relaxations to be tight. How to certify the tightness and how to extract minimizers are also discussed. When the optimization problem is convex, we prove some sufficient conditions for sparse matrix Moment-SOS relaxations to be tight. In particular, we show that each sparse matrix Moment-SOS relaxation is tight when the problem is SOS-convex. Numerical experiments are provided to support the proposed method.

Talk 2: Symmetries in kernel learning
Speaker: Jack Kendrick
Abstract: The talk introduces symmetries in kernel learning, which is a mix of polynomials and representation theory with kernel methods.

Talk 3: Distributional robust optimization with semi-algebraic structure
Speaker: Guoyin Li
Abstract: Real-world optimization often deals with problems involving uncertain input data due to prediction or measurement errors. Distributional robust optimization (DRO) has emerged as an important tool in handling optimization under data uncertainty. In this talk, we show that a class of DRO problems whose loss function enjoys some suitable semi-algebraic structures can be equivalently reformulated as a conic programming problem, under moment or Wasserstein ambiguity set. If time is permitted, we will demonstrate the computational tractability and applicability of our reformulation results through numerical experiments in a portfolio optimization and option pricing model.

Speakers
XT

Xindong Tang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JK

Jack Kendrick

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GL

Guoyin Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8H: Federated optimization and learning algorithms
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Federated optimization and learning algorithms
Chair: Laurent Condat
Cluster: Optimization For Data Science

Talk 1: Stabilized Proximal-Point Methods for Federated Optimization
Speaker: Sebastian Stich
Abstract: Federated learning has emerged as an important paradigm in modern large-scale machine learning. Unlike traditional centralized learning, where models are trained using large datasets stored on a central server, federated learning keeps the training data distributed across many clients, such as phones, network sensors, hospitals, or other local information sources. In this setting, communication-efficient optimization algorithms are crucial. In this talk, we introduce a generic framework based on a distributed proximal point algorithm. This framework consolidates many of our insights and allows for the adaptation of arbitrary centralized optimization algorithms to the convex federated setting (even with acceleration). Our theoretical analysis shows that the derived methods enjoy faster convergence if the similarity among clients is high.

Talk 2: Taming Heterogeneity in Federated Linear Stochastic Approximation
Speaker: Paul Mangold
Abstract: In federated learning, multiple agents collaboratively train a machine learning model without exchanging local data. To achieve this, each agent locally updates a global model, and the updated models are periodically aggregated. In this talk, I will focus on federated linear stochastic approximation (FedLSA), with a strong focus on agents heterogeneity. I will derive upper bounds on the sample and communication complexity of FedLSA, and present a method to reduce communication cost using control variates. Particular attention will be put on the "linear speed-up" phenomenon, showing that the sample complexity scales with the inverse of the number of agents in both methods.

Talk 3: Convergence results for some federated learning algorithms
Speaker: Ming Yan
Abstract: In Federated Learning (FL), multiple nodes collaborate to solve a shared problem while keeping their private data decentralized, ensuring privacy by not transferring raw data. This process is typically framed as minimizing the average of private functions across individual nodes. The FL algorithm FedDyn is particularly effective in handling heterogeneous and non-IID data. In this talk, I will present recent advancements in FedDyn, where we relax the strong convexity requirement from individual functions to the averaged function. I will also discuss the addition of nonsmooth convex functions, where the proximal operator can be computed efficiently.

Speakers
avatar for Laurent Condat

Laurent Condat

Senior Research Scientist, King Abdullah University of Science and Technology (KAUST)
Laurent Condat received a PhD in applied mathematics in 2006 from Grenoble Institute of Technology, Grenoble, France. After a postdoc in the Helmholtz Zentrum Muenchen, Munich, Germany, he was hired in 2008 as a permanent researcher by the French National Center for Scientific Research... Read More →
SS

Sebastian Stich

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PM

Paul Mangold

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MY

Ming Yan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8I: Mobility Data Modeling: Values, Security, and Privacy
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Mobility Data Modeling: Values, Security, and Privacy
Chair: Jeff Ban
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Evaluation of data value in a transportation network
Speaker: Yueyue Fan
Abstract: Major transportation network operators, such as the California Department of Transportation, depend on good quality data to operate and manage their systems. Data could come from various sources, such as internally-owned sensors that collect traffic state information including speed, density, and volume, as well as third-party vendors that acquire, package, and sell mobile sensing data for road networks. A practical question would be: What data should the network operator acquire to best support planning and operations of a transportation network? In this talk, we will discuss the conception and modeling of the problem for understanding the value of data in the context of missing data imputation, and we will showcase its applicability using a case study based on PeMS data.

Talk 2: Data Poisoning Attack and Defense in Intelligent Transportation Systems
Speaker: Jeff Ban
Abstract: As data is becoming ubiquitous in intelligent transportation systems (ITS), data poisoning attacks emerge as a new threat. This research discusses the specific features of data poisoning attacks in ITS and proposes sensitivity-based optimization models for data poisoning. Lipschitz-based analysis methods are developed, with applications on well-studied ITS applications. Insights on how to defense data poisoning attacks are also presented with a few case studies.

Talk 3: Efficient Privacy-Preserved Processing of Multimodal Data for Vehicular Traffic Analysis
Speaker: Meisam Mohammady
Abstract: We estimate vehicular traffic states from multi- modal data collected by single-loop detectors while preserving the privacy of the individual vehicles contributing to the data. To this end, we propose a novel hybrid differential privacy (DP) approach that utilizes minimal randomization to preserve privacy by taking advantage of the relevant traffic state dynamics and the concept of DP sensitivity. Through theoretical analysis and experiments with real-world data, we show that the proposed approach significantly outperforms the related baseline non- private and private approaches in terms of accuracy and privacy preservation.

Speakers
YF

Yueyue Fan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JB

Jeff Ban

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8J: New Advances in Robust Optimization
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: New Advances in Robust Optimization
Chair: Shimrit Shtern
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Robust Extensible Bin Packing and Revisiting the Convex Knapsack Problem
Speaker: Noam Goldberg
Abstract: We study a robust extensible bin packing problem with budgeted uncertainty, an uncertainty model where item sizes are defined to lie in the intersection of a box with a one-norm ball. We propose a scenario generation algorithm for this problem, which alternates between solving a master robust bin-packing problem with finite uncertainty set and solving a separation problem. We first show that the separation is strongly NP-hard given solutions to the continuous relaxation of the master problem. Then, focusing on the separation problem for the integer master problem, we show that this problem becomes a special case of the continuous convex knapsack problem, which is known to be weakly NP-hard. Next, we prove that our special case when each of the functions is piecewise linear, having only two pieces, remains NP-hard. We develop a pseudo-polynomial dynamic program (DP) and a fully polynomial-time approximation scheme (FPTAS) for general convex knapsack with improved running times. For our special case, we develop an even faster variant of the DP and FPTAS whose running time matches that of a binary knapsack FPTAS. Finally, our computational study shows that the DP can be significantly more efficient in practice compared with solving the problem with specially ordered set (SOS) constraints using advanced mixed-integer (MIP) solvers. Our experiments also demonstrate the application of our pricing problem method to solving the robust extensible bin packing problem, including the evaluation of deferring the exact solution of the master problem, separating based on approximate master solutions in intermediate iterations.

Talk 2: Distributionally robust optimization through the lens of submodularity
Speaker: Karthik Natarajan
Abstract: Distributionally robust optimization is used to solve decision making problems under adversarial uncertainty where the distribution of the uncertainty is itself ambiguous. In this paper, we identify a class of these instances that is solvable in polynomial time by viewing it through the lens of submodularity. We discuss applications in multimarginal optimal transport and generalized moment problems  using this approach.

Talk 3: Robust Fluid Models with Adjustable Controllers
Speaker: Shimrit Shtern
Abstract: Fluid models are widely used to approximately optimize the service policy in complicated service networks, where the stochasticity is replaced by arrival and service rates. These models can be cast as structured semicontinuous linear programs (SCLP) which can be solved using dedicated simplex based solvers. Uncertainty can appear in these models when the arrival and service rates are not known exactly. In the literature, such models are addressed via robust optimization. These robust optimization models restrict the controller, determining the service policy at each point in time, to be set in the start of the planning horizon, and do not allow to change it based on revealed information about the uncertainty. We suggest a novel partially adaptive model, in which we can change the controller based on the state of the system in predefined time points, and reformulate the resulting problem as a SCLP for various uncertainty sets.

Speakers
NG

Noam Goldberg

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KN

Karthik Natarajan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SS

Shimrit Shtern

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8K: Learning in PDE-based optimization and control
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Learning in PDE-based optimization and control
Chair: Michael Hintermüller
Cluster: PDE-constrained Optimization

Talk 1: Efficient Computational Methods for Wasserstein Natural Gradient Descent
Speaker: Levon Nurbekyan
Abstract: Natural gradient descent (NGD) is a preconditioning technique that incorporates the geometry of the forward model space to accelerate gradient-based optimization techniques in inverse and learning problems. One such relevant geometry is the optimal transportation (OT) or the Wasserstein geometry, which is useful when recovering or learning probability measures. One of the critical challenges in NGD is the preconditioning cost. If performed naively, this cost is particularly taxing for the OT geometry due to the high computational cost of OT distances. In this talk, I'll present an efficient way of performing large-scale NGD with a particular emphasis on OT geometry. The talk is based on a joint work with Yunan Yang (Cornell) and Wanzhou Lei (Brown Grad School).

Talk 2: Derivative-informed neural operators for efficient PDE-constrained optimization under uncertainty
Speaker: Dingcheng Luo
Abstract: We consider the use of derivative-informed neural operators (DINOs) as surrogates for PDE-constrained optimization subject to uncertainty in the model parameters. Optimization under uncertainty (OUU) is often orders of magnitude more expensive compared to its deterministic counterpart due to the need to evaluate statistical/risk measures by stochastic integration, requiring a large number PDE solves. To address this challenge, we propose a neural operator surrogate for the underlying PDE, which is trained on derivatives of the solution operator. This ensures that the neural operator can be used to accurately approximate the cost function and it’s gradient with respect to the optimization variable, thereby improving its fitness for OUU tasks. We present some supporting theoretical results and demonstrate the performance of our method over numerical experiments, showcasing how DINOs can be used to solve OUU problems in a sample efficient manner.

Talk 3: A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem
Speaker: Denis Korolev
Abstract: The physics-informed neural network (PINN) approach relies on approximating the solution to a partial differential equation (PDE) using a neural network by solving an associated non-convex and highly nonlinear optimization task. Despite the challenges of such an ansatz, the optimization-based formulation of PINNs provides rich flexibility and holds great promise for unifying various techniques into monolithic computational frameworks. Inspired by the Liquid Composite Molding process for fiber-reinforced composites and its related multiscale fluid flow structure, we present a novel framework for optimizing PINNs constrained by partial differential equations, with applications to multiscale PDE systems. Our hybrid approach approximates the fine-scale PDE using PINNs, producing a PDE residual-based objective subject to a coarse-scale PDE model parameterized by the fine-scale solution. Multiscale modeling techniques introduce feedback mechanisms that yield scale-bridging coupling, resulting in a non-standard PDE-constrained optimization problem. From a discrete standpoint, the formulation represents a hybrid numerical solver that integrates both neural networks and finite elements, for which we present a numerical algorithm. The latter combines the natural gradient descent technique for optimizing PINNs with the adjoint-state method, resulting in a Newton-type optimization update for our hybrid solver. We further demonstrate that our hybrid formulation enchances the overall modelling and can substantially improve the convergence properties of PINNs in the context of material science applications. The talk is based on a joint work with Michael Hintermüller (Weierstrass Institute, Humboldt University of Berlin).

Speakers
LN

Levon Nurbekyan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DL

Dingcheng Luo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DK

Denis Korolev

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8L: Derivative-free optimization for special classes of problems II
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Derivative-free optimization for special classes of problems II
Chair: Ana Luísa Custódio
Cluster: Derivative-free Optimization

Talk 1: A direct multisearch approach for many-objective derivative-free optimization
Speaker: Ana Luísa Custódio
Abstract: From a theoretical point of view, Direct Multisearch (DMS) was developed for continuous constrained multiobjective derivative-free optimization, with a general number of components in the objective function. However, the algorithmic performance was never assessed in problems with more than three objectives. In this work, we propose DMS-Reduction, a variant of DMS based on reduction approaches, using correlation and sketching techniques. This approach is an attempt to tackle larger problems, in what respects the number of components of the objective function and the number of variables. At each iteration, the reduction in the number of components of the objective function to be optimized has the possible additional benefit of conducting to a reduction in the number of variables to be considered, since there could be the case that not all variables are related to the objective function components selected. We will detail the proposed algorithmic structure and report promising numerical results in addressing many-objective optimization problems.

Talk 2: Optimal zeroth-order methods for bilevel optimization
Speaker: Saeed Ghadimi
Abstract: In this talk, we present fully zeroth-order stochastic approximation algorithms for solving stochastic bilevel optimization problems assuming that neither the upper/lower loss functions, nor their unbiased gradient estimates are available. To do so, we first generalize the Gaussian convolution technique to the functions with two block variables and establish all corresponding relationships between such functions and their smoothed Gaussian approximations. By using these results, we estimate the first- and second-order derivatives of the objective functions and provide a fully zeroth-order double-loop algorithm whose sample complexity is optimal in terms of dependence on the target accuracy while polynomially dependent on the problem dimension. Furthermore, by using recent developments in designing fully first-order methods for bilevel optimization, we provide our second fully zeroth-order bilevel optimization algorithm whose sample complexity is optimal in terms of both the target accuracy and the problem dimension.

Talk 3: Derivative-free Frank-Wolfe recursion for optimization over probability spaces
Speaker: Raghu Pasupathy
Abstract: It has recently been observed that a number of important problem classes, e.g., statistical experimental design, continuous linear programming, and DRO, are usefully seen as constrained optimization problems over the space of probability measures. Various aspects, including the fact that the space of probability measures is not a linear space, make a primal recursion such as Frank-Wolfe (suitably adapted to function on probability spaces) a natural choice for solution. In this talk, we present a framework that suitably combines interior point ideas along with a Frank-Wolf recursion to minimize linearly constrained convex functionals over probability spaces. We emphasize a derivative-free version of this framework that estimates the von Mises derivative through an analogue of finite-differencing in the Euclidean context. We will illustrate through several examples and numerical experiments.

Speakers
AL

Ana Luísa Custódio

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SG

Saeed Ghadimi

University of Waterloo
RP

Raghu Pasupathy

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8M: Techniques for Solving High-Dimensional and Nonlinear Optimization Problems
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Techniques for Solving High-Dimensional and Nonlinear Optimization Problems
Chair: Wenqi Zhu
Cluster: Nonlinear Optimization

Talk 1: Solving exact and noisy rank-one tensor completion with semidefinite programming
Speaker: Zhuorui Li
Abstract: Consider the problem of retrieving a d-th order rank-one tensor given (exact or noisy) values of some of its entries. We address this problem via semidefinite programming (SDP). Concretely, we propose SDP relaxations involving a N-by-N matrix variable where N is the number of entries of the tensor. We show that our SDPs solve the exact and noisy rank-one tensor completion problems when the observations satisfy a deterministic combinatorial condition, which we call square restricted propagation. We prove that the square restricted propagation condition is satisfied with high probability when the number of uniformly random observations is more than certain quantity. Moreover, for matrix completion, the square restricted propagation is satisfied precisely when the completion problem admits a unique solution. Preliminary computational experiments show that our methods allow to solve tensor completion problems using significantly less observations than alternative methods.

Talk 2: Numerical Solution for Nonlinear 4D Variational Data Assimilation (4D-Var) via ADMM
Speaker: Bowen Li
Abstract: The four-dimensional variational data assimilation (4D-Var) has emerged as an important methodology, widely used in numerical weather prediction, oceanographic modeling, and cli- mate forecasting. Classical unconstrained gradient-based algorithms often struggle with local minima, making their numerical performance highly sensitive to the initial guess. In this study, we exploit the separable structure of the 4D-Var problem to propose a practical variant of the alternating direction method of multipliers (ADMM), referred to as the linearized multi-block ADMM with regularization. Unlike classical first-order optimization methods that primarily focus on initial conditions, our approach derives the Euler-Lagrange equation for the entire dynamical system, enabling more comprehensive and effective utilization of observational data. When the initial condition is poorly chosen, the argmin operation steers the iteration towards the observational data, thereby reducing sensitivity to the initial guess. The quadratic subproblems further simplify the solution process, while the parallel structure enhances computational efficiency, especially when utilizing modern hardware. To validate our approach, we demonstrate its superior performance using the Lorenz system, even in the presence of noisy observational data. Furthermore, we showcase the effectiveness of the linearized multi-block ADMM with regularization in solving the 4D-Var problems for the viscous Burgers’ equation, across various numerical schemes, including finite difference, finite element, and spectral methods. Finally, we illustrate the recovery of dynamics under noisy observational data in a 2D turbulence scenario, particularly focusing on vorticity concentration, highlighting the robustness of our algorithm in handling complex physical phenomena.

Talk 3: Generalized Nash equilibrium problems with quasi-linear constraints
Speaker: Jiyoung Choi
Abstract: We discuss generalized Nash equilibrium problems (GNEPs) that are given by polynomial functions. We consider cases each player's constraints are linear in their own strategy vectors. For this kind of GNEPs, the KKT set can be conveniently represented as a union of simpler sets, by using partial Lagrange multiplier expressions. This representation produces a set of branch polynomial optimization problems, each of which can be conveniently solved by Moment-SOS relaxations. We give a method to compute all generalized Nash equilibria, under some genericity assumptions. Extensive numerical experiments are provided to demonstrate the efficiency of the proposed method.

Speakers
WZ

Wenqi Zhu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZL

Zhuorui Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Bowen Li

Bowen Li

Name: Bowen LiPhD CandidateAffiliation: Academy of Mathematics and Systems Sciences, Chinese Academy of SciencesBio:Bowen Li is currently working toward the Ph.D. degree at the Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China. His research interests... Read More →
JC

Jiyoung Choi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8N: Recent advances in first-order methods
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Recent advances in first-order methods
Chair: Lexiao Lai
Cluster: Nonlinear Optimization

Talk 1: Heavy-ball ODE converges at rate $O(T^{-4/7})$ with averaging for nonconvex functions
Speaker: Naoki Marumo
Abstract: First-order optimization methods for nonconvex functions with Lipschitz continuous gradients and Hessian have been studied intensively. State-of-the-art methods finding an $\varepsilon$-stationary point within $O(\varepsilon^{-{7/4}})$ or $\tilde{O}(\varepsilon^{-{7/4}})$ gradient evaluations are based on Nesterov's accelerated gradient descent (AGD) or Polyak's heavy-ball method. However, these algorithms employ additional mechanisms, such as restart schemes and negative curvature exploitation, which complicate the algorithms' behavior and make it challenging to apply them to more advanced settings (e.g., stochastic optimization). To realize a simpler algorithm, we investigate the heavy-ball differential equation, a continuous-time analogy of the AGD and heavy-ball methods; we prove that the dynamics attains an $\varepsilon$-stationary point within $O(\varepsilon^{-{7/4}})$ time.

Talk 2: On Optimal Smoothings of Convex Functions and Sets
Speaker: Thabo Samakhoana
Abstract: A nonsmooth objective function can be optimized at an improved rate by optimizing its smooth approximation at an accelerated rate. Given that better smoothings lead to improved convergence guarantees, it is natural to seek the best possible smoothing. In this work, we provide a theoretical study of optimal smoothings of functions and sets. In particular, we characterize the set of optimal smoothings of sublinear functions and cones. We also provide a framework of extending conic smoothings to functions and conic sections.

Talk 3: Inexact subgradient methods for semialgebraic functions
Speaker: Tam Le
Abstract: Motivated by the widespread use of approximate derivatives in machine learning and optimization, we study inexact subgradient methods with non-vanishing additive errors and step sizes. In the nonconvex semialgebraic setting, under boundedness assumptions, we prove that the method provides points that eventually fluctuate close to the critical set at a distance proportional to ρ where ρ is the error in subgradient evaluation and ρ relates to the geometry of the problem. In the convex setting, we provide complexity results for the averaged values. We also obtain byproducts of independent interest, such as descent-like lemmas for nonsmooth nonconvex problems and some results on the limit of affine interpolants of differential inclusions.

Speakers
NM

Naoki Marumo

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TS

Thabo Samakhoana

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TL

Tam Le

Assistant Professor, Université Paris Cité - LPSM
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8O: Optimization problems arising in quantum computing and physics
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Optimization problems arising in quantum computing and physics
Chair: Ojas Parekh
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: The quantum nature of classical problems in topological data analysis
Speaker: Sankara Sai Chaithanya Rayudu
Abstract: Recent work has exposed unexpected connections between the complexity of problems in topological data analysis (TDA) and quantum local Hamiltonian problems. We show that a natural TDA problem, gapped homology in independence complexes, is QMA-complete, where QMA is a quantum version of the complexity class NP. We accomplish this by introducing a related new quantum (fermionic) local Hamiltonian version of the classical independent set problem.

Talk 2: Second order cone relaxations for Quantum Max Cut
Speaker: Felix Huber
Abstract: Quantum Max Cut (QMC), also known as the quantum anti-ferromagnetic Heisenberg model, is a QMA-complete problem relevant to quantum many-body physics and computer science. Semidefinite programming relaxations have been fruitful in designing theoretical approximation algorithms for QMC, but are computationally expensive for systems beyond tens of qubits. We give a second order cone relaxation for QMC, which optimizes over the set of mutually consistent three-qubit reduced density matrices. In combination with Pauli level-1 of the quantum Lasserre hierarchy, the relaxation achieves an approximation ratio of 0.526 to the ground state energy. Our relaxation is solvable on systems with hundreds of qubits and paves the way to computationally efficient lower and upper bounds on the ground state energy of large-scale quantum spin systems.

Talk 3: Generalized Quantum State Discrimination Tasks
Speaker: Jamie Sikora
Abstract: Quantum state discrimination is a central task in many quantum computing settings where one wishes to identify what quantum state they are holding. We introduce a framework that generalizes many of its variants and present a hybrid quantum-classical algorithm based on semidefinite programming to calculate the maximum reward when the states are pure and have efficient circuits. Using this, optimal bounds for antidistinguishability will also be presented. This talk is based on arXiv:2312.04023 and arxiv:2311.17047.

Speakers
SS

Sankara Sai Chaithanya Rayudu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FH

Felix Huber

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JS

Jamie Sikora

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8P: Algebraic Methods in Optimization (Part 1)
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Algebraic Methods in Optimization (Part 1)
Chair: Shixuan Zhang
Cluster: Conic and Semidefinite Optimization

Talk 1: Spurious minima in nonconvex sum-of-square optimization via syzygies
Speaker: Shixuan Zhang
Abstract: We study spurious local minima in a nonconvex low-rank formulation of sum-of-squares optimization on a real variety X. We reformulate the problem of finding a spurious local minimum or stationary points in terms of syzygies of the underlying linear series, and also bring in topological tools to study this problem. When the variety X is of minimal degree, there exist spurious second-order stationary points if and only if both the dimension and the codimension of the variety are greater than one, answering a question by Legat, Yuan, and Parrilo. Moreover, for surfaces of minimal degree, we provide sufficient conditions to exclude points from being spurious local minima. In particular, we characterize all spurious second-order stationary points on the Veronese surface, corresponding to ternary quartics, which either have finitely many Gram matrices or can be written as a binary quartic, complementing work by Scheiderer on decompositions of ternary quartics as a sum of three squares. For general varieties of higher degree, we give examples and characterizations of spurious local minima, and provide extensive numerical experiments demonstrating the effectiveness of the low-rank formulation.

Talk 2: Optimizing rational neural networks with trainable parameters
Speaker: Jiayi Li
Abstract: In the modern practice of deep learning, the performance of an artificial neural network is heavily dependent on the architecture and the choice of non-linear activations between each layer. We consider feedforward neural networks with activations being rational functions, whose coefficients are trainable. We characterize the critical points and study the geometry of the optimization landscape. This is joint work with Angelica Torres and Guido Montufar.

Talk 3: Pythagoras Numbers of Ternary Forms
Speaker: Alex Dunbar
Abstract: The representation of nonnegative polynomials as sums of squares is a fundamental idea in polynomial optimization. We study the \emph{Pythagoras number} $py(3,2d)$ of real ternary forms, defined as the minimal number $r$ such that every degree $2d$ form which is a sum of squares can be written as the sum of at most $r$ squares of degree $d$ forms. It is well-known that $d+1\leq py(3,2d)\leq d+2$. We show that $py(3,2d) = d+1$ for $2d = 8,10,12$. The main technical tool is Diesel's characterization of height 3 Gorenstein algebras. Based on joint work with Greg Blekherman and Rainer Sinn

Speakers
SZ

Shixuan Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jiayi Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8Q: Splitting methods in multi-agent optimization
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Splitting methods in multi-agent optimization
Chair: Felipe Atenas
Cluster: Multi-agent Optimization and Games

Talk 1: Decentralized forward-backward splitting methods for composite monotone inclusions
Speaker: David Torregrosa-Belén
Abstract: Splitting algorithms are a class of numerical methods adept at solving the monotone inclusion problem consisting in finding a zero in the sum of a finite number of monotone operators. Until very recently the resolution of problems involving a large number of operators was limited to the use of product space reformulations. In general, these reformulations are not appropriate for distributed implementations, as they usually require the computation of a global sum across all nodes in every iteration. More recently, new splitting schemes suitable for different network topologies have been proposed. In this talk, we will discuss a new family of forward-backward splitting algorithms whose communication pattern is induced by the choice of multiple graphs.

Talk 2: An efficient single loop algorithm for a new class of solvable GNEPs
Speaker: Puya Latafat
Abstract: Despite their extensive use in real-world applications, generalized Nash equilibrium problems (GNEPs) remain challenging to solve. Most existing approaches rely on stringent assumptions or involve double-loop procedures. In this work, we identify a novel structure for the constraint set map that fundamentally differs from Rosen's framework. Additionally, we develop a single-loop algorithm to solve the corresponding GNEP efficiently. 

Talk 3: A distributed proximal splitting method with linesearch for locally Lipschitz data
Speaker: Felipe Atenas
Abstract: We consider finitely many agents over a connected network working cooperatively to solve a consensus optimization problem. Each agent owns a private convex cost function with a decomposable structure given by the sum of two terms, one smooth and one nonsmooth. In our decentralized setting, no agent has direct access to the information of the overall network, but instead they can only communicate with their neighbors in the network. We propose a distributed primal-dual splitting method of proximal-gradient type with backtracking linesearch with convergence guarantees to minimizers. Our approach allows gradients to be only locally Lipschitz, relaxing the common assumption of existing methods that require global Lipschitz continuity and predefined stepsizes, making it suitable for problems where global constants are unavailable or difficult to compute.

Speakers
DT

David Torregrosa-Belén

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PL

Puya Latafat

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FA

Felipe Atenas

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8R: Algorithms for structured Riemannian optimization problems I
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Algorithms for structured Riemannian optimization problems I
Chair: Jiang Hu
Cluster: Optimization on Manifolds

Talk 1: Bridging Convex and Nonconvex Approaches: Geodesic Strong Convexity of QCQPs from Tightness and Strict Complementarity of SDR
Speaker: Jinxin Wang
Abstract: Nonconvex quadratically constrained quadratic programs (QCQPs) has received increasing attention in recent decades. Two typical approaches to tackling such challenging problems are convex semidefinite relaxation (SDR) and nonconvex iterative methods (e.g., gradient descent). Although the convex SDR and nonconvex approaches have been extensively studied, they have been largely explored in parallel, and few things can be said about their connections. In this talk, we take the first step in building connections between convex and nonconvex approaches. Under the tightness and strict complementarity of SDR, we find that nonconvex QCQPs admit nice geometric properties, including quadratic growth, error bound, and even (quotient) geodesic strong convexity. These geometric properties quantify the growth behavior of optimization problems around global optimal solution sets and can imply fast convergence rates of various nonconvex iterative methods. This is a joint work with Huikang Liu, Chonghe Jiang, and Anthony Man-Cho So.

Talk 2: Rotation Group Synchronization via Quotient Manifold
Speaker: Linglingzhi Zhu
Abstract: Rotation group synchronization is a significant inverse problem and has attracted intense attention from numerous application fields such as graph realization, computer vision, and robotics. In this talk, we focus on the least squares estimator of rotation group synchronization with general additive noise, which is a nonconvex optimization problem with manifold constraints. Departing from the standard approach of utilizing the geometry of the ambient Euclidean space to study phase/orthogonal group synchronization, we adopt an intrinsic Riemannian approach to study rotation group synchronization. Benefiting from a quotient geometric view, we prove the negative definite condition of quotient Riemannian Hessian around the optimal solution of orthogonal group synchronization problem. Consequently, the Riemannian local error bound property holds and can be applied to analyze the convergence properties of various Riemannian algorithms. On the other hand, improved estimation results of the spectral and least squares estimator are derived, which guarantee the tightness of orthogonal group synchronization for solving rotation group version under certain noise level. The sequential convergence guarantee of the Riemannian (quotient) gradient method for solving orthogonal/rotation group synchronization problem is studied and we derive its linear convergence rate to the optimal solution with the spectral initialization. All results are deterministic without any probabilistic model.

Talk 3: Randomized Submanifold Subgradient Method for Optimization over Stiefel Manifolds
Speaker: Andy Cheung
Abstract: Optimization over Stiefel manifolds has found wide applications in many scientific and engineering domains. Despite considerable research effort, high-dimensional optimization problems over Stiefel manifolds remain challenging, and the situation is exacerbated by nonsmooth objective functions. The purpose of this paper is to propose and study a novel coordinate-type algorithm for weakly convex (possibly nonsmooth) optimization problems over high-dimensional Stiefel manifolds, named randomized submanifold subgradient method (RSSM). Similar to coordinate-type algorithms in the Euclidean setting, RSSM exhibits low per-iteration cost and is suitable for high-dimensional problems. We prove that RSSM converges to the set of stationary points and attains ε-stationary points with respect to a natural stationarity measure in (ε−4) iterations in both expectation and the almost-sure senses. To the best of our knowledge, these are the first convergence guarantees for coordinate-type algorithms to optimize nonconvex nonsmooth functions over Stiefel manifolds. An important technical tool in our convergence analysis is a new Riemannian subgradient inequality for weakly convex functions on proximally smooth matrix manifolds, which could be of independent interest.

Speakers
JH

Jiang Hu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Jinxin Wang

Jinxin Wang

Postdoc, University of Chicago
Name: Dr. Jinxin WangTitle: Bridging Convex and Nonconvex Approaches: Geodesic Strong Convexity of QCQPs from Tightness and Strict Complementarity of SDRAffiliation: University of Chicago Booth School of BusinessBio:Dr. Jinxin Wang works in nonconvex optimization (e.g., QCQPs, manifold... Read More →
avatar for Linglingzhi Zhu

Linglingzhi Zhu

Postdoctoral Fellow, Georgia Institute of Technology
Name: Dr. Linglingzhi ZhuTitle: Postdoctoral FellowAffiliation: Georgia Institute of TechnologyBio:Dr. Linglingzhi Zhu's research is grounded in the field of mathematical optimization and its interplay with variational analysis, Riemannian geometry, and high-dimensional statistical... Read More →
AC

Andy Cheung

Instructor, The Hong Kong Polytechnic University
I am currently an instructor in the Department of Applied Mathematics at The Hong Kong Polytechnic University. Alongside my teaching role, I pursued a part-time Ph.D. at The Chinese University of Hong Kong from 2017 to 2025, in the Department of Systems Engineering and Engineering... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8S: Recent advances in mixed-integer programming
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Recent advances in mixed-integer programming
Chair: Linchuan Wei
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Serial and Parallel Two-Column Probing for Mixed-Integer Programming
Speaker: Yongzheng Dai
Abstract: Probing in mixed-integer programming (MIP) is a technique of temporarily fixing variables to discover implications that are useful to branch-and-cut solvers. Such fixing is typically performed one variable at a time— this paper develops instead a two-column probing scheme that instead fixes a pair of variables per iteration. Although the scheme involves more work per iteration compared to the one-column approach, stronger implied bounds as well as more conflicts identified may compensate. Indeed, our prototype implementation was awarded first prize at the MIP Workshop 2024 Computational Competition on novel presolving approaches. This paper presents the aforementioned (serial) prototype and additionally develops an efficient parallelization, leveraging hardware acceleration to further improve overall solve times. Compared to serial two-column probing, our parallel version sacrifices some strength per-pair probed in exchange for greatly increasing the total number of such probings; computational experiments demonstrate its promise.

Talk 2: Integer Programming for Learning Directed Acyclic Graphs from Non-identifiable Gaussian Models
Speaker: Tong Xu
Abstract: We study the problem of learning directed acyclic graphs from continuous observational data, generated according to a linear Gaussian structural equation model. State-of-the-art structure learning methods for this setting have at least one of the following shortcomings: i) they cannot provide optimality guarantees and can suffer from learning sub-optimal models; ii) they rely on the stringent assumption that the noise is homoscedastic, and hence the underlying model is fully identifiable. We overcome these shortcomings and develop a computationally efficient mixed-integer programming framework for learning medium-sized problems that accounts for arbitrary heteroscedastic noise. We present an early stopping criterion under which we can terminate the branch-and-bound procedure to achieve an asymptotically optimal solution and establish the consistency of this approximate solution. In addition, we show via numerical experiments that our method outperforms state-of-the-art algorithms and is robust to noise heteroscedasticity, whereas the performance of some competing methods deteriorates under strong violations of the identifiability assumption. The software implementation of our method is available as the Python package \emph{micodag}.

Talk 3: Disjunctive Sum of Squares
Speaker: Yixuan Hua
Abstract: We introduce the concept of disjunctive sum of squares for certifying nonnegativity of polynomials. Unlike the popular sum of squares approach where nonnegativity is certified by a single algebraic identity, the disjunctive sum of squares approach certifies nonnegativity with multiple algebraic identities. Our main result is a disjunctive Positivstellensatz proving that we can keep the degree of each algebraic identity as low as the degree of the polynomial whose nonnegativity is in question. Based on this result, we construct a semidefinite programming based converging hierarchy of lower bounds for the problem of minimizing a polynomial over a compact basic semialgebraic set, where the size of the largest semidefinite constraint is fixed throughout the hierarchy. We further prove a second disjunctive Positivstellensatz which leads to an optimization-free hierarchy for polynomial optimization. We specialize this result to the problem of proving copositivity of matrices. Finally, we describe how the disjunctive sum of squares approach can be combined with a branch-and-bound algorithm and we present numerical experiments on polynomial, copositive, and combinatorial optimization problems.

Speakers
YD

Yongzheng Dai

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TX

Tong Xu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Yixuan Hua

Yixuan Hua

Princeton University
I am a third year PhD student in the Operations Research and Financial Engineering (ORFE) department at Princeton University, co-advised by Prof. Amir Ali Ahmadi and Prof. Bartolomeo Stellato. My research focuses on mathematical optimization, with various applications in machine learning... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8T: Advanced Techniques in Global Optimization
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Advanced Techniques in Global Optimization
Chair: Sonia Cafieri
Cluster: Global Optimization

Talk 1: Towards McCormick Envelopes for Mixed-integer PDE-constrained Optimization
Speaker: Sven Leyffer
Abstract: McCormick envelopes are a well-known standard tool for deriving convex relaxations that can, for example, be used in branch-and-bound procedures for mixed-integer nonlinear programs but have not gained much attention in PDE-constrained optimization so far. We analyze McCormick envelopes for a model problem class that is governed by a semilinear PDE involving a bilinearity and integrality constraints. We approximate the McCormick nonlinearity and in turn the McCormick envelopes by averaging the involved terms over the cells of a partition of the computational domain on which the PDE is defined. The resulting approximate McCormick relaxations can be improved by means of an optimization-based bound-tightening procedure. We prove that their minimizers converge to minimizers to a limit problem with a pointwise formulation of the McCormick envelopes when driving the mesh size to zero.

Talk 2: An Adaptive Proximal ADMM for Nonconvex Composite Programs
Speaker: Leandro Maia
Abstract: We developed an adaptive Proximal Alternating Direction Method of Multipliers (P-ADMM) for solving linearly-constrained, weakly convex, composite optimization problems. This method is adaptive to all problem parameters, including smoothness and weak convexity constants. Without any rank assumptions on the constraint matrices, it is shown that the adaptive P-ADMM obtains an approximate first-order stationary point of the constrained problem in a number of iterations that matches the state-of-the-art complexity for the class of P-ADMMs.

Talk 3: On the computation of upper bounds for some structured nonlinear minimization problems
Speaker: Sonia Cafieri
Abstract: We focus on nonlinear minimization problems that share a common structure, namely a combinatorial aspect that comes only from disjunctive constraints. For this kind of constraints, usually addressed introducing auxiliary binary variables and then handling mixed-integer formulations, a continuous-optimization alternative named 'continuous quadrant penalty formulation' was recently introduced. It is based on the introduction of a smooth piecewice-quadratic penalty function, and yields a continuous nonconvex problem. We build on this problem, to derive an efficient computation of upper bounds to be used within Branch-and-Bound-based approaches for problems arising in different domains of application. Numerical experiences show the effectiveness of these approaches at speeding up the computational convergence.

Speakers
LM

Leandro Maia

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SC

Sonia Cafieri

Professor, ENAC, Université de Toulouse
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8U: Splitting Algorithms
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Splitting Algorithms
Chair: Rubén Campoy
Cluster: Nonsmooth Optimization

Talk 1: Graph splitting methods: Fixed points and strong convergence for linear subspaces
Speaker: César López-Pastor
Abstract: In this talk, we develop a general analysis for the fixed points of the operators defining the graph splitting methods from [SIAM J. Optim., 34 (2024), pp. 1569–1594] by Bredies, Chenchene and Naldi. We particularize it to the case where the maximally monotone operators are normal cones of closed linear subspaces and provide an explicit formula for the limit points of the graph splitting schemes. We exemplify these results on some particular algorithms, unifying in this way some results previously derived as well as obtaining new ones.

Talk 2: Coordinating a virtual power plant with a decentralised forward-backward-type algorithm with independent step sizes
Speaker: Liam Timms
Abstract: A wide range of problems can be expressed as finding a zero of a finite sum of set-valued monotone operators and Lipschitz continuous monotone operators. Splitting algorithms are powerful methods for solving this type of problem, and some are suitable for a distributed implementation. However, these distributed splitting algorithms require potentially restrictive assumptions such as cocoercivity, a centrally coordinating agent in the distributed implementation, and/or agents' step sizes being constrained by the communication graph topology. We propose a decentralised forward-backward-type algorithm that does not require additional assumptions about the forward operators and allows independent agent step sizes. Each iteration of this algorithm uses a backward evaluation of the set-valued operators and a reflected forward evaluation of the single-valued operators, which only requires one new evaluation of the forward operator per iteration. To demonstrate the utility of this algorithm, we develop a model for coordinating a network of independent power banks and find optimal charging/discharging schedules using our algorithm.

Talk 3: Forward-backward algorithms devised by graphs
Speaker: Rubén Campoy
Abstract: In this talk, we present a methodology for devising forward-backward methods for finding zeros in the sum of a finite number of maximally monotone operators. We extend the framework and techniques from [SIAM J. Optim., 34 (2024), pp. 1569–1594] to cover the case involving a finite number of cocoercive operators, which should be directly evaluated instead of computing their resolvent. The algorithms are induced by three graphs that determine how the algorithm variables interact with each other and how they are combined to compute each resolvent. The hypotheses on these graphs ensure that the algorithms obtained have minimal lifting and are frugal, meaning that the ambient space of the underlying fixed point operator has minimal dimension and that each resolvent and each cocoercive operator is evaluated only once per iteration. This framework not only allows to recover some known methods, but also to generate new ones, as the forward-backward algorithm induced by a complete graph. We conclude with a numerical experiment showing how the choice of graphs influences the performance of the algorithms.

Speakers
CL

César López Pastor

PhD Student, Universidad de Alicante
LT

Liam Timms

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RC

Rubén Campoy

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8V: Quantum Computing and continuous optimization
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Quantum Computing and continuous optimization
Chair: David Bernal Neira
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: QHDOPT: A Software for Nonlinear Optimization with Quantum Hamiltonian Descent
Speaker: Yuxiang Peng
Abstract: We develop an open-source, end-to-end software (named QHDOPT), which can solve nonlinear optimization problems using the quantum Hamiltonian descent (QHD) algorithm. QHDOPT offers an accessible interface and automatically maps tasks to various supported quantum backends (i.e., quantum hardware machines). These features enable users, even those without prior knowledge or experience in quantum computing, to utilize the power of existing quantum devices for nonlinear and nonconvex optimization tasks. In its intermediate compilation layer, QHDOPT employs SimuQ, an efficient interface for Hamiltonian-oriented programming, to facilitate multiple algorithmic specifications and ensure compatible cross-hardware deployment. The detailed documentation of QHDOPT is available at https://github.com/jiaqileng/QHDOPT.

Talk 2: Minimization of Multi-variate Polynomials Using Entropy Computing
Speaker: Wesley Dyk
Abstract: The computing paradigm known as entropy quantum computing (EQC) has the capability of minimizing multi-variate polynomials. This paradigm operates by conditioning a quantum reservoir to promote the stabilization of the ground state in a quantum optical system. By mapping a polynomial to a Hamiltonian operator representing the total energy of a quantum system, the polynomial can be minimized using the optical quantum system as an analog computer. In one encoding scheme, variables are mapped to qudits, which exist in the photon number Hilbert space. Qudits are valued using normalized photon counts. This encoding scheme produces an approximation of continuous values for the variables, with an additional restriction that all variables sum to a user-specified total. We demonstrate this paradigm and encoding scheme together to solve QCL (quadratic objective, continuous variables, linear constraints) class problems from QPLIB of up to 500 variables. In many applications of model predictive control, relationships are simplified to accommodate the available solver tools. We demonstrate how EQC can overcome some limitations in solving complex models and then be used in just-in-time optimal control applications.

Talk 3: Non-convex Continuous Optimization Using Coherent Optical Networks
Speaker: Pooya Ronagh
Abstract: Analog computing using photonic integrated circuits (PIC) is a leading approach to surpassing the computational speed and energy limitations of von Neumann architectures. The challenges of manufacturing large-scale PICs have led to hybrid solutions that integrate optical analog and electronic digital components. A notable example is the coherent Ising machine (CIM), initially designed for solving quadratic binary optimization problems. We reinterpret the dynamics of optical pulses in the CIM as solutions to Langevin dynamics, a stochastic differential equation (SDE) that plays a key role in non-convex optimization and generative AI. This interpretation establishes a computational framework for understanding the system’s operation, the critical role of each component, and its performance, strengths, and limitations. Notably, we demonstrate that the CIM is inherently a continuous solver, capable of being extended to solve more general SDEs. Finally, we observe that the iterative digital-to-analog and analog-to-digital conversions within the protocol create a bottleneck for the low power and high speed of optics to shine and envision that fully analog opto-electronic realizations of such experiments can open doors for broader applications, and orders of magnitude improvements in speed and energy consumption.

Speakers
DB

David Bernal Neira

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YP

Yuxiang Peng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WD

Wesley Dyk

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PR

Pooya Ronagh

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8W: Bilevel Optimization - Algorithms
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Bilevel Optimization - Algorithms
Chair: Yuchen Lou
Cluster: nan

Talk 1: A Fully First-Order Method for Stochastic Bilevel Optimization
Speaker: Dohyun Kwon
Abstract: The problem of stochastic bilevel optimization has been the focus of extensive study in recent years. Although many optimization methods have been proposed to address bilevel problems, existing approaches often require potentially expensive calculations involving the Hessians of lower-level objectives. The primary technical challenge lies in tracking the lower-level solutions as upper-level variables change. We introduce a Fully First-order Stochastic Approximation (F2SA) method [1, 2], which only relies on first-order gradient oracles. Additionally, we analyze the complexity of first-order methods under minimal assumptions and provide matching lower bounds [3]. This talk is based on joint work with Jeongyeol Kwon, Hanbaek Lyu, Stephen Wright, and Robert Nowak (UW-Madison). [1] Kwon, J., Kwon, D., Wright, S., & Nowak, R. D. (2023). A fully first-order method for stochastic bilevel optimization. In Proceedings of the 40th International Conference on Machine Learning (pp. 18083-18113). PMLR. (ICML 2023, Oral) [2] Kwon, J., Kwon, D., Wright, S., & Nowak, R. D. (2024). On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation. In The Twelfth International Conference on Learning Representations. (ICLR 2024, Spotlight) [3] Kwon, J., Kwon, D., & Lyu, H. (2024). On the complexity of first-order methods in stochastic bilevel optimization. In Proceedings of the 41st International Conference on Machine Learning (pp. 25784-25811). PMLR. (ICML 2024)

Talk 2: Scalable Subspace Minimization for Parameter Efficient Fine-Tuning in LLM
Speaker: Yuchen Lou
Abstract: Given the growing sizes of language models, there is a pressing need for memory-efficient optimizers that enable scalable training and fine-tuning. A common strategy is to restrict updates to a small subset of parameters, significantly reducing memory usage. We propose a unified framework for such optimizers, specifically designed for Parameter-Efficient Fine-Tuning (PEFT). Grounded in classical subspace minimization, our approach achieves superior memory and computational efficiency compared to existing PEFT methods such as LoRA and GaLore. Crucially, it provides theoretical convergence guarantees---an element largely missing in current PEFT literature due to challenges arising from degenerate updates. Drawing on insights into the intrinsic dimensionality of fine-tuning, we introduce a novel algorithm that integrates well-established subspace optimization techniques with streaming PCA and a low-rank gradient restart mechanism. Our method matches state-of-the-art performance while significantly reducing memory overhead.

Talk 3: A gradient-based method for bilevel optimization with convex lower-level problems
Speaker: Mattia Solla Saez
Abstract: This talk presents an algorithm to compute approximate solutions to a class of bilevel optimization problems. Recent advances in variational analysis have shown that the solution mapping of certain generalized equations is continuously differentiable at their nondegenerate solutions. We use this result to construct a sequence of approximate solutions of an appropriate regularization of the problem that converges to a feasible point of the original problem. Under an additional second-order assumption, the sequence is shown to converge to a stationary point of the original problem. Our framework allows for a wide class of nonsmooth problems in the lower level, including (convex) nonlinear programming problems and (convex) minimax problems. We show that, in these cases, the nondegeneracy assumption reduces to known conditions like strict complementarity, and discuss how to deal with the absence of this assumption algorithmically. This is joint work with Dr. Johannes Royset (royset@usc.edu).

Speakers
DK

Dohyun Kwon

Assistant Professor, University of Seoul / KIAS
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Yuchen Lou

Northwestern University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MS

Mattia Solla Saez

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8X: Recent Analysis in Proximal Algorithms and KL Exponents
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Session: Recent Analysis in Proximal Algorithms and KL Exponents
Chair: Xiaopeng Li
Cluster: nan

Talk 1: Proximal random reshuffling under local Lipschitz continuity
Speaker: Xiaopeng Li
Abstract: We study proximal random reshuffling for minimizing the sum of locally Lipschitz functions and a proper lower semicontinuous convex function without assuming coercivity or the existence of limit points. The algorithmic guarantees pertaining to near approximate stationarity rely on a new tracking lemma linking the iterates to trajectories of conservative fields. One of the novelties in the analysis consists in handling conservative fields with unbounded values.

Talk 2: Normal map-based stochastic algorithms
Speaker: Junwen Qiu
Abstract: The proximal stochastic gradient method (PSGD) is one of the state-of-the-art approaches for stochastic composite-type problems. In contrast to its deterministic counterpart, PSGD has been found to have difficulties with the correct identification of underlying substructures (such as supports, low rank patterns, or active constraints) and it does not possess a finite-time manifold identification property. Existing solutions rely on additional convexity assumptions or on the usage of variance reduction techniques. We address these limitations and present a simple variant of PSGD based on Robinson's normal map. The proposed normal map-based proximal stochastic gradient method (NSGD) is shown to converge globally. In addition, we establish complexity bounds for NSGD that match the known results for PSGD and we prove that NSGD can almost surely identify active manifolds in finite-time.

Talk 3: Kurdyka-Lojasiewicz exponent via square transformation
Speaker: Wenqing Ouyang
Abstract: Square transformation is a natural way to eliminate the nonnegative constraint to gain smoothness. Despite being smooth, it remains unknown about how the convergence rate would change via square transformation. We address this question by studying the connection between the Kurdyka-Lojasiewicz (KL) exponent of the transformed function and the one of the original function. We show that under strict complementarity condition, the KL exponent can be inherited perfectly. When strict complementarity condition fails, by additionally assuming convexity, we show that the KL exponent would depend on a constant that is related to an error bound condition of the solution set.

Speakers
XL

Xiaopeng Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JQ

Junwen Qiu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WO

Wenqing Ouyang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 8Y
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Wednesday July 23, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

2:30pm PDT

Coffee & Snack Break (Provided)
Wednesday July 23, 2025 2:30pm - 3:00pm PDT
Wednesday July 23, 2025 2:30pm - 3:00pm PDT
TBA

3:00pm PDT

Parallel Semi-Plenary Talk 3A
Wednesday July 23, 2025 3:00pm - 4:00pm PDT
Speakers
avatar for Ruth Misener

Ruth Misener

Professor of Computational Optimisation, Imperial College London
Ruth Misener is Professor of Computational Optimisation at Imperial College where she holds a BASF / Royal Academy of Engineering (RAEng) Research Chair in Data-Driven Optimisation. In 2017, Ruth received the MacFarlane Medal as overall winner of the RAEng Young Engineer Trust Engineer... Read More →
Wednesday July 23, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

3:00pm PDT

Parallel Semi-Plenary Talk 3B
Wednesday July 23, 2025 3:00pm - 4:00pm PDT
Speakers
AM

Anthony Man-Cho So

Anthony Man-Cho So is currently Dean of the Graduate School, Deputy Master of Morningside College, and Professor in the Department of Systems Engineering and Engineering Management of The Chinese University of Hong Kong (CUHK). His research focuses on the theory and applications of... Read More →
Wednesday July 23, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:00pm PDT

Break
Wednesday July 23, 2025 4:00pm - 4:15pm PDT
Wednesday July 23, 2025 4:00pm - 4:15pm PDT
TBA

4:15pm PDT

Parallel Sessions 9A: Dynamic flexibility, risk-awareness, and mechanisms for price-quantity alignment in electricity markets
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Dynamic flexibility, risk-awareness, and mechanisms for price-quantity alignment in electricity markets
Chair: Andy Sun
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Dynamic transmission capacity adjustment in large-scale electricity markets
Speaker: Baptiste Rabecq
Abstract: In this talk, we present a new optimization model for the real-time scheduling of a power grid that incorporates the so-called dynamic line ratings (DLR), where transmission lines' capacity can be adjusted in real-time according to the ambient weather conditions and the line flows through differential equations. We develop a new decomposition method that decouples AC optimal power flows from temperature dynamics with guaranteed global convergence. We will present a computational study to demonstrate the benefits of dynamic flexibility from DLR in a large-scale power grid. This is joint work with Thomas Lee and Andy Sun.

Talk 2: Handling real-time volatility in the SCUC computation
Speaker: Daniel Bienstock
Abstract: Under the typical real-time energy markets setup, any real-time load in excess of forecast must be paid for using the real-time LMPs. This introduces a form of risk faced by load-serving entities (and hence, by the public) that real-time volatility in generation, or in loads, will give rise to significant excess payments. Here, "real-time volatility" describes deviations from expected values which take place in short time frames -- a phenomenon that has been observed in power systems with high renewable penetration, or intelligent loads, especially under peak conditions. We describe an adjustment to the SCUC computation that is explicitly aware of potential volatility conditions during peak hours of the forthcoming day. The output of this updated SCUC has the same format as that for current SCUC; it is the nature of the computed commitments that will change. We will present computational experiments and describe the algorithm. Joint work with Y. Dvorkin, C. Guo, R. Mieth and J. Wang.

Talk 3: Mechanisms for Ensuring Price and Quantity Agreements in Electricity Day-Ahead Markets
Speaker: John Birge
Abstract: Many electricity markets include day-ahead energy markets as a mechanism to commit generators which require fixed costs for setup and have operating constraints. Based these markets on deterministic forecasts inherently cannot match both prices and quantities with the expected real-time values. This talk will discuss mechanisms that build stochastic representations of the market and how these can be solved to obtain efficient generator commitments that match day-ahead prices and quantities with their real-time expectations.

Speakers
AS

Andy Sun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BR

Baptiste Rabecq

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DB

Daniel Bienstock

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JB

John Birge

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9B: Mixed-Integer Nonlinear Programming
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Mixed-Integer Nonlinear Programming
Chair: Jan Kronqvist
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: A graphical framework for global optimization of mixed-integer nonlinear programs
Speaker: Danial Davarnia
Abstract: Despite advances in mixed-integer linear and convex programming solvers, general mixed-integer nonlinear programs (MINLPs) remain difficult to solve due to the complex algebraic structures that modern solvers struggle to handle. This work presents a novel graphical framework for globally solving MINLPs using decision diagrams (DDs), which model complex structures beyond the reach of conventional techniques. Key components include graphical reformulation of constraints, convexification methods, efficient cutting planes, and a spatial branch-and-bound method with convergence guarantees. This work fills a gap in DD literature by offering a general-purpose solution method for MINLPs and demonstrates its efficacy on challenging test cases from the MINLP Library that cannot be solved by current global solvers

Talk 2: New perspectives on invexity and its algorithmic applications
Speaker: Ksenia Bestuzheva
Abstract: One of the key properties of convex problems is that every stationary point is a global optimum, and nonlinear programming algorithms that converge to local optima are thus guaranteed to find the global optimum. However, some nonconvex problems possess the same property. This observation has motivated research into generalizations of convexity. This talk proposes a new generalization which we refer to as optima-invexity: the property that only one connected set of optimal solutions exists. We state conditions for optima-invexity of unconstrained problems and discuss structures that are promising for practical use, and outline algorithmic applications of these structures.

Talk 3: To be updated: Mixed-integer SDP
Speaker: Jan Kronqvist
Abstract: To be updated: We consider different methods for generating cuts and solving mixed-integer semidefinite programming (MISDP) instances within an outer approximation framework. In fact, the main components of the classical outer approximation algorithm for convex mixed-integer nonlinear programming can easily be tailored towards MISDP such that similar convergence properties are obtained. We propose some new methods for generating cuts that have desirable theoretical and computational properties, and we present a numerical comparison.

Speakers
DD

Danial Davarnia

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KB

Ksenia Bestuzheva

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JK

Jan Kronqvist

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9C: Fixed Point and Min-Max Theory in Data Science
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Fixed Point and Min-Max Theory in Data Science
Chair: Ahmet Alacaoglu
Cluster: Fixed Points and Variational Inequalities

Talk 1: Fixed-point error bounds for mean-payoff Markov decision processes
Speaker: Roberto Cominetti
Abstract: We discuss the use of optimal transport techniques to derive finite-time error bounds for reinforcement learning in mean-payoff Markov decision processes. The results are obtained as a special case of stochastic Krasnoselski—Mann fixed point iterations for nonexpansive maps. We present sufficient conditions on the stochastic noise and stepsizes that guarantee almost sure convergence of the iterates towards a fixed point, as well as non-asymptotic error bounds and convergence rates. Our main results concern the case of a martingale difference noise with variances that can possibly grow unbounded. We also analyze the case of uniformly bounded variances, and how they apply for Stochastic Gradient Descent in convex optimization.

Talk 2: Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithms
Speaker: Haipeng Luo
Abstract: Self-play via online learning is one of the premier ways to solve large-scale two-player zero-sum games, both in theory and practice. Particularly popular algorithms include optimistic multiplicative weights update (OMWU) and optimistic gradient-descent-ascent (OGDA). While both algorithms enjoy O(1/T) ergodic convergence to Nash equilibrium in two-player zero-sum games, OMWU offers several advantages including logarithmic dependence on the size of the payoff matrix and fast convergence to coarse correlated equilibria even in general-sum games. However, in terms of last-iterate convergence in two-player zero-sum games, an increasingly popular topic in this area, OGDA guarantees that the duality gap shrinks at a rate of (1/sqrt{T}), while the best existing last-iterate convergence for OMWU depends on some game-dependent constant that could be arbitrarily large. This begs the question: is this potentially slow last-iterate convergence an inherent disadvantage of OMWU, or is the current analysis too loose? Somewhat surprisingly, we show that the former is true. More generally, we prove that a broad class of algorithms that do not forget the past quickly all suffer the same issue: for any arbitrarily small delta > 0, there exists a 2 by 2 matrix game such that the algorithm admits a constant duality gap even after 1/delta rounds. This class of algorithms includes OMWU and other standard optimistic follow-the-regularized-leader algorithms.

Talk 3: A first-order algorithm for decentralised min-max problems
Speaker: Matthew Tam
Abstract: In this work, we consider a connected network of finitely many agents working cooperatively to solve a min-max problem with convex-concave structure. We propose a decentralised first-order algorithm which can be viewed as combining features of two algorithms: PG-EXTRA for decentralised minimisation problems and the forward reflected backward method for (non-distributed) min-max problems. In each iteration of our algorithm, each agent computes the gradient of the smooth component of its local objective function as well as the proximal operator of its nonsmooth component, following by a round of communication with its neighbours.

Speakers
AA

Ahmet Alacaoglu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RC

Roberto Cominetti

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HL

Haipeng Luo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MT

Matthew Tam

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9D: Manifolds, samples, and learning II
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Manifolds, samples, and learning II
Chair: Ralf Zimmermann
Cluster: Optimization on Manifolds

Talk 1: A Riemannian Douglas-Rachford Algorithm for Low-Rank and Row-Sparse Matrix Recovery
Speaker: Lukas Klingbiel
Abstract: In this work, we consider a matrix recovery problem under low-rank and row-sparse constraints using a Riemannian Douglas-Rachford algorithm. We introduce a retraction-based Riemannian Douglas-Rachford based on the Riemannian Douglas-Rachford on symmetric Hadamard manifolds. We show local convergence of this method for nonexpansive reflections and manifolds with locally bounded sectional curvature. We give an explicit form of the algorithm on the fixed-rank manifold to solve the matrix recovery problem. In particular, numerical experiments suggest that we require the minimal number of measurements for the recovery of a low-rank and row-sparse matrix.

Talk 2: Approximating maps into manifolds
Speaker: Simon Jacobsson
Abstract: Many interesting functions arising in applications map into Riemannian manifolds. The distance between two points on a manifold depends on the intrinsic geometry of that manifold. When approximating functions that map into manifolds, it is natural to measure the error on the manifold, rather than in some ambient space where the manifold might be embedded. Especially, when the dimension of the ambient space is much larger than the dimension of the manifold, such as for low rank tensors, it becomes unfeasible to work in the ambient space. In this presentation, we present a scheme to approximate maps into manifolds by first pulling back the problem to the tangent space and then applying a scheme for approximating maps into vector spaces. Our main result is a theorem that bounds the approximation error on the manifold in terms of an error bound on the tangent space and a lower bound for the manifold's sectional curvature. Example applications to Krylov subspaces and low-rank approximation are discussed as well. This is joint work with Raf Vandebril (KU Leuven), Joeri Van der Veken (KU Leuven), and Nick Vannieuwenhoven (KU Leuven).

Talk 3: Second-order differential operators, stochastic differential equations and Brownian motions on embedded manifolds
Speaker: Du Nguyen
Abstract: We provide a framework to simulate Riemannian Brownian and Riemannian Langevin equations on embedded manifolds in global coordinates. We specify the conditions when a manifold M embedded in an inner product space E is an invariant manifold of a stochastic differential equation (SDE) on E, linking it with the notion of second-order differential operators on M. When M is given a Riemannian metric, we derive a simple formula for the Laplace-Beltrami operator in terms of the gradient and Hessian on E and construct the Riemannian Brownian motions on M as solutions of conservative Stratonovich and Ito SDEs on E. We derive explicitly the SDE for Brownian motions on several important manifolds in applications, including left-invariant matrix Lie groups using embedded coordinates, Stiefel, Grassmann and symmetric positive definite (SPD) manifolds. Numerically, we propose three simulation schemes to solve SDEs on manifolds. In addition to the stochastic projection method, to simulate Riemannian Brownian motions, we construct a second-order tangent retraction of the Levi-Civita connection using a given E-tubular retraction. We also propose the retractive Euler-Maruyama method to solve a SDE, taking into account the second-order term of a tangent retraction. We verify numerically that on several Riemannian manifolds, our approach can sample in global coordinates a given distribution as a long-term limit of Riemannian Brownian or Riemannian Langevin equations. This is joint work with Stefan Sommer (Technical University of Denmark)

Speakers
RZ

Ralf Zimmermann

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LK

Lukas Klingbiel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SJ

Simon Jacobsson

Simon Jacobsson received his Master's degree in physics at Chalmers University of Technology. Currently, he is a doctoral student at KU Leuven. His research is a joint project between the numerical analysis group at the KU Leuven computer science department and the geometry group... Read More →
DN

Du Nguyen

Researcher, Independent
Name:Du NguyenTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing in algorithms that eventuallyconverge—if... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9E: Recent Progress on Bilevel Optimization for Machine Learning
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Recent Progress on Bilevel Optimization for Machine Learning
Chair: Tong Zhang
Cluster: Nonlinear Optimization

Talk 1: Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis
Speaker: Kaiyi Ji
Abstract: Bilevel optimization has recently attracted considerable attention due to its abundant applications in machine learning problems. However, existing methods rely on prior knowledge of problem parameters to determine stepsizes, resulting in significant effort in tuning stepsizes when these parameters are unknown. In this paper, we propose two novel tuning-free algorithms, D-TFBO and S-TFBO. D-TFBO employs a double-loop structure with stepsizes adaptively adjusted by the "inverse of cumulative gradient norms" strategy. S-TFBO features a simpler fully single-loop structure that updates three variables simultaneously with a theory-motivated joint design of adaptive stepsizes for all variables. We provide a comprehensive convergence analysis for both algorithms and show that D-TFBO and S-TFBO respectively require $\mathcal{O}(\frac{1}{\epsilon})$ and $\mathcal{O}(\frac{1}{\epsilon}\log^4(\frac{1}{\epsilon}))$ iterations to find an $\epsilon$-accurate stationary point, (nearly) matching their well-tuned counterparts using the information of problem parameters. Experiments on various problems show that our methods achieve performance comparable to existing well-tuned approaches, while being more robust to the selection of initial stepsizes. To the best of our knowledge, our methods are the first to completely eliminate the need for stepsize tuning, while achieving theoretical guarantees.

Talk 2: Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions
Speaker: Xuxing Chen
Abstract: Stochastic Bilevel optimization usually involves minimizing an upper-level (UL) function that is dependent on the arg-min of a strongly-convex lower-level (LL) function. Several algorithms utilize Neumann series to approximate certain matrix inverses involved in estimating the implicit gradient of the UL function (hypergradient). The state-of-the-art StOchastic Bilevel Algorithm (SOBA) instead uses stochastic gradient descent steps to solve the linear system associated with the explicit matrix inversion. This modification enables SOBA to match the lower bound of sample complexity for the single-level counterpart in non-convex settings. Unfortunately, the current analysis of SOBA relies on the assumption of higher-order smoothness for the UL and LL functions to achieve optimality. In this talk, I will introduce a novel fully single-loop and Hessian-inversion-free algorithmic framework for stochastic bilevel optimization and present a tighter analysis under standard smoothness assumptions (first-order Lipschitzness of the UL function and second-order Lipschitzness of the LL function). Furthermore, I will show that by a slight modification of our approach, our algorithm can handle a more general multi-objective robust bilevel optimization problem. For this case, we obtain the state-of-the-art oracle complexity results demonstrating the generality of both the proposed algorithmic and analytic frameworks. Numerical experiments demonstrate the performance gain of the proposed algorithms over existing ones.

Talk 3: Two-Timescale Linear Stochastic Approximation: Constant Stepsizes Go a Long Way
Speaker: Jeongyeol Kwon
Abstract: Previous studies on two-timescale stochastic approximation (SA) mainly focused on bounding mean-squared errors under diminishing stepsize schemes. In this work, we investigate {\it constant} stpesize schemes through the lens of Markov processes, proving that the iterates of both timescales converge to a unique joint stationary distribution in the Wasserstein metric. We derive explicit geometric and non-asymptotic convergence rates, as well as the variance and bias introduced by constant stepsizes in the presence of Markovian noise. Specifically, with two constant stepsizes $\alpha < \beta$, we show that the biases scale linearly with both stepsizes as $\Theta(\alpha)+\Theta(\beta)$ up to higher-order terms, while the variance of the slower iterate (resp., faster iterate) scales only with its own stepsize as $O(\alpha)$ (resp., $O(\beta)$). Unlike previous work, our results require no additional assumptions such as $\beta^2 \ll \alpha$ nor extra dependence on dimensions. These fine-grained characterizations allow tail-averaging and extrapolation techniques to reduce variance and bias, improving mean-squared error bound to $O(\beta^4 + \frac{1}{t})$ for both iterates.

Speakers
KJ

Kaiyi Ji

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XC

Xuxing Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JK

Jeongyeol Kwon

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9F: Certified Data-Driven Optimization via the Scenario Approach
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Certified Data-Driven Optimization via the Scenario Approach
Chair: Simone Garatti
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Scenario Optimization: Data-Driven Goal-Oriented Designs with Certified Reliability
Speaker: Marco C. Campi
Abstract: The utilization of data is becoming paramount in addressing modern optimization problems characterized by increasing complexity. However, for a true science of data-driven optimization to emerge, fundamental questions need to be addressed: (i) should data be used to reconstruct the unknown distribution of uncertainty, or should we rather directly optimize to achieve the final design? (ii) is it possible to endow data-driven, optimization-based designs with certifications of quality that hold beyond any assumed characteristics of the underlying distribution? In this presentation, we introduce "Scenario Optimization", a collection of data-driven optimization techniques that provide algorithmic and theoretical responses to the questions posed above. After introducing robust data-driven techniques, we move toward relaxed schemes, showing how each plays a prominent role in specific application domains. The algorithmic methods will be accompanied by results certifying their reliability, providing users with a comprehensive suite of data-driven optimization techniques.

Talk 2: Expanding the Scope of the Scenario Approach with the Pick-to-Learn (P2L) Algorithm
Speaker: Simone Garatti
Abstract: The scenario approach is a powerful and widely-used framework in data-driven optimization and decision-making, also offering a rigorous methodology to evaluate the risk of solutions derived from data. This makes it a pivotal tool for decision-making under uncertainty. On the other hand, the scenario approach relies on structural assumptions that are not always met in applications. A notable example is the minimization of mean error cost functions, which is frequently used in machine learning, particularly in the training of neural networks. In this talk, we delve into the deeper theoretical foundations behind the scenario approach, and present the powerful framework of preferent sample compression. By leveraging this deeper understanding of the method, a meta-algorithm, the Pick-to-Learn (P2L) algorithm, is introduced to broaden the applicability of the scenario approach. P2L builds upon any learning-based decision scheme as a black-box to create a new scheme that conform to the theory of preferent sample compression, while maintaining the design goals of the original scheme. This makes the reach of the theory of preferent compression virtually infinite, and licenses the utilization of the powerful risk evaluation results of the scenario approach to a wide range of problems of great interest that would otherwise fall outside its traditional scope.

Talk 3: Randomized Algorithms and PAC Bounds for Inverse Reinforcement Learning in Continuous Spaces
Speaker: Tobias Sutter
Abstract: This work studies discrete-time discounted Markov decision processes with continuous state and action spaces and addresses the inverse problem of inferring a cost function from observed optimal behavior. We first consider the case in which we have access to the entire expert policy and characterize the set of solutions to the inverse problem by using occupation measures, linear duality, and complementary slackness conditions. To avoid trivial solutions and ill-posedness, we introduce a natural linear normalization constraint. This results in an infinite-dimensional linear feasibility problem, prompting a thorough analysis of its properties. Next, we use linear function approximators and adopt a randomized approach, namely the scenario approach and related probabilistic feasibility guarantees, to derive epsilon-optimal solutions for the inverse problem. We further discuss the sample complexity for a desired approximation accuracy. Finally, we deal with the more realistic case where we only have access to a finite set of expert demonstrations and a generative model and provide bounds on the error made when working with samples.

Speakers
MC

Marco C. Campi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SG

Simone Garatti

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TS

Tobias Sutter

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9G: Aspects of NLP solver implementations
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Aspects of NLP solver implementations
Chair: Andreas Waechter
Cluster: Nonlinear Optimization

Talk 1: The JuliaSmoothOptimizers Ecosystem for Numerical Linear Algebra and Optimization in Julia
Speaker: Tangi Migot
Abstract: JuliaSmoothOptimizers (JSO) is an organization that provides infrastructure and solvers for smooth and nonsmooth optimization, and numerical linear algebra in the Julia programming language. Those packages are intended to work consistently together and exploit the structure present in problems. They offer modeling facilities, widely useful known solvers, either in the form of interfaces or pure Julia implementations, but also unique methods that are the product of active research. JSO provides building blocks to quickly prototype solvers in a high-level language and implement efficient large-scale solvers. We present an overview of the organization and show how its facilities address the needs of students, instructors, modelers, users of optimization and researchers.

Talk 2: The Journey of a nonlinear expression through the Gurobi Optimizer
Speaker: Robert Luce
Abstract: The Gurobi optimizer can solve nonlinear optimization problems to local or global optimality. Gurobi expects these problems being modeled equation-based, that is, as explicit expressions for all nonlinear constraints. In this talk we trace the journey of a nonlinear expression through our solution framework. It starts on our Python based modeling API gurobipy, where such expressions are naturally integrated with the greater modeling functionality. Once transferred to the Optimizer, these expressions undergo our presolving algorithms, which may simplify and homogenize the expressions. In this form the expression becomes part of the solution process: Our nonlinear interior point algorithm will evaluate expressions to obtain residuals as given points, as well as evaluate derivatives in order to obtain data on first order optimality.

Talk 3: Implemention of the Gurobi NLP solver
Speaker: Andreas Waechter
Abstract: We present details on the implementation of a local nonlinear optimization solver in Gurobi. The method is based on a primal-dual interior-point method with line search. Numerical experiments for a large set of test problems is presented.

Speakers
TM

Tangi Migot

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RL

Robert Luce

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AW

Andreas Waechter

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9H: On Hierarchical Optimization, Games, and Federated Learning
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: On Hierarchical Optimization, Games, and Federated Learning
Chair: Farzad Yousefian
Cluster: Multi-agent Optimization and Games

Talk 1: Modelling the non-stationarity of capacity in continual learning
Speaker: Krishnan Raghavan
Abstract: Continual learning is the problem of learning on a sequence of tasks and the core issue in this domain is that of balancing catastrophic forgetting of prior knowledge with generalization over new tasks, known as the stability-plasticity dilemma. This work introduces CL's effective model capacity~(CLEMC) to theoretically formalize how this balance depends on the neural network (NN), the tasks, and the optimization procedure. In this talk, we demonstrate that CLEMC, and thus the balance point, is non-stationary and the interplay between tasks, neural network and the optimization procedure is an evolving dynamical game. We discuss the use of optimal control techniques to model this dynamical game and study the evolution of CLEMC. We hypothesize that regardless of the NN architecture and optimization method, the network's ability to represent new tasks diminishes if the new tasks' data distributions differ significantly from previous ones, i.e. the Nash equilibrium becomes more and more difficult to find. We cement this hypothesis theoretically and then using various NNs, from small feed-forward and convolutional networks to transformer-based language models with millions of parameters (8M and 134M).

Talk 2: Federated Simple Bilevel Optimization: A Universal Regularized Scheme with Guarantees
Speaker: Yuyang Qiu
Abstract: We study a class of bilevel federated learning (FL) problem, where clients cooperatively seek to find among multiple optimal solutions of a primary distributed learning problem, a solution that minimizes a secondary distributed global loss function. This problem has attracted increasing attention in machine learning, in particular, in over-parameterized learning and hyperparameter optimization. Despite some recent progress, communication-efficient FL methods equipped with complexity guarantees for resolving this problem are primarily absent. Motivated by this lacuna, we propose a universal regularized scheme and derive promising error bounds in terms of both the lower-level and upper-level loss functions. Leveraging this unifying theory, we then enable existing FL methods, including FedAvg and SCAFFOLD, to solve the corresponding bilevel FL problem, and derive novel communication complexity guarantees for each method. Intriguingly, the universal scheme can be employed to provably enable many other state-of-the-art optimization methods to address the bilevel problem. We validate the theoretical findings on EMNIST and CIFAR-10 datasets.

Talk 3: Improved guarantees for optimal Nash equilibrium seeking and bilevel variational inequalities
Speaker: Sepideh Samadi
Abstract: We consider a class of hierarchical variational inequality (VI) problems that subsumes VI-constrained optimization and several other important problem classes including the optimal solution selection problem and the optimal Nash equilibrium (NE) seeking problem. Our main contributions are threefold. (i) We consider bilevel VIs with monotone and Lipschitz continuous mappings and devise a single-timescale iteratively regularized extragradient method, named IR-EG(m,m). We improve the existing iteration complexity results for addressing both bilevel VI and VI-constrained convex optimization problems. (ii) Under the strong monotonicity of the outer level mapping, we develop a method named IR-EG(s,m) and derive faster guarantees than those in (i). We also study the iteration complexity of this method under a constant regularization parameter. These results appear to be new for both bilevel VIs and VI-constrained optimization. (iii) To our knowledge, complexity guarantees for computing the optimal NE in nonconvex settings do not exist. Motivated by this lacuna, we consider VI-constrained nonconvex optimization problems and devise an inexactly-projected gradient method, named IPR-EG, where the projection onto the unknown set of equilibria is performed using IR-EG(s,m) with a prescribed termination criterion and an adaptive regularization parameter. We obtain new complexity guarantees in terms of a residual map and an infeasibility metric for computing a stationary point. We validate the theoretical findings using preliminary numerical experiments for computing the best and the worst Nash equilibria.

Speakers
FY

Farzad Yousefian

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KR

Krishnan Raghavan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Yuyang Qiu

Yuyang Qiu

PhD from Rutgers U. Incoming postdoc at UCSB.
Hi, I'm Yuyang, currently a 5th-year Ph.D. candidate in the ISE department at Rutgers University. My advisor is Prof. Farzad Yousefian. My research spans federated learning, hierarchical optimization, as well as distributed optimization over networks. I am currently interested on... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9I: Convexity and First-Order Riemannian methods
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Convexity and First-Order Riemannian methods
Chair: David Martínez-Rubio
Cluster: Optimization on Manifolds

Talk 1: Online Implicit Riemannian Optimization
Speaker: Christophe Roux
Abstract: Many optimization problems such as eigenvalue problems, principal component analysis and low-rank matrix completion can be interpreted as optimization problems over Riemannian manifolds, which allows for exploiting the geometric structure of the problems. While Riemannian optimization has been studied extensively in the offline setting, the /online/ setting is not well understood. A major challenge in prior works was handling in-manifold constraints that arise in the online setting. We leverage /implicit/ methods to address this problem and improve over existing regret bounds. Furthermore, using our new implicit online algorithms, we achieve accelerated rates for /constrained/ Riemannian geodesically convex-concave min-max problems, which was previously only possible under additional assumptions.

Talk 2: What do global Polyak-Łojasiewicz functions look like?
Speaker: Christopher Criscitiello
Abstract: In optimization and machine learning literature, it is common to assume a smooth function $f \colon \mathbb{R}^n \to \mathbb{R}$ globally satisfies the Polyak-Łojasiewicz (PŁ) condition, as it guarantees global exponential convergence of gradient descent. We ask: what do such functions look like? Specifically, is the set of minimizers of $f$ necessarily a smooth manifold diffeomorphic to a Euclidean space? And if so, is there a diffeomorphism of $\mathbb{R}^n$ taking $f$ to a quadratic function? We completely answer both questions. The set of minimizers of $f$ forms a smooth manifold of dimension $k \geq 0$. We show that if $k = 0,1,2$, then the answer to both questions is yes: the set of minimizers of $f$ is necessarily diffeomorphic to a point, line or plane, and there exists a diffeomorphism taking $f$ to a quadratic. If $k \geq 3$, surprisingly the answer is no: there exists a global PŁ function whose set of minimizers is not diffeomorphic to a Euclidean space. We then give a necessary and sufficient condition under which the answer to both questions is yes for $k \geq 5$. An immediate consequence of our results is that every smooth global Polyak-Łojasiewicz function is geodesically convex in some metric. Moreover, if the function has a compact set of minimizers, then it is geodesically convex in a *flat* metric. Our proofs use tools from differential topology, like local stable manifold theorems and the Morse Lemma, and our counterexamples build on the famous Whitehead manifold. This is joint work with Quentin Rebjock and Nicolas Boumal.

Talk 3: Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point
Speaker: David Martínez-Rubio
Abstract: We revisit the analyses of the two most fundamental algorithms in geodesically-convex optimization: Riemannian gradient descent and Riemannian proximal point. Previous analyses did not fully quantify rates of convergence under standard assumptions or were of limited generality. We show that for different step sizes the iterates naturally stay in balls of different radii around an optimizer, depending on the initial distance and the curvature. This along with different subproblems one may solve, allows to design different variants that trade-off curvature dependence for a different dependence of rates on condition numbers or subproblem types to solve. The question of whether we can get the best convergence of all of our algorithms with a single efficient algorithm remains open. We also provide new properties of proximal methods on Riemannian manifolds and an implementable inexact proximal point algorithm yielding new results on minmax geodesically convex-concave problems.

Speakers
CR

Christophe Roux

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CC

Christopher Criscitiello

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DM

David Martínez-Rubio

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9J: Automatic Differentiation as a Tool for Computational Science
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Automatic Differentiation as a Tool for Computational Science
Chair: Sri Hari Krishna Narayanan
Cluster: Computational Software

Talk 1: EnzymeMLIR: High-Performance Automatic Differentiation of Tensor Code
Speaker: William Moses
Abstract: Automatic differentiation (AD) is key to training neural networks, Bayesian inference, and scientific computing. Applying these techniques requires rewriting code in a specific machine learning framework or manually providing derivatives. This talk presents Enzyme, a high-performance automatic differentiation compiler plugin for the LLVM and MLIR compiler frameworks. Enzyme differentiates programs in any language whose compiler targets LLVM/MLIR, including C/C++, Fortran, Julia, Rust, Swift, JaX, etc., thereby providing native AD capabilities in these languages with state-of-the-art performance. Unlike traditional tools, Enzyme performs AD on optimized IR. On a combined machine-learning and scientific computing benchmark suite, AD on optimized IR achieves a geometric mean speedup of 4.2x over AD on IR before optimization. This talk will also include work that makes Enzyme the first fully automatic reverse-mode AD tool to generate gradients of existing GPU kernels as well as the benefits of operating within high-level structured representations, like MLIR.

Talk 2: Challenges with Implementing Differentiable Quantum Dynamics
Speaker: Sri Hari Krishna Narayanan
Abstract: Differentiable quantum dynamics require automatic differentiation of a complex-valued initial value problem, which numerically integrates a system of ordinary differential equations from a specified initial condition, as well as the eigendecomposition of a matrix. This work is a survey of existing differentiable programming frameworks for these tasks, finding that no framework natively supports our application requirements fully. We therefore demonstrate a need for broader support of complex-valued, differentiable numerical integration in scientific computing libraries. We further demonstrate that the derivatives of our quantum dynamics application can be computed through a combination of differentiable programming frameworks and handcoding.

Talk 3: Leveraging Automatic Differentiation to Improve Ice-Sheet and Ocean Modeling
Speaker: Shreyas Gaikwad
Abstract: Mathematical modeling of geophysical fluids is a complex undertaking that necessarily involves several approximations (constitutive models, spatial discretization, and subgrid-scale parameterizations) to close a system of high-fidelity equations such as conservation of mass, momentum, energy, and tracers. Examples of such parameters include those used to represent aerosol and cloud microphysics in the atmosphere, the coefficients of the mixing parameterizations in the ocean, and the basal sliding coefficients below ice sheets. Model boundary and initial conditions are also required and often poorly constrained. Meaningful interpretation of model output therefore demands investigation of the impact of these uncertain parameters, initial conditions, and boundary conditions on the simulated state, and an effort to identify their "best" values for some specific metric. In the context of ice sheet and ocean modeling, gradients of model-data misfits or other QoI with respect to the various uncertain parameters, boundary conditions, or initial conditions are a key ingredient for performing sensitivity analysis, model calibration, state estimation, or uncertainty quantification (UQ), which guide the improvement of model simulations through PDE-constrained gradient-based optimization. We present new frameworks for generating derivative code, i.e., tangent linear and adjoint models, of an ice sheet model, SICOPOLIS, and an ocean model, MITgcm. These derivative operators are powerful computational engines to efficiently compute comprehensive gradients or sensitivities of scalar-valued model output, including least-squares model-data misfits or important quantities of interest, to high-dimensional model inputs (such as model initial conditions, parameter fields, or boundary conditions). Both frameworks leverage Tapenade, an open-source Automatic Differentiation tool, to generate and maintain up-to-date derivative codes. Both frameworks are open-source and freely available.

Speakers
WM

William Moses

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SH

Sri Hari Krishna Narayanan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SG

Shreyas Gaikwad

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9K: Modern Polynomial Optimization in Data Science II
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Modern Polynomial Optimization in Data Science II
Chair: Xindong Tang
Cluster: Conic and Semidefinite Optimization

Talk 1: Loss surface of deep neural networks with polynomial activation functions
Speaker: Tingting Tang
Abstract: TBD

Talk 2: Low-precision tensor decomposition and its applications in data science
Speaker: Zi Yang
Abstract: Tensors are high-order generalizations of matrices and are widely used to represent multi-dimensional data arrays in data science. Dealing with large-scale tensors is memory and computation intensive, prohibiting their applications in many resource-limited scenarios. Low-precision computation is to save and compute using lower bits, reducing memory, and accelerating computation. In this talk, we will explore the application of low-precision computation to large-scale tensor problems. Specifically, we present a mixed-precision block stochastic gradient descent method for CP tensor decomposition. Our approach uses lower-bit fixed-point representations, such as INT8, INT4, and INT2, to compute gradients more efficiently. Numerical experiments on both synthetic and real-world tensor datasets demonstrate the superior efficiency of our mixed-precision algorithm compared to full-precision CP decomposition. This work significantly reduces memory, computing, and energy costs, making it particularly useful for resource-constrained edge computing devices. We will also discuss how low-precision tensor computation can compress large AI models and accelerate both their training and inference.

Talk 3: Global Convergence of High-Order Regularization Methods with Sums-of-Squares Taylor model
Speaker: Wenqi Zhu
Abstract: High-order tensor methods that employ Taylor-based local models (of degree $p\ge 3$) within adaptive regularization frameworks have been recently proposed for both convex and nonconvex optimization problems. They have been shown to have superior, and even optimal, worst-case global convergence rates and local rates compared to Newton's method. Finding rigorous and efficient techniques for minimizing the Taylor polynomial sub-problems remains a challenging aspect for these algorithms. Ahmadi et al \cite{ahmadi2023higher} recently introduced a tensor method based on sum-of-squares (SoS) reformulations, so that each Taylor polynomial sub-problem in their approach can be tractably minimized using semidefinite programming (SDP) \cite{ahmadi2023higher}; however, the global convergence and complexity of their method have not been addressed for general nonconvex problems. In this talk, we introduce an algorithmic framework that combines the Sum of Squares (SoS) Taylor model with adaptive regularization techniques for nonconvex smooth optimization problems. Each iteration minimizes an SoS-convex Taylor model, offering a polynomial cost per iteration. For general nonconvex functions, the worst-case evaluation complexity bound is $\mathcal{O}(\epsilon^{-2})$, while for strongly convex functions, an improved evaluation complexity bound of $\mathcal{O}(\epsilon^{-\frac{1}{p}})$ is established. To the best of our knowledge, this is the first global rate analysis for an adaptive regularization algorithm with a tractable high-order sub-problem in nonconvex smooth optimization, opening the way for further improvements.

Speakers
XT

Xindong Tang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TT

Tingting Tang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZY

Zi Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WZ

Wenqi Zhu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9L: Novel First-Order Methods via Performance Estimation I
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Novel First-Order Methods via Performance Estimation I
Chair: Alex Wang
Cluster: Conic and Semidefinite Optimization

Talk 1: Nonlinear Conjugate Gradient Methods: Worst-case Convergence Rates via Computer-assisted Analyses
Speaker: Shuvomoy Das Gupta
Abstract: We propose a computer-assisted approach to the analysis of the worst-case convergence of nonlinear conjugate gradient methods (NCGMs). Those methods are known for their generally good empirical performances for large-scale optimization, while having relatively incomplete analyses. Using our computer-assisted approach, we establish novel complexity bounds for the Polak-Ribière-Polyak (PRP) and the Fletcher- Reeves (FR) NCGMs for smooth strongly convex minimization. In particular, we construct mathematical proofs that establish the first non- asymptotic convergence bound for FR (which is historically the first developed NCGM), and a much improved non-asymptotic convergence bound for PRP. Additionally, we provide simple adversarial examples on which these methods do not perform better than gradient descent with exact line search, leaving very little room for improvements on the same class of problems.

Talk 2: Convergence Rate of Boosted Difference of Convex Algorithm (BDCA)
Speaker: Hadi Abbaszadehpeivasti
Abstract: In recent years, the Difference of Convex Algorithm (DCA) has received significant attention for its ability in solving a wide range of non-convex optimization problems. Its efficiency and flexibility are enhanced through appropriate decompositions, which show that other methods, such as the fixed-step gradient method and the proximal gradient method, can be viewed as special cases of DCA. A variant of DCA, called boosted DCA, has been developed in recent years which outperforms the standard DCA in practice. In this talk, I will discuss the worst-case convergence rate of the boosted DCA, a topic that has limited study on its worst-case convergence rate.

Talk 3: Composing Optimized Stepsize Schedules for Gradient Descent
Speaker: Alex Wang
Abstract: Recent works by Altschuler and Parrilo and Grimmer, Shu, and Wang have shown that it is possible to accelerate the convergence of gradient descent on smooth convex functions, even without momentum, just by picking special stepsizes. In this paper, we provide a general theory for composing stepsize schedules capturing all recent advances in this area and more. We propose three notions of ``composable'' stepsize schedules with elementary associated composition operations for combining them. From these operations, in addition to recovering recent works, we construct three highly optimized sequences of stepsize schedules. We first construct optimized stepsize schedules of every length generalizing the exponentially spaced silver stepsizes of Altschuler and Parrilo. We then construct highly optimized stepsizes schedules for minimizing final objective gap or gradient norm, improving on prior rates by constants and, more importantly, matching or beating the numerically computed minimax optimal schedules of Das Gupta et al.. We conjecture these schedules are in fact minimax (information theoretic) optimal.

Speakers
SD

Shuvomoy Das Gupta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HA

Hadi Abbaszadehpeivasti

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AW

Alex Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9M: Randomized algorithms with applications to machine learning
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Randomized algorithms with applications to machine learning
Chair: Laurent Condat
Cluster: Optimization For Data Science

Talk 1: Subspace Optimization for Large Language Models with Convergence Guarantees
Speaker: Kun Yuan
Abstract: Subspace optimization algorithms, with GaLore (Zhao et al., 2024) as a representative method, have gained popularity for pre-training or fine-tuning large language models (LLMs) due to their memory efficiency. However, their convergence guarantees remain unclear, particularly in stochastic settings. In this paper, we unexpectedly discover that GaLore does not always converge to the optimal solution and substantiate this finding with an explicit counterexample. We then investigate the conditions under which GaLore can achieve convergence, demonstrating that it does so either in deterministic scenarios or when using a sufficiently large mini-batch size. More significantly, we introduce GoLore (Gradient random Low-rank projection), a novel variant of GaLore that provably converges in stochastic settings, even with standard batch sizes. Our convergence analysis can be readily extended to other sparse subspace optimization algorithms. Finally, we conduct numerical experiments to validate our theoretical results and empirically explore the proposed mechanisms.

Talk 2: Convergence of Gradient Descent with Linearly Correlated Noise and Applications to Differentially Private Learning
Speaker: Anastasia Koloskova
Abstract: We study gradient descent under linearly correlated noise. Our work is motivated by recent practical methods for optimization with differential privacy (DP), such as DP-FTRL, which achieve strong performance in settings where privacy amplification techniques are infeasible (such as in federated learning). These methods inject privacy noise through a matrix factorization mechanism, making the noise linearly correlated over iterations. We propose a simplified setting that distills key facets of these methods and isolates the impact of linearly correlated noise. We analyze the behavior of gradient descent in this setting, for both convex and non-convex functions. Our analysis is demonstrably tighter than prior work and recovers multiple important special cases exactly (including anticorrelated perturbed gradient descent). We use our results to develop new, effective matrix factorizations for differentially private optimization, and highlight the benefits of these factorizations theoretically and empirically.

Talk 3: Optimal Stochastic Algorithms for Distributionally Robust Learning
Speaker: Zaid Harchaoui
Abstract: We consider the penalized distributionally robust optimization (DRO) problem with a closed, convex uncertainty set, a setting that encompasses the f-DRO, Wasserstein-DRO, and spectral/L-risk formulations used in practice. We present Drago, a stochastic primal-dual algorithm that achieves a state-of-the-art linear convergence rate on strongly convex-strongly concave DRO problems. The method combines both randomized and cyclic components with mini-batching, which effectively handles the unique asymmetric nature of the primal and dual problems in DRO. We support our theoretical results with numerical benchmarks in classification and regression.

Speakers
avatar for Laurent Condat

Laurent Condat

Senior Research Scientist, King Abdullah University of Science and Technology (KAUST)
Laurent Condat received a PhD in applied mathematics in 2006 from Grenoble Institute of Technology, Grenoble, France. After a postdoc in the Helmholtz Zentrum Muenchen, Munich, Germany, he was hired in 2008 as a permanent researcher by the French National Center for Scientific Research... Read More →
KY

Kun Yuan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AK

Anastasia Koloskova

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZH

Zaid Harchaoui

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9N: Control and Optimization of AVs for Transportation Solutions
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Control and Optimization of AVs for Transportation Solutions
Chair: Jeff Ban & Ruolin Li
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Cooperative Optimization of Traffic Signals and Mixed flow of Connected/Automated Vehicles and Human Driven Vehicles
Speaker: Shakiba Naderian
Abstract: Transportation is under rapid transformation with emerging technologies and systems. On the one hand, vehicles are equipped with advanced communication and automation capabilities, leading to connected and automated vehicles (CAVs). On the other hand, infrastructure (such as traffic signals and intersections) is increasingly installed with sensing and data collection systems (such as video cameras, Lidars, edge computing devices, etc.), enabling robust and fast data collection, sharing, and control of traffic flow. Naturally, infrastructure (e.g., traffic signals) and vehicles (e.g., CAVs) should be jointly optimized and controlled in future urban areas to improve safety, mobility, and other related goals. This research concerns about the models and algorithms of cooperative optimization and control of traffic signals and mixed flow of CAV and human driven vehicles (HDVs). The performance of the models and algorithms are tested in simulation, digital twins, and the Mcity 2.0 mixed reality testing environment.

Talk 2: Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles
Speaker: Zhili Zhang
Abstract: We address the problem of coordination and control of Connected and Automated Vehicles (CAVs) in the presence of imperfect observations in mixed traffic environment. A commonly used approach is learning-based decision-making, such as reinforcement learning (RL). However, most existing safe RL methods suffer from two limitations: (i) they assume accurate state information, and (ii) safety is generally defined over the expectation of the trajectories. It remains challenging to design optimal coordination between multi-agents while ensuring hard safety constraints under system state uncertainties (e.g., those that arise from noisy sensor measurements, communication, or state estimation methods) at every time step. We propose a safety guaranteed hierarchical coordination and control scheme called Safe-RMM to address the challenge. Specifically, the high-level coordination policy of CAVs in mixed traffic environment is trained by the Robust Multi-Agent Proximal Policy Optimization (RMAPPO) method. Though trained without uncertainty, our method leverages a worst-case Q network to ensure the model's robust performances when state uncertainties are present during testing. The low-level controller is implemented using model predictive control (MPC) with robust Control Barrier Functions (CBFs) to guarantee safety through their forward invariance property. We compare our method with baselines in different road networks in the CARLA simulator. Results show that our method provides best evaluated safety and efficiency in challenging mixed traffic environments with uncertainties.

Talk 3: Game-Theoretic Lane Choice at Highway Weaving Ramps: The Role of AV Altruism
Speaker: Ruolin Li
Abstract: Highway weaving ramps are notorious bottlenecks in modern traffic networks, where merging, exiting, and through flows co-exist in complex, often conflicting ways. In our work, we propose a comprehensive game-theoretic model that predicts the collective lane-changing behavior of mainline vehicles as they approach these high-conflict zones. By modeling drivers’ choices, whether to bypass the merging and exiting chaos by switching lanes or to remain in their current lane, using a concise set of parameters calibrated with minimal traffic data, our approach achieves remarkable predictive accuracy as demonstrated by microscopic SUMO simulations. We further introduce a two-level Stackelberg game framework tailored for mixed traffic weaving ramps that incorporate connected autonomous vehicles (CAVs). At the upper level, we govern the proactive, social-optimal lane-changing strategies of CAVs, while the lower level captures the reactive, self-interested behavior of human-driven vehicles (HDVs). Our analysis quantifies the optimal degree of altruism under varying CAV penetration rates, and reveals the delicate balance between individual benefits, fleet advantages, and societal gains, uncovering the interplay between selfish and altruistic driving behaviors. This adaptable framework offers a powerful tool for diagnosing and alleviating bottlenecks in a series of traffic scenarios such as weaving-ramps. Our findings not only deepen our understanding of AV-HV interactions but also pave the way for smarter, more efficient traffic management strategies in mixed autonomy environments.

Speakers
SN

Shakiba Naderian

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZZ

Zhili Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Ruolin Li

Ruolin Li

Assistant Professor, University of Southern California
Harnessing Autonomous Vehicles for Smarter Traffic Management - Autonomous vehicles (AVs) offer new opportunities to improve traffic flow, enhance system-wide coordination, and maximize societal benefits through their increased controllability and adaptability. However, their effective... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9O: Robustness in learning from data
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Robustness in learning from data
Chair: Jun-ya Gotoh
Cluster: Optimization For Data Science

Talk 1: The exploration-exploitation-robustness tradeoff for multi-period data driven problems with learning
Speaker: Andrew Lim
Abstract: We study the tradeoff between exploration, exploration and robustness in the setting of a robust optimal stopping problem with learning. We show that a decision maker (DM) concerned about model uncertainty explores less, even though additional data reduces model uncertainty, because the “learning shock” when it is collected increases the sensitivity of the expected reward to worst-case deviations from the nominal model. We also show that this “conservatism” can be fixed by introducing hedging instruments that offset the learning shocks. (With Thaisiri Watewai (Chulalongkorn University) and Anas Abdelhakmi (National University of Singapore)).

Talk 2: Biased Mean Quadrangle and Applications
Speaker: Anton Malandii
Abstract: The \emph{Risk Quadrangle} (RQ) is a framework that bridges risk management, optimization, and statistical estimation. Each RQ consists of four stochastic functionals—error, regret, risk, and deviation—linked together by a statistic. This paper introduces a new quadrangle, the \emph{biased mean quadrangle}, and studies its mathematical properties. In this quadrangle, the risk can be applied for risk management, while the error, referred to as the \emph{superexpectation error}, can be used for regression analysis. The statistic in this quadrangle is the mean value, adjusted by a real-valued parameter known as bias. Consequently, the linear regression within this quadrangle can be used to estimate the conditional biased mean. We demonstrate the extended regularity of this quadrangle and establish its connection to the quantile quadrangle. In particular, we prove the equivalence between biased mean and quantile regressions. Notably, when the bias is set to zero, the linear biased mean regression becomes equivalent to ordinary least squares (OLS) regression under standard statistical assumptions. Furthermore, the minimization of the superexpectation error reduces to a linear programming (LP) problem, allowing OLS to be reformulated as an LP. We provide numerical experiments that support these theoretical findings.

Talk 3: Convex vs. Nonconvex Regularization Terms---A Comparative Study of Regression B-Splines with Knot and Spline Selection
Speaker: Jun-ya Gotoh
Abstract: Robustness is important in learning from data. From the perspective of mathematical optimization, there are two contrasting approaches: robust optimization, which emphasizes worst-case samples, and robust regression, which reduces/ignores the contribution of unfavorable samples. The former tends to be realized by convex regularization, and the latter by non-convex regularization. On the other hand, l1 regularization, which is popular because it often leads to sparsity of the solution or associated quantities, is somewhere in between, but is closer to robust optimization in that it preserves convexity. In this presentation, we will compare convex and non-convex regularizations using knot selection and spline selection in multivariate B-spline regression as an example and discuss the choice between the two regularization methods from a practical per

Speakers
AL

Andrew Lim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AM

Anton Malandii

PhD Student, Stony Brook University
Name: Dr. Anton "Lightning Convergence" MalandiiTitle: Distinguished Professor of Discrete Acceleration & Energy MaximizationAffiliation: The Meteoric Summit Institute of Hyperactive ComputationBio:Dr. Anton Malandii is a trailblazing authority on rapid-fire optimization, renowned... Read More →
JG

Jun-ya Gotoh

Professor, Chuo university
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9P: Multi-Stage Stochastic Programming
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Multi-Stage Stochastic Programming
Chair: Yifan Hu
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Polynomial Complexity of Multi-Stage Stochastic Programming with Stagewise Dependent Randomness
Speaker: Yifan Hu
Abstract: We revisit the empirical approximation of multi-stage stochastic programming studied in Shapiro, 2006. For 3-stage problems, the previous results obtain an O(epsilon^{-4}) complexity bound to obtain an epsilon approximation. It was thus inferred that the sample complexity of T-stage problem admits an exponential dependence on T, i.e., O(epsilon^{-2T+2}). Such a conjecture forms a long-standing belief that multi-stage stochastic programming suffers from the curse of dimensionality. In this work, we construct a novel method for 3-stage problem that achieves an O(epsilon^{-2}) complexity bound, indicating that the complexity bounds for T-stage problem can be improved. We further construct a novel empirical approximation of T-stage problem and establish a polynomial complexity bound in terms of 1/epsilon.

Talk 2: Sample Complexity of Data-driven Multistage Stochastic Programming under Stagewise Dependent Uncertainty
Speaker: Hyuk Park
Abstract: This work addresses the challenges of applying the Sample Average Approximation (SAA) method to multistage stochastic programming under a stagewise-dependent data process. While SAA is commonly used in static and two-stage stochastic optimization, it becomes computationally intractable in general multistage settings as the time horizon $T$ increases, resulting in an exponential growth of approximation error, known as the curse of dimensionality in the time horizon. To overcome this limitation, we introduce a novel data-driven approach, the Markov Recombining Scenario Tree (MRST) method. MRST constructs an approximate problem using only two independent trajectories of historical data. We show that our method achieves polynomial sample complexity, providing a more efficient data-driven alternative to SAA. Numerical experiments on the Linear Quadratic Regulator (LQR) problem demonstrate that MRST outperforms SAA, successfully mitigating the curse of dimensionality.

Talk 3: Dual dynamic programming for stochastic programs over an infinite horizon
Speaker: Caleb Ju
Abstract: We consider a dual dynamic programming algorithm for solving stochastic programs over an infinite horizon. We show non-asymptotic convergence results when using an explorative strategy, and we then enhance this result by reducing the dependence of the effective planning horizon from quadratic to linear. This improvement is achieved by combining the forward and backward phases from dual dynamic programming into a single iteration. We then apply our algorithms to a class of problems called hierarchical stationary stochastic programs, where the cost function is a stochastic multi-stage program. The hierarchical program can model problems with a hierarchy of decision-making, e.g., how long-term decisions influence day-to-day operations. We show that when the subproblems are solved inexactly via a dynamic stochastic approximation-type method, the resulting hierarchical dual dynamic programming can find approximately optimal solutions in finite time. Preliminary numerical results show the practical benefits of using the explorative strategy for solving the Brazilian hydro-thermal planning problem and economic dispatch, as well as the potential to exploit parallel computing.

Speakers
YH

Yifan Hu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CJ

Caleb Ju

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9Q: Infinite-dimensional and dynamic aspects in optimization under uncertainty
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Infinite-dimensional and dynamic aspects in optimization under uncertainty
Chair: Caroline Geiersbach
Cluster: Nonsmooth Optimization

Talk 1: Scenario Approximation for Nonsmooth PDE-Constrained Optimization under Uncertainty
Speaker: Johannes Milz
Abstract: We study statistical guarantees for the scenario approximation method for PDE-constrained optimization with chance constraints. This sample-based technique replaces the original chance constraint with computationally tractable constraints derived from random samples. For example, when a chance constraint requires that a parameterized PDE state constraint be satisfied with high probability, the scenario approximation reformulates it into a standard PDE-constrained optimization problem, where the number of state constraints equals the number of samples. We derive estimates for the sample size needed to ensure, with high confidence, that feasible solutions to the original problem can be obtained through the scenario approximation. We then use these results to establish optimality guarantees for solutions to scenario-based PDE-constrained optimization problems. Our analysis is applicable to both linear and nonlinear PDEs with random inputs.

Talk 2: Risk-adjusted feedback control with PDE constraints
Speaker: Philipp Guth
Abstract: Effective control strategies that are directly applicable to different problem configurations, such as varying initial conditions, are highly desirable--especially in the presence of uncertainty. Unlike open-loop controllers, closed-loop (or feedback) controllers can be constructed independently of the initial condition; hence, this is one reason why they are favourable in the presence of uncertainty. This talk introduces a novel risk-adjusted feedback law specifically designed for risk-averse linear quadratic optimal control problems under uncertainty.

Talk 3: Pontryagin principle for deterministic control of random semilinear parabolic equations with almost sure state constraints
Speaker: Piero Visconti
Abstract: We study a class of optimal control problems governed by random semilinear parabolic equations with almost sure state constraints in the space of continuous functions. We obtain necessary conditions of optimality in the form of a maximum principle with two multipliers, one for the state constraint and one for the cost function, the multiplier for the state constraint takes values in a space of measures. We prove the nontriviality of the multipliers when the state constraint set has nonempty interior. Under a strong stability condition, the multiplier for the cost function can be suppressed.

Speakers
CG

Caroline Geiersbach

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PG

Philipp Guth

Postdoc, RICAM, Austrian Academy of Sciences
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PV

Piero Visconti

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9R: Algorithms for structured Riemannian optimization problems II
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Algorithms for structured Riemannian optimization problems II
Chair: Jiang Hu
Cluster: Optimization on Manifolds

Talk 1: An Inexact Riemannian Proximal DC Algorithm for Nonsmooth Riemannian DC Optimization
Speaker: Bo Jiang
Abstract: In this talk, we address a new class of nonsmooth Riemannian optimization problems where the objective is the sum of a smooth function and a nonsmooth difference-of-convex (DC) function. We first present application examples demonstrating the equivalence between our considered DC formulation (with an appropriately chosen nonsmooth DC term) and its corresponding $\ell_0$-regularized or $\ell_0$-constrained Riemannian optimization problem. We then propose an inexact Riemannian proximal algorithmic framework for solving these nonsmooth Riemannian DC problems and show that the proposed framework can return an $\epsilon$-Riemannian critical point of the considered problems within $\mathcal{O}(\epsilon^{-2})$ iterations. To efficiently solve the subproblems in this framework, we propose to solve their corresponding regularized dual problems in an inexact but controllable fashion. Specifically, by carefully selecting inexact criteria and leveraging first-order methods, we develop a practical inexact Riemannian proximal algorithm and establish its overall iteration complexity. To the best of our knowledge, this is the first algorithm for solving the considered nonsmooth Riemannian DC optimization problem with such a theoretical guarantee. Numerical results on the sparse principal component analysis problem validate the effectiveness of the DC models and the efficiency of the proposed algorithms.

Talk 2: A Flexible Algorithmic Framework for Nonconvex-Linear Minimax Problems on Riemannian Manifolds
Speaker: Meng Xu
Abstract: Recently, there has been growing interest in minimax problems on Riemannian manifolds due to their wide applications in machine learning and signal processing. Although many algorithms have been developed for minimax problems in the Euclidean setting, relatively few works studied minimax problems on manifolds. In this talk, we focus on the nonconvex-linear minimax problem on Riemannian manifolds. We propose a flexible Riemannian alternating descent ascent algorithmic framework and prove that the proposed framework achieves the best-known iteration complexity known to date. Various customized simple yet efficient algorithms can be incorporated within the proposed algorithmic framework and applied to different problem scenarios. We also reveal intriguing similarities and differences between the algorithms developed within our proposed framework and existing algorithms, which provide important insights into why the former outperform the latter. Lastly, we report extensive numerical results on sparse principal component analysis (PCA), fair PCA, and sparse spectral clustering to demonstrate the superior performance of the proposed algorithms.

Talk 3: Low-Rank Tensor Learning by Generalized Nonconvex Regularization
Speaker: Xiongjun Zhang
Abstract: In this paper, we study the problem of low-rank tensor learning, where only a few of training samples are observed and the underlying tensor has a low-rank structure. The existing methods are based on the sum of nuclear norms of unfolding matrices of a tensor, which may be suboptimal. In order to explore the low-rankness of the underlying tensor effectively, we propose a nonconvex model based on transformed tensor nuclear norm for low-rank tensor learning. Specifically, a family of nonconvex functions are employed onto the singular values of all frontal slices of a tensor in the transformed domain to characterize the low-rankness of the underlying tensor. An error bound between the stationary point of the nonconvex model and the underlying tensor is established under restricted strong convexity on the loss function (such as least squares loss and logistic regression) and suitable regularity conditions on the nonconvex penalty function. By reformulating the nonconvex function into the difference of two convex functions, a proximal majorization-minimization (PMM) algorithm is designed to solve the resulting model. Then the global convergence and convergence rate of PMM are established under very mild conditions. Numerical experiments are conducted on tensor completion and binary classification to demonstrate the effectiveness of the proposed method over other state-of-the-art methods.

Speakers
JH

Jiang Hu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Bo Jiang

Bo Jiang

Professor, Nanjing Normal University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MX

Meng Xu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XZ

Xiongjun Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9S: Robust Learning in Stochastic and Adaptive Environments
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Robust Learning in Stochastic and Adaptive Environments
Chair: Jiajin Li
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Limit Theorems for Stochastic Gradient Descent with Infinite Variance
Speaker: Wenhao Yang
Abstract: Stochastic gradient descent is a classic algorithm that has gained great popularity especially in the last decades as the most common approach for training models in machine learning. While the algorithm has been well-studied when stochastic gradients are assumed to have a finite variance, there is significantly less research addressing its theoretical properties in the case of infinite variance gradients. In this paper, we establish the asymptotic behavior of stochastic gradient descent in the context of infinite variance stochastic gradients, assuming that the stochastic gradient is regular varying. The closest result in this context was established in 1969, in the one-dimensional case and assuming that stochastic gradients belong to a more restrictive class of distributions. We extend it to the multidimensional case, covering a broader class of infinite variance distributions. As we show, the asymptotic distribution of the stochastic gradient descent algorithm can be characterized as the stationary distribution of a suitably defined Ornstein-Uhlenbeck process driven by an appropriate stable Lévy process.

Talk 2: Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification
Speaker: Chao Qin
Abstract: Practitioners conducting adaptive experiments often encounter two competing priorities: maximizing total welfare (or reward') through effective treatment assignment and swiftly concluding experiments to implement population-wide treatments. Current literature addresses these priorities separately, with regret minimization studies focusing on the former and best-arm identification research on the latter. This paper bridges this divide by proposing a unified model that simultaneously accounts for within-experiment performance and post-experiment outcomes. We provide a sharp theory of optimal performance in large populations that not only unifies canonical results in the literature but also uncovers novel insights. Our theory reveals that familiar algorithms, such as the recently proposed top-two Thompson sampling algorithm, can optimize a broad class of objectives if a single scalar parameter is appropriately adjusted. In addition, we demonstrate that substantial reductions in experiment duration can often be achieved with minimal impact on both within-experiment and post-experiment regret.

Talk 3: A Definition of Non-Stationary Bandits
Speaker: Yueyang Liu
Abstract: Despite the subject of non-stationary bandit learning having attracted much recent attention, we have yet to identify a formal definition of non-stationarity that can consistently distinguish non-stationary bandits from stationary ones. Prior work has characterized non-stationary bandits as bandits for which the reward distribution changes over time. We demonstrate that this definition can ambiguously classify the same bandit as both stationary and non-stationary; this ambiguity arises in the existing definition’s dependence on the latent sequence of reward distributions. Moreover, the definition has given rise to two widely used notions of regret: the dynamic regret and the weak regret. These notions are not indicative of qualitative agent performance in some bandits. Additionally, this definition of non-stationary bandits has led to the design of agents that explore excessively. We introduce a formal definition of non-stationary bandits that resolves these issues. Our new definition provides a unified approach, applicable seamlessly to both Bayesian and frequentist formulations of bandits. Furthermore, our definition ensures consistent classification of two bandits offering agents indistinguishable experiences, categorizing them as either both stationary or both non-stationary. This advancement provides a more robust framework for agent design and analysis in non-stationary bandit learning.

Speakers
JL

Jiajin Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WY

Wenhao Yang

Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CQ

Chao Qin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Yueyang Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9T: Relaxations of Optimization Problems and Extreme Point Results in Infinite Dimensions
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Relaxations of Optimization Problems and Extreme Point Results in Infinite Dimensions
Chair: Rahul Parhi
Cluster: Nonsmooth Optimization

Talk 1: On Extremal Points for Some Vectorial Total Variation Seminorms
Speaker: Daniel Walter
Abstract: We consider the set of extremal points of the generalized unit ball induced by gradient total variation seminorms for vector-valued functions on bounded Euclidean domains. These extremal points are central to the understanding of sparse solutions and sparse optimization algorithms for variational regularization problems posed among such functions. For not fully vectorial cases in which either the domain or the target are one dimensional, or the sum of the total variations of each component is used, we prove that these extremals are fully characterized as in the scalar-valued case, that is, they consist of piecewise constant functions with two regions. For definitions involving more involved matrix norms and in particular spectral norms, which are of interest in image processing, we produce families of examples to show that the resulting set of extremal points is larger and includes piecewise constant functions with more than two regions. We also consider the total deformation induced by the symmetrized gradient, for which minimization with linear constraints appears in problems of determination of limit loads in a number of continuum mechanical models involving plasticity, bringing relevance to the corresponding extremal points. For this case, we show piecewise infinitesimally rigid functions with two pieces to be extremal under mild assumptions. Finally, as an example of an extremal which is not piecewise constant, we prove that unit radial vector fields are extremal for the Frobenius total variation in the plane.

Talk 2: Exact Sparse Representation Recovery for Convex Optimization Problems
Speaker: Marcello Carioni
Abstract: We investigate the recovery of the sparse representation of data in general infinite-dimensional optimization problems regularized by convex functionals. We show that it is possible to define a suitable non-degeneracy condition on the minimal-norm dual certificate, extending the well-established non-degeneracy source condition (NDSC) associated with total variation regularized problems in the space of measures, as introduced in (Duval and Peyré, FoCM, 15:1315-1355, 2015). In our general setting, we need to study how the dual certificate is acting, through the duality product, on the set of extreme points of the ball of the regularizer, seen as a metric space. This justifies the name Metric Non-Degenerate Source Condition (MNDSC). More precisely, we impose a second-order condition on the dual certificate, evaluated on curves with values in small neighbourhoods of a given collection of n extreme points. By assuming the validity of the MNDSC, together with the linear independence of the measurements on these extreme points, we establish that, for a suitable choice of regularization parameters and noise levels, the minimizer of the minimization problem is unique and is uniquely represented as a linear combination of n extreme points. The paper concludes by obtaining explicit formulations of the MNDSC for three problems of interest. First, we examine total variation regularized deconvolution problems, showing that the classical NDSC implies our MNDSC, and recovering a result similar to (Duval and Peyré, FoCM, 15:1315-1355, 2015). Then, we consider 1-dimensional BV functions regularized with their BV-seminorm and pairs of measures regularized with their mutual 1-Wasserstein distance. In each case, we provide explicit versions of the MNDSC and formulate specific sparse representation recovery results.

Talk 3: Extensions of Optimization Problems and Representer Theorems
Speaker: Thibaut Horel
Abstract: We conduct a general investigation of extensions of convex optimization problems in infinite-dimensional spaces, including for example regularized empirical risk minimization and signal reconstruction problems. It turns out that many such problems (such as those minimizing L^1-type norms) do not admit minimizers over their primal space, but do exhibit a minimizer over a minimally extended space (for L^1-type spaces, the extension could be a space of Radon measures). With this observation in hand, we provide a systematic treatment of extensions of optimization problems in the sense of Ioffe and Tikhimirov (Tr. Mosk. Mat. Obs., 1968). In particular, we show how to extend, in a principled manner, a convex optimization problem to its bidual space in a way that preserves the optimal value and such that the extended problem admits a minimizer. The objective function of the extended problem is derived by taking the biconjugate of the original function. Under mild regularity conditions, biconjugation commutes with addition and linear operators, allowing the extended problem to retain the structure of the original. This allows us to extend the scope of recently proposed abstract representer theorems to problems that did not admit a minimizer in their primal space by considering their bidual extension. As a byproduct, the interplay between different extensions provides a fresh perspective on previously studied sparse representation recovery problems. (Joint work with Rahul Parhi)

Speakers
RP

Rahul Parhi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DW

Daniel Walter

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MC

Marcello Carioni

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TH

Thibaut Horel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9U: Quantum Computing for combinatorial optimization
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Quantum Computing for combinatorial optimization
Chair: David Bernal Neira
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Super-Quadratic Speedups in Exact Combinatorial Optimization
Speaker: Shouvanik Chakrabarti
Abstract: Recent studies of error correction overheads for quantum algorithms have highlighted the need to identify large algorithmic speedups for industrial problems. In this talk, we describe some recent efforts to rigorously develop and prove these speedups. The focus will be on super-quadratic speedups over Markov Chain search (arXiv:2410.23270) arising from a generalization and extensions of the short path framework of Hastings, and its future refinements by Dalzell, Pancotti, Campbell, and Brandao.

Talk 2: Mixed-integer programming using a photonic quantum computer
Speaker: Farhad Khosravi
Abstract: We propose a scheme for solving mixed-integer programming problems in which the optimization problem is translated to a ground-state preparation problem on a set of bosonic quantum field modes (qumodes). We perform numerical demonstrations by simulating a circuit-based optical quantum computer with each individual qumode prepared in a Gaussian state. We simulate an adiabatic evolution from an initial mixing Hamiltonian, written in terms of the momentum operators of the qumodes, to a final Hamiltonian which is a polynomial of the position and boson number operators. In these demonstrations, we solve a variety of small non-convex optimization problems in integer programming, continuous non-convex optimization, and mixed-integer programming.

Talk 3: Quantum-Inspired Heuristic solvers for Binary Higher-Order and Mixed-Integer Constrained Problems
Speaker: Niraj Kumar
Abstract: Quantum-Inspired Heuristic solvers have recently emerged as techniques inspired from quantum annealing to solve hard optimization problems (arXiv: 2104.14096). While previous studies have primarily focused on graph-based problems with binary variables and quadratic unconstrained objective functions (e.g., MaxCUT), we numerically analyze the performance of these solvers for more complex problems such as Low-correlation Binary Sequence (binary quartic objective function), and mixed-integer problem formulation with inequality constraints. Our results indicate the promise these solvers towards industrially relevant problems, along with highlighting the outstanding unresolved questions.

Speakers
DB

David Bernal Neira

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SC

Shouvanik Chakrabarti

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FK

Farhad Khosravi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
NK

Niraj Kumar

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9V: Modeling Oriented Software and Methods (1)
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Modeling Oriented Software and Methods (1)
Chair: Jean-Paul Watson
Cluster: Computational Software

Talk 1: Advances in Modeling and Solving Stochastic Programs
Speaker: Jean-Paul Watson
Abstract: TBd

Talk 2: What's new in Pyomo?
Speaker: Bethany Nicholson
Abstract: TBD

Talk 3: Tbd
Speaker: Robert Parker
Abstract: TBD

Speakers
JW

Jean-Paul Watson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BN

Bethany Nicholson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9W: Bilevel Optimization - Applications
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Bilevel Optimization - Applications
Chair: Tommaso Giovannelli
Cluster: nan

Talk 1: (Canceled) Optimal Electric Vehicle Charging with Dynamic Pricing, Customer Preferences and Power Peak Reduction
Speaker: Luce Brotcorne
Abstract: (Canceled) We consider a provider of electric vehicle charging stations that operates a network of charging stations and use time varying pricing to maximize profit and reduce the impact on the electric grid. We propose a bilevel model with a single leader and multiple disjoint followers. The provider (leader) sets the price of charging for each station at each time slot, and ensures there is enough energy to charge. The charging choice of each customer is represented by a combination of a preference list of (station, time) pairs and a reserve price. The proposed model takes thus into accounts for the heterogeneity of customers with respect to price sensitivity. We define a single-level reformulation based on a reformulation approach from the literature on product line optimization, and we report computational results that highlight the efficiency of the new reformulation and the potential impact for reducing peaks on the electricity grid.

Talk 2: A Stochastic Gradient Method for Trilevel Optimization
Speaker: Tommaso Giovannelli
Abstract: With the recent success of bilevel optimization in machine learning applications, stochastic optimization formulations have begun to emerge for trilevel optimization, such as those involving hyperparameter tuning via adversarial training. In these formulations, the upper level minimizes the loss on validation data over the neural network's hyperparameters, the middle level determines the weights to minimize the loss on the training data, and the lower level maximizes such a training loss by adding worst-case perturbations to the data. The challenge is that trilevel first-order methods require second- or third-order derivatives, which become impractical to compute in problems involving a large number of variables. In this work, we propose the first-ever stochastic gradient descent method for solving unconstrained trilevel optimization problems. We also present a convergence theory that covers all inexact calculations of the trilevel adjoint gradient, such as the inexact solutions of the middle- and lower-level problems, inexact computation of the adjoint formula, and noisy estimates of the gradients, Hessians, Jacobians, and tensors of third-order derivatives involved. To promote the use of trilevel optimization in large-scale learning, we have developed practical trilevel stochastic gradient methods that extend approaches proposed for bilevel optimization and do not require second- or third-order derivatives.

Talk 3: Learning prosumer behavior in energy communities: Fusing bilevel programming and online learning
Speaker: Lesia Mitridati
Abstract: Dynamic pricing through bilevel programming is widely used for demand response but often assumes perfect knowledge of prosumer behavior, which is unrealistic in practical applications. This paper presents a novel framework that integrates bilevel programming with online learning, specifically Thompson sampling, to overcome this limitation. The approach dynamically sets optimal prices while simultaneously learning prosumer behaviors through observed responses, eliminating the need for extensive pre-existing datasets. Applied to an energy community providing capacity limitation services to a distribution system operator, the framework allows the community manager to infer individual prosumer characteristics, including usage patterns for photovoltaic systems, electric vehicles, home batteries, and heat pumps. Numerical simulations with 25 prosumers, each represented by 10 potential signatures, demonstrate rapid learning with low regret, with most prosumer characteristics learned within five days and full convergence achieved in 100 days.

Speakers
LB

Luce Brotcorne

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TG

Tommaso Giovannelli

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LM

Lesia Mitridati

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9X: Conic and Semidefinite Programming (SDP)
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Session: Conic and Semidefinite Programming (SDP)
Chair: Takashi Tsuchiya
Cluster: nan

Talk 1: Computational complexity of sum-of-squares bounds for copositive programs
Speaker: Marilena Palomba
Abstract: In recent years, copositive programming has received significant attention for its ability to model hard problems in both discrete and continuous optimization. Several relaxations of copositive programs based on semidefinite programming (SDP) have been proposed in the literature, meant to provide tractable bounds. However, while these SDP-based relaxations are amenable to the ellipsoid algorithm and interior point methods, it is not immediately obvious that they can be solved in polynomial time (even approximately). In this paper, we consider the sum-of-squares (SOS) hierarchies of relaxations for copositive programs introduced by Parrilo (2000), de Klerk & Pasechnik(2002) and Peña, Vera & Zuluaga (2006), which can be formulated as SDPs. We establish sufficient conditions that guarantee the polynomial-time computability (up to fixed precision) of these relaxations. These conditions are satisfied by copositive programs that represent standard quadratic programs and their reciprocals. As an application, we show that the SOS bounds for the (weighted) stability number of a graph can be computed efficiently. Additionally, we provide pathological examples of copositive programs (that do not satisfy the sufficient conditions) whose SOS relaxations admit only feasible solutions of doubly-exponential size. This work was conducted in collaboration with Lucas Slot (ETH Zurich, lucas.slot@inf.ethz.ch), Luis Felipe Vargas (SUPSI, IDSIA, luis.vargas@supsi.ch) and Monaldo Mastrolilli (SUPSI, IDSIA, monaldo.mastrolilli@supsi.ch) Full article available here: https://doi.org/10.48550/arXiv.2501.03698

Talk 2: Towards closing nonzero duality gaps of highly singular SDPs by perturbation
Speaker: Takashi Tsuchiya
Abstract: Let us consider general conic convex programs with finite nonzero duality gaps, where both Primal (P) and Dual (D) are weakly feasible. The optimal values of primal and dual are different, but there are arbitrary small perturbations which can close the duality gap (to zero). Let (eps,eta) be nonnegative, and, Let P(eps,eta) and D(eps,eta) are perturbed primal and dual problems with the conic constraints shifted by eps*e and eta*e' to make the both problems strongly feasible where e and e' are interior-points of the primal and dual cones, respectively. If (eps,eta) is nonnegative and nonzero, then at least one of P(eps,eta) and D(eps,eta) is strongly feasible, ensuring strong duality to hold. Let v(eps,eta) be the common optimal value of P(eps, eta) and D(eps,eta). v is well-defined except for (eps,eta)=(0,0). In the case of SDP, we can show that lim v(t*a,t*b) exists when t>0 goes to zero for any (a,b) nonnegative and nonzero. We denote this limit by lv(a,b). Since lv depends on the ratio between a and b, we define lv'(theta)=lv(cos(theta),sin(theta)), where the domain of lv' is [0,pi/2]. We can show that lv' is monotonically decreasing on [0,pi/2] and further continuous on (0,pi/2), with v(0) and v(pi/2) are primal and dual optimal values, respectively [1]. We can prove continuity at theta=0 and theta=pi/2 when singularity degree of (P) and (D) is one, but this does not holds in general singular SDPs with higher singularity degree [2]. It is an interesting problem if we can recover ``continuity'' by considering lim v(f(t)*a,g(t)*b) instead lim v(t*a,t*b), where f and g are appropriate power functions in t. In this talk, we discuss recent developments in this direction and possible extensions to general conic convex programs. [1] Takashi Tsuchiya, Bruno F. Lourenco, Masakazu Muramatsu and Takayuki Okuno: A limiting analysis on regularization of singular SDP and its implication to infeasible interior-point algorithms. Mathematical Programming, Vol.200 (2023), pp.531–568. [2] Takashi Tsuchiya, Bruno F. Lourenco, Masakazu Muramatsu and Takayuki Okuno: Closing duality gaps of SDPs completely through perturbation when singularity degree is one. Optimization Methods and Software, Vol.39 (2024), pp.1040-1067.

Talk 3: Extending the Dahl-Andersen interior-point algorithm to general power cone: practicalities and benchmarks
Speaker: Utkarsh Detha
Abstract: The interior-point algorithm for non-symmetric conic problems as prescribed by Dahl and Andersen [DA] uses primal-dual scalings analogous to Nesterov-Todd's approach for symmetric cones. The practical performance of [DA] method is good, driven in part by the use of a higher-order correction similar to the Mehrotra corrector in LPs. The [DA] algorithm is outlined for a 3-D exponential cone while applicability to the 3-D power cone is mentioned. This method for both cones is implemented in MOSEK, which also handles general (m,n) power cones [Cha09] by splitting them into m-2 cones of three dimensions. A follow-up consideration is extending the method to directly handle general (m, n) power cone. Among other things, this is motivated by an expected reduction in iterations since an LHSCB barrier for the general power cone with barrier parameter m+1 exists, in contrast to a parameter of nearly 3m when the cone is split into m-2 3-D cones. The aim of this talk will be two-fold. Firstly, the extension of the DA algorithm to the (m,n) power cone will be discussed with a focus on implementation details and comparison with the approach of splitting into smaller cones. To understand if the scaling approach used in 3-D case also extends its effectiveness over to larger cones, benchmarks will be considered. Secondly, the talk will provide an alternative approach to obtain the higher-order corrector introduced in [DA]. References: [DA]: Joachim Dahl and Erling D. Andersen. A primal-dual interior-point algorithm for nonsymmetric exponential-cone optimization. 194(1):341–370. [Cha09]: Robert Chares. Cones and interior-point algorithms for structured convex optimization involving powers and exponentials. PhD thesis, Université catholique de Louvain, 2009

Speakers
avatar for Marilena Palomba

Marilena Palomba

PhD Student, IDSIA USI-SUPSI
Name: Marilena PalombaTitle: PhD StudentAffiliation: IDSIA USI-SUPSI (University of Applied Sciences and Arts of Southern Switzerland (SUPSI) - Dalle Molle Institute for Artificial Intelligence (IDSIA) - Università della Svizzera italiana (USI))Bio:I am a Ph.D. student in Theoretical... Read More →
TT

Takashi Tsuchiya

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
UD

Utkarsh Detha

Industrial PhD, MOSEK ApS
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 9Y
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Wednesday July 23, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

5:30pm PDT

Break or End of day
Wednesday July 23, 2025 5:30pm - 6:30pm PDT
Wednesday July 23, 2025 5:30pm - 6:30pm PDT
TBA

6:30pm PDT

Conference banquet (ADVANCED PURCHASE REQUIRED)
Wednesday July 23, 2025 6:30pm - 9:00pm PDT
Wednesday July 23, 2025 6:30pm - 9:00pm PDT
Town & Gown USC 665 W Exposition Blvd, Los Angeles, CA 90089
 
Thursday, July 24
 

8:15am PDT

Auditorium Opens Doors for seating
Thursday July 24, 2025 8:15am - 8:45am PDT
Thursday July 24, 2025 8:15am - 8:45am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

8:45am PDT

Last Day Remarks
Thursday July 24, 2025 8:45am - 9:00am PDT
Thursday July 24, 2025 8:45am - 9:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

9:00am PDT

Plenary 4
Thursday July 24, 2025 9:00am - 10:00am PDT
Speakers
AD

Alexandre d'Aspremont

After dual PhDs from Ecole Polytechnique and Stanford University in optimisation and finance, followed by a postdoc at U.C. Berkeley, Alexandre d'Aspremont joined the faculty at Princeton University as an assistant then associate professor with joint appointments at the ORFE department... Read More →
Thursday July 24, 2025 9:00am - 10:00am PDT
USC Bovard Auditorium 3551 Trousdale Pkwy, Los Angeles, CA 90089

10:00am PDT

Coffee & Snack Break (Provided)
Thursday July 24, 2025 10:00am - 10:30am PDT
Thursday July 24, 2025 10:00am - 10:30am PDT
TBA

10:30am PDT

Parallel Sessions 10A: AI Meets Optimization (Part 3)
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: AI Meets Optimization (Part 3)
Chair: Wotao Yin
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: TBD
Speaker: Ming Jin
Abstract: TBD

Talk 2: TBD
Speaker: Pascal Van Hentenryck
Abstract: TBD

Talk 3: TBD
Speaker: Samy Wu Fung
Abstract: TBD

Speakers
MJ

Ming Jin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PV

Pascal Van Hentenryck

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SW

Samy Wu Fung

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10B: Special Session in Honor of Suvrajeet Sen: Stochastic Mixed-Integer Programming
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Special Session in Honor of Suvrajeet Sen: Stochastic Mixed-Integer Programming
Chair: Lewis Ntaimo
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: On Disjunctive Decomposition for Stochastic Mixed-Integer Programming: A Reflection on Algorithm Development and Applications
Speaker: Lewis Ntaimo
Abstract: Two-stage stochastic mixed-integer programming (SMIP) involves making discrete decisions in the face of future uncertainty and has many applications in science and engineering. However, solving SMIP is very challenging mainly due to its nonconvexity and large-scale nature. In this talk, we reflect on the development of disjunctive decomposition for SMIP initiated by Suvrajeet Sen. Disjuctive decomposition relies on generating disjunctive cutting planes that are shared among scenarios to sequentially convexify the nonconvex expected recourse function. We review the basic theory and derivation of a class of disjunctive decomposition algorithms, and illustrate the algorithms using simple numerical examples. Finally, we discuss computer implementation and application of the algorithms towards solving standard SMIP problems.

Talk 2: An Asymptotically Optimal Coordinate Descent Algorithm for Learning Bayesian Networks from Gaussian Models
Speaker: Simge Kucukyavuz
Abstract: We study the problem of learning Bayesian networks from continuous observational data, generated according to a linear Gaussian structural equation model. We consider an ℓ0-penalized maximum likelihood estimator for this problem which is known to have favorable statistical properties but is computationally challenging to solve, especially for medium-sized Bayesian networks. We propose a new coordinate descent algorithm to approximate this estimator and prove several remarkable properties of our procedure: the algorithm converges to a coordinate-wise minimum, and despite the non-convexity of the loss function, as the sample size tends to infinity, the objective value of the coordinate descent solution converges to the optimal objective value of the ℓ0-penalized maximum likelihood estimator. Finite-sample optimality and statistical consistency guarantees are also established. To the best of our knowledge, our proposal is the first coordinate descent procedure endowed with optimality and statistical guarantees in the context of learning Bayesian networks. Numerical experiments on synthetic and real data demonstrate that our coordinate descent method can obtain near-optimal solutions while being scalable.

Talk 3: A Stochastic Diversion Path Problem
Speaker: Cole Smith
Abstract: We examine a stochastic network optimization problem in which the goal is to modify arc lengths so that a specified path (the “diversion path”) will be optimal with sufficiently high probability. Given modification costs for each arc in the network, the objective in a deterministic form of the problem would be to minimize the sum of modification costs needed to guarantee that the diversion path is optimal. In the stochastic version, each arc length is an independent, uniformly-distributed random variable. (The lower bound on the arc lengths is nonnegative.) Given a parameter 0 < tau
Speakers
avatar for Lewis Ntaimo

Lewis Ntaimo

Professor and Head, Texas A&M University
Name: Lewis NtaimoTitle: Professor and Department HeadAffiliation: Texas A&M UniversityBio:Fun Fact:
avatar for Simge Kucukyavuz

Simge Kucukyavuz

Chair and David A. and Karen Richards Sachs Professor, Northwestern University
Name: Simge KüçükyavuzTitle: Chair and David A. and Karen Richards Sachs ProfessorAffiliation: Northwestern UniversityBio:Simge Küçükyavuz is Chair and David A. and Karen Richards Sachs Professor in the Industrial Engineering and Management Sciences Department at Northwestern... Read More →
CS

Cole Smith

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10C: Mixed-Integer Programming-Based Techniques for the Global Optimization of MINLPs
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Mixed-Integer Programming-Based Techniques for the Global Optimization of MINLPs
Chair: Rohit Kannan
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Discrete nonlinear functions: formulations and applications
Speaker: Taotao He
Abstract: This paper examines nonlinear optimization problems that incorporate discrete decisions. We introduce new improved formulation techniques that take advantage of the simplotope structure present in the domain of the binarization variables. Our technique identifies new polynomially solvable instances for price promotion problem initially studied by Cohen et al. (2021) and allows us to develop a linear programming (LP) formulation for inventory optimization problem under a choice model proposed by Boada-Collado and Martinez-de Albeniz (2020). The techniques we develop rely on ideal formulations for submodular and fractional compositions of discrete functions, improving prior formulations for bilinear products suggested by Adams and Henry (2012). Submodular compositions also generalize L natural functions over bounded domains and our construction provides new insights into Lovasz-extension based formulations for this class of problems while expanding the class of nonlinear discrete optimization problems amenable to linear programming based techniques.

Talk 2: Quadratic Cuts for Non-Convex MINLP in Practice
Speaker: Adrian Göß
Abstract: It is only half the job to find a good solution for a mathematical optimization problem, as one needs to verify its quality by specifying a dual bound. When it comes to mixed-integer nonlinear programming (MINLP), strong prerequisites such as constraint qualifications appear suitable, but may be difficult to verify computationally. In practice, solvers apply local refinement or convexification strategies to retrieve tight dual bounds. However, these concepts require appropriate big-M formulations, generate new sub-problems, or struggle to represent non-convex characteristics in terms of high accuracy, all of which can lead to long running times. As an alternative, we aim to leverage recent advances in mixed-integer quadratically-constrained programming (MIQCP) and propose a global approximation of constraint functions by paraboloids, \ie, univariate quadratic terms. The approximation is retrieved as a solution to a mixed-integer linear programming (MIP) problem. Further, for each nonlinear constraint function, we solve such MIPs and determine small numbers of paraboloids approximating it from either side. A replacement of the nonlinearities with the corresponding quadratic functions leads to a quadratically-constrained relaxation of the original problem. Solving the MIQCP relaxation then leads to a dual bound whose tightness depends on the approximation guarantee of the paraboloids. In summary, this approach enables solvers that are explicitly tailored for quadratic constraints to solve MINLPs to global optimality.

Talk 3: Learning to Accelerate the Global Optimization of QCQPs: Strong Partitioning and an End-to-End Graph ML-Based Approach
Speaker: Rohit Kannan
Abstract: We learn optimal instance-specific heuristics for the global minimization of nonconvex quadratically-constrained quadratic programs (QCQPs). Specifically, we consider partitioning-based convex mixed-integer programming relaxations for nonconvex QCQPs and propose the novel problem of strong partitioning, which aims to optimally partition variable domains without sacrificing global optimality. Given that solving this max-min strong partitioning problem exactly can be highly challenging, we design a local optimization method that leverages the generalized gradients of the value function in the inner minimization problem. However, solving the strong partitioning problem to local optimality can still be computationally expensive. To address this, we propose a simple AdaBoost regression-based machine learning (ML) approximation for homogeneous families of QCQPs. We conduct a detailed computational study on randomly generated QCQP families, including instances of the pooling problem, using the open-source global solver Alpine. Numerical experiments demonstrate that our AdaBoost regression-based approximation of strong partitioning reduces Alpine’s solution time by a factor of 2 to 4.5 on average, with maximum reduction factors ranging from 10 to 200 across different QCQP families. Additionally, we design a novel end-to-end loss function to train a neural network-based policy that directly maps QCQP instance features to corresponding partitioning points, eliminating the need for an expert strong partitioning policy. We also demonstrate how to apply the graph scattering transform to train a single neural network model that generalizes across QCQP families of various sizes. Our results show that this graph scattering-based end-to-end neural network learns a partitioning policy that significantly outperforms the AdaBoost-based approach and generalizes well to larger QCQPs beyond those encountered in training.

Speakers
TH

Taotao He

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AG

Adrian Göß

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RK

Rohit Kannan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10D: Optimization Meets Generative AI: Insights and New Designs
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Optimization Meets Generative AI: Insights and New Designs
Chair: Minshuo Chen
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Fine Tuning And Guidance of Diffusion Models
Speaker: Wenpin Tang
Abstract: The past decade has witnessed the success of generative modeling (e.g. GANs, VAEs,...) in creating high quality samples in a wide variety of data modalities. In the first part of this talk, I will briefly introduce the recently developed diffusion models from a continuous-time perspective. Then in the second part, I will discuss three different approaches to fine-tune the diffusion models: conditioning (classifier guidance), stochastic control and reinforcement learning. Each of these approaches will lead to a nice theory with a few application fields. If time permits, I will also discuss the DPO (Direct preference optimization) approach to fine-tuning text-to-image models.

Talk 2: How Does Gradient Descent Learn Features -- A Local Analysis for Regularized Two-Layer Neural Networks
Speaker: Mo Zhou
Abstract: The ability of learning useful features is one of the major advantages of neural networks. Although recent works show that neural networks can operate in a neural tangent kernel (NTK) regime that does not allow feature learning, many works also demonstrate the potential for neural networks to go beyond NTK regime and perform feature learning. Recently, a line of work highlighted the feature learning capabilities of the early stages of gradient-based training. In this paper we consider another mechanism for feature learning via gradient descent through a local convergence analysis. We show that once the loss is below a certain threshold, gradient descent with a carefully regularized objective will capture ground-truth directions. Our results demonstrate that feature learning not only happens at the initial gradient steps, but can also occur towards the end of training.

Talk 3: Theoretical Implications of Training And Sampling Diffusion Models
Speaker: Yuqing Wang
Abstract: Most existing theoretical investigations of the accuracy of diffusion models, albeit significant, assume the score function has been approximated to a certain accuracy, and then use this a priori bound to control the error of generation. In this talk, I will show a quantitative understanding of the whole generation process, i.e., both training and sampling. More precisely, it conducts a non-asymptotic convergence analysis of denoising score matching under gradient descent. In addition, a refined sampling error analysis for variance exploding models is also provided. The combination of these two results yields a full error analysis, which elucidates (again, but this time theoretically) how to design the training and sampling processes for effective generation. For instance, our theory implies a preference toward noise distribution and loss weighting in training that qualitatively agree with the ones used in Karras et al. 2022. It also provides perspectives on the choices of time and variance schedules in sampling: when the score is well trained, the design in Song et al. 2021 is more preferable, but when it is less trained, the design in Karras et al. 2022 becomes more preferable.

Speakers
MC

Minshuo Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WT

Wenpin Tang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MZ

Mo Zhou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YW

Yuqing Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10E: Data-Driven Optimization with Structured Models
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Data-Driven Optimization with Structured Models
Chair: Krishnakumar Balasubramanian
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Feature Learning in Two-layer Neural Networks under Structured Data
Speaker: Murat Erdogdu
Abstract: We study the effect of gradient-based optimization on feature learning in two-layer neural networks. We consider a setting where the number of samples is of the same order as the input dimension and show that, when the input data is isotropic, gradient descent always improves upon the initial random features model in terms of prediction risk, for a certain class of targets. Further leveraging the practical observation that data often contains additional structure, i.e., the input covariance has non-trivial alignment with the target, we prove that the class of learnable targets can be significantly extended, demonstrating a clear separation between kernel methods and two-layer neural networks in this regime. We additionally consider sparse settings and show that pruning methods can lead to optimal sample complexity.

Talk 2: Control, Transport and Sampling: Towards Better Loss Design
Speaker: Qijia Jiang
Abstract: Leveraging connections between diffusion-based sampling, optimal transport, and stochastic optimal control through their shared links to the Schrodinger bridge problem, we propose novel objective functions that can be used to transport ν to μ, consequently sample from the target μ, via optimally controlled dynamics. We highlight the importance of the pathwise perspective and the role various optimality conditions on the path measure can play for the design of valid training losses, the careful choice of which offer numerical advantages in implementation. Basing the formalism on Schrodinger bridge comes with the additional practical capability of baking in inductive bias when it comes to Neural Network training.

Talk 3: Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
Speaker: Mladen Kolar
Abstract: In this work, we consider solving optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Sequential Quadratic Programming method to find both first- and second-order stationary points. Our method utilizes a random model to represent the objective function, which is constructed from stochastic observations of the objective and is designed to satisfy proper adaptive accuracy conditions with a high but fixed probability. To converge to first-order stationary points, our method computes a gradient step in each iteration defined by minimizing a quadratic approximation of the objective subject to a (relaxed) linear approximation of the problem constraints and a trust-region constraint. To converge to second-order stationary points, our method additionally computes an eigen step to explore the negative curvature of the reduced Hessian matrix, as well as a second-order correction step to address the potential Maratos effect, which arises due to the nonlinearity of the problem constraints. Such an effect may impede the method from moving away from saddle points. Both gradient and eigen step computations leverage a novel parameter-free decomposition of the step and the trust-region radius, accounting for the proportions among the feasibility residual, optimality residual, and negative curvature. We establish global almost sure first- and second-order convergence guarantees for our method, and present computational results on CUTEst problems, regression problems, and saddle-point problems to demonstrate its superiority over existing line-search-based stochastic methods. Joint work with Yuchen Fang, Sen Na, and Michael W Mahoney.

Speakers
KB

Krishnakumar Balasubramanian

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ME

Murat Erdogdu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
QJ

Qijia Jiang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MK

Mladen Kolar

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10F: Optimisation and machine learning in energy
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Optimisation and machine learning in energy
Chair: Hongyu Zhang
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: A Computationally Efficient Cutting Plane Modelling to Generate Alternatives Algorithm
Speaker: Michael Lau
Abstract: Contemporary macro-energy systems modelling is characterized by the need to represent energy systems strategic and operational decisions with high temporal and spatial resolution, which provides more accurate results than more abstracted models. This drive towards greater fidelity, however, conflicts with a push towards greater model representation of inherent complexity in decision-making, including methods like Modelling to Generate Alternatives. Modelling to Generate Alternatives aims to map the feasible space of a model within a cost slack by varying investment parameters without changing the operational constraints, a process which frequently requires hundreds of solutions. For large, highly representative energy system models this is impossible with traditional methods, leading researchers to reduce complexity with either more zonal or temporal aggregation. This research presents a new solution method for Modelling to Generate Alternatives-type problems. Using Cutting Plane methods based on a reformulation of Bender’s Decomposition, we break down the problem structure into a strategic master problem and operational subproblems and pass information between master problems to accelerate convergence with each new objective. We find that our new solution method is several times faster and requires less memory than existing parallelized monolithic Modelling to Generate Alternatives solution methods, enabling rapid computation of a greater number of solutions to highly resolved models.

Talk 2: Multi-timescale stochastic programming with application in power systems
Speaker: Yihang Zhang
Abstract: We introduce a multi-timescale stochastic programming framework for decision-making under multi-timescale uncertainty. Aggregated state variables are used to coordinate decisions across timescales, similar to the role of state variables in a multistage problem. Based on this setup, we describe instantiation strategies that use either multi-horizon scenario trees (to model multi-lag dependence on a timescale) or a specialized value function to fully exploit independence of the randomness within the timescale. We develop decomposition algorithms (price-directive or resource-directive) to incrementally approximate and solve the resulting problem. In addition to techniques used in multistage problems, we describe solution-reusing heuristics to accelerate the solution process by leveraging similarities between subproblems.

Talk 3: A Deep Generative Learning Approach for Two-stage Adaptive Robust Optimization
Speaker: Aron Brenner
Abstract: Two-stage adaptive robust optimization (ARO) is a powerful approach for planning under uncertainty, balancing first-stage decisions with recourse decisions made after uncertainty is realized. To account for uncertainty, modelers typically define a simple uncertainty set over which potential outcomes are considered. However, classical methods for defining these sets unintentionally capture a wide range of unrealistic outcomes, resulting in overly-conservative and costly planning in anticipation of unlikely contingencies. In this work, we introduce AGRO, a solution algorithm that performs adversarial generation for two-stage adaptive robust optimization using a variational autoencoder. AGRO generates high-dimensional contingencies that are simultaneously adversarial and realistic, improving the robustness of first-stage decisions at a lower planning cost than standard methods. To ensure generated contingencies lie in high-density regions of the uncertainty distribution, AGRO defines a tight uncertainty set as the image of “latent" uncertainty sets under the VAE decoding transformation. Projected gradient ascent is then used to maximize recourse costs over the latent uncertainty sets by leveraging differentiable optimization methods. We demonstrate the cost-efficiency of AGRO by applying it to both a synthetic production-distribution problem and a real-world power system expansion setting. We show that AGRO outperforms the standard column-and-constraint algorithm by up to 1.8% in production-distribution planning and up to 11.6% in power system expansion.

Speakers
ML

Michael Lau

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YZ

Yihang Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AB

Aron Brenner

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10G: Novel First-Order Methods via Performance Estimation II
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Novel First-Order Methods via Performance Estimation II
Chair: Alex Wang
Cluster: Conic and Semidefinite Optimization

Talk 1: Computer-assisted design of complexity lower bounds for accelerated composite optimization methods
Speaker: Uijeong Jang
Abstract: The accelerated composite optimization method FISTA (Beck, Teboulle 2009) is suboptimal by a constant factor, and we present a new method OptISTA that improves FISTA by a constant factor of 2. The performance estimation problem (PEP) has recently been introduced as a new computer-assisted paradigm for designing optimal first-order methods. In this work, we establish the exact optimality of OptISTA with a lower-bound construction that extends the semi-interpolated zero-chain construction (Drori, Taylor 2022) to the double-function setup of composite optimization. By establishing exact optimality, our work concludes the search for the fastest first-order methods, with respect to the performance measure of worst-case function value suboptimality, for the proximal, projected-gradient, and proximal-gradient setups involving a smooth convex function and a closed proper convex function.

Talk 2: Automated tight Lyapunov analysis for first-order methods
Speaker: Manu Upadhyaya
Abstract: We present a methodology for establishing the existence of quadratic Lyapunov inequalities for a wide range of first-order methods used to solve convex optimization problems. In particular, we consider (i) classes of optimization problems of finite-sum form with (possibly strongly) convex and possibly smooth functional components, (ii) first-order methods that can be written as a linear system on state-space form in feedback interconnection with the subdifferentials of the functional components of the objective function, and (iii) quadratic Lyapunov inequalities that can be used to draw convergence conclusions. We present a necessary and sufficient condition for the existence of a quadratic Lyapunov inequality within a predefined class of Lyapunov inequalities, which amounts to solving a small-sized semidefinite program. We showcase our methodology on several first-order methods that fit the framework. Most notably, our methodology allows us to significantly extend the region of parameter choices that allow for duality gap convergence in the Chambolle-Pock method.

Talk 3: Accelerating Proximal Gradient Descent via Silver Stepsizes
Speaker: Jinho Bok
Abstract: Surprisingly, recent work has shown that gradient descent can be accelerated without using momentum -- just by judiciously choosing stepsizes. An open question raised by several papers is whether this phenomenon of stepsize-based acceleration holds more generally for constrained and/or composite convex optimization via projected and/or proximal versions of gradient descent. We answer this in the affirmative by proving that the silver stepsize schedule yields analogously accelerated rates in these settings. These rates are conjectured to be asymptotically optimal among all stepsize schedules, and match the silver convergence rate of vanilla gradient descent (Altschuler and Parrilo, 2023), namely O(ε^{−log_ρ 2}) for smooth convex optimization and O(κ^{log_ρ 2}log(1/ε)) under strong convexity, where ε is the precision, κ is the condition number, and ρ=1+sqrt(2) is the silver ratio. The key technical insight is the combination of recursive gluing -- the technique underlying all analyses of gradient descent accelerated with time-varying stepsizes -- with a certain Laplacian-structured sum-of-squares certificate for the analysis of proximal point updates. Joint work with Jason M. Altschuler.

Speakers
AW

Alex Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
UJ

Uijeong Jang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Manu Upadhyaya

Manu Upadhyaya

PhD student, Lund University
JB

Jinho Bok

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10H: Advances in Bilevel Optimization: Algorithms and Applications
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Advances in Bilevel Optimization: Algorithms and Applications
Chair: Mingyi Hong & Prashant Khanduri
Cluster: Nonlinear Optimization

Talk 1: Barrier Function for Bilevel Optimization with Coupled Lower-Level Constraints: Formulation, Approximation and Algorithms
Speaker: Xiaotian Jiang
Abstract: In this paper, we consider bilevel optimization problem where the lower-level has coupled constraints, i.e. the constraints depend both on the upper- and lower-level variables. In particular, we consider two settings for the lower-level problem. The first is when the objective is strongly convex and the constraints are convex with respect to the lower-level variable; The second is when the lower-level is a linear program. We propose to utilize a barrier function reformulation to translate the problem into an unconstrained problem. By developing a series of new techniques, we proved that both the hyperfunction value and hypergradient of the barrier reformulated problem (uniformly) converge to those of the original problem under minimal assumptions. Further, to overcome the non-Lipschitz smoothness of hyperfunction and lower-level problems for barrier reformulated problems, we design an adaptive algorithm that ensures a non-asymptotic convergence guarantee. We also design an algorithm that converges to the stationary point of the original problem asymptotically under certain assumptions. The proposed algorithms require minimal assumptions, and to our knowledge, they are the first with convergence guarantees when the lower-level problem is a linear program.

Talk 2: A Discretization Approach for Low-Dimensional Non-Convex Bilevel Optimization
Speaker: Prashant Khanduri
Abstract: Bilevel optimization has become an invaluable tool in areas like machine learning and operations research. Most modern bilevel algorithms are developed for problems with strongly convex and/or unconstrained lower-level tasks. In this work, we deal with a class of problems where the lower-level task is non-convex and constrained. To solve this class of challenging problems, we propose a novel discretized value-function-based approach wherein the value function and the respective problem reformulation are discretized. Under the proposed approach, the value function’s optimization problem is transformed into an easier convex optimization task via discretization. We tackle the discretized problem using a penalty approach and establish the relation between the KKT points, and the local and global minima of the two problems. We develop a gradient descent-based algorithm to solve the penalty reformulated problem. We establish finite-time convergence guarantees for the developed algorithm. Finally, we perform numerical experiments to confirm the effectiveness of the proposed method.

Talk 3: An Accelerated Gradient Method for Convex Simple Bilevel Optimization
Speaker: Jincheng Cao
Abstract: In this paper, we focus on simple bilevel optimization problems, where we minimize a convex smooth objective function over the optimal solution set of another convex smooth constrained optimization problem. We present a novel bilevel optimization method that locally approximates the solution set of the lower-level problem using a cutting plane approach and employs an accelerated gradient-based update to reduce the upper-level objective function over the approximated solution set. We measure the performance of our method in terms of suboptimality and infeasibility errors and provide non-asymptotic convergence guarantees for both error criteria. Specifically, when the feasible set is compact, we show that our method requires at most $\mathcal{O}(\max\{1/\sqrt{\epsilon_{f}}, 1/\epsilon_g\})$ iterations to find a solution that is $\epsilon_f$-suboptimal and $\epsilon_g$-infeasible. Moreover, under the additional assumption that the lower-level objective satisfies the $r$-th Hölderian error bound, we show that our method achieves an iteration complexity of $\tilde{\mathcal{O}}(\max\{\epsilon_{f}^{-\frac{2r-1}{2r}},\epsilon_{g}^{-\frac{2r-1}{2r}}\})$, which matches the optimal complexity of single-level convex constrained optimization when $r=1$.

Speakers
XJ

Xiaotian Jiang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PK

Prashant Khanduri

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JC

Jincheng Cao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10I: Advances in Variational Inequalities and Saddle Point Problems
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Advances in Variational Inequalities and Saddle Point Problems
Chair: Afrooz Jalilzadeh
Cluster: Multi-agent Optimization and Games

Talk 1: Convergence Analysis of Stochastic Quasi-Variational Inequalities
Speaker: Afrooz Jalilzadeh
Abstract: While Variational Inequality (VI) is a well-established mathematical framework that subsumes Nash equilibrium and saddle-point problems, less is known about its extension, Quasi-Variational Inequalities (QVI). QVI allows for cases where the constraint set changes as the decision variable varies allowing for a more versatile setting. In this talk, we propose extra-gradient and gradient-based methods for solving Stochastic Quasi-Variational Inequalities (SQVI) and establish a rigorous convergence rate analysis for these methods. Our approach not only advances the theoretical understanding of SQVI but also demonstrates its practical applicability. Specifically, we highlight its effectiveness in reformulating and solving problems such as generalized Nash Equilibrium, bilevel optimization, and saddle-point problems with coupling constraints.

Talk 2: Linear Complementarity Systems for the Morning Commute Problem with Ridesharing and Dynamic Pricing
Speaker: Wei Gu
Abstract: The emerging ridesharing services and relevant infrastructures such as High-Occupancy Toll (HOT) lanes, provide more flexibility for travelers, more opportunities for sustainable transportation systems, and at the same time, more challenges for the classical morning commute problem. To capture traffic dynamics, we propose a modified bottleneck model that avoids time-delayed terms in computing travel times and maintains desired mathematical properties. Then we develop a general mathematical modeling framework for the morning commute problem with ridesharing, including travel modes, infrastructures, and operators. Formulated as Linear Complementary Systems (LCS), the proposed model simultaneously captures travelers’ departure time choice, lane choice between HOT lane and general purpose lane, as well as mode choice between ridesharing and solo driving. We show the solution existence for the LCS-based general modeling framework. To approximate the proposed continuous-time model, a discrete-time model is generated using an implicit time discretization scheme, with the theoretical guarantee to converge back to the original continuous-time form. Analytical solutions for dynamic prices, including drivers’ incomes, passengers’ payments, and HOT lane toll charges, are derived to balance the various demands of travelers, operators, and society. The proposed models and dynamic prices are validated in numerical examples. Results show that we simultaneously benefit travelers, operators, and society toward urban sustainability through ridesharing: smaller travel costs, positive net cash flow and toll collection for ridesharing and HOT lane operators, and better system performance.

Talk 3: Simultaneous Learning and Optimization for Misspecified Saddle Point Problems
Speaker: Erfan Yazdandoost Hamedani
Abstract: With recent technological advancements and data growth, there is increasing interest in optimization problems where key parameters are unknown or misspecified. A common approach, "estimate-then-optimize", involves learning the unknown parameter by optimizing a secondary objective function in a preliminary estimation stage. However, such methods lack asymptotic convergence guarantees, as the parameter is typically estimated within finite time, often resulting in suboptimal performance in the optimization phase. This highlights the need for methods that simultaneously handle learning and optimization. In this talk, we address a class of misspecified convex-concave saddle point (SP) problems, where the objective function contains an unknown vector of parameters, which can be learned through a parallel SP problem. We propose an accelerated primal-dual algorithm, analyzing its convergence guarantee and complexity bound based on the parameter estimate. Additionally, we demonstrate the algorithm's overall complexity under various assumptions on the secondary objective function.

Speakers
AJ

Afrooz Jalilzadeh

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WG

Wei Gu

PhD Candidate, University of Southern California
Generalized Traffic Equilibrium with Ride-hailing and Customer WaitingWe develop a generalized traffic equilibrium model that considers ride-hailing services provided by Transportation Network Companies (TNCs, e.g., Uber and Lyft) and accounts for customer waiting. The generalized... Read More →
EY

Erfan Yazdandoost Hamedani

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10J: Randomized algorithms beyond minimization
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Randomized algorithms beyond minimization
Chair: Laurent Condat
Cluster: Fixed Points and Variational Inequalities

Talk 1: Splitting the forward-backward algorithm
Speaker: Emanuele Naldi
Abstract: In this talk, we introduce an extension of the forward-backward splitting method, designed to address monotone inclusion problems involving sums of multiple maximal monotone operators, with some of them being cocoercive operators. Our approach builds upon recent developments in splitting methods and offers two significant innovations. First, we generalize the classical forward-backward and Davis-Yin algorithms by incorporating flexible, arbitrary network architectures, making our method well-suited for distributed optimization problems. Second, we relax the common assumption of uniform cocoercivity, allowing the use of larger, more adaptive step sizes based on individual cocoercivity constants, which enables us to accelerate convergence in some instances. We provide a comprehensive convergence rate analysis and demonstrate the practical benefits of this enhanced flexibility through numerical experiments. We investigate and test further the methods introducing also stochasticity in the proposed algorithm.

Talk 2: Solving Hidden Monotone Variational Inequalities with Surrogate Losses
Speaker: Gauthier Gidel
Abstract: Deep learning has proven to be effective in a wide variety of loss minimization problems. However, many applications of interest, like minimizing projected Bellman error and min-max optimization, cannot be modelled as minimizing a scalar loss function but instead correspond to solving a variational inequality (VI) problem. This difference in setting has caused many practical challenges as naive gradient-based approaches from supervised learning tend to diverge and cycle in the VI case. In this talk, I will introduce a principled surrogate-based approach compatible with deep learning to solve VIs. I will show that our surrogate-based approach has three main benefits: (1) under assumptions that are realistic in practice (when hidden monotone structure is present, interpolation, and sufficient optimization of the surrogates), it guarantees convergence, (2) it provides a unifying perspective of existing methods, and (3) is amenable to existing deep learning optimizers like ADAM. I will demonstrate that this surrogate-based approach is effective in min-max optimization and minimizing projected Bellman error. Furthermore, in the deep reinforcement learning case, we propose a novel variant of TD(0) which is more compute and sample efficient.

Talk 3: A Simple Finite-Time Analysis of Temporal Difference Learning with Linear Function Approximation
Speaker: Aritra Mitra
Abstract: We study the finite-time convergence of TD learning with linear function approximation under Markovian sampling. Existing proofs for this setting either assume a projection step in the algorithm to simplify the analysis, or require a fairly intricate argument to ensure stability of the iterates. We ask: Is it possible to retain the simplicity of a projection-based analysis without actually performing a projection step in the algorithm? Our main contribution is to show this is possible via a novel two-step argument. In the first step, we use induction to prove that under a standard choice of a constant step-size α, the iterates generated by TD learning remain uniformly bounded in expectation. In the second step, we establish a recursion that mimics the steady-state dynamics of TD learning up to a bounded perturbation on the order of O(α^2) that captures the effect of Markovian sampling. Combining these pieces leads to an overall approach that considerably simplifies existing proofs. We conjecture that our inductive proof technique will find applications in the analyses of more complex stochastic approximation algorithms, and provide some examples of such applications.

Speakers
avatar for Laurent Condat

Laurent Condat

Senior Research Scientist, King Abdullah University of Science and Technology (KAUST)
Laurent Condat received a PhD in applied mathematics in 2006 from Grenoble Institute of Technology, Grenoble, France. After a postdoc in the Helmholtz Zentrum Muenchen, Munich, Germany, he was hired in 2008 as a permanent researcher by the French National Center for Scientific Research... Read More →
EN

Emanuele Naldi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GG

Gauthier Gidel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AM

Aritra Mitra

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10K: Nonconvex methods and SDPs
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Nonconvex methods and SDPs
Chair: Florentin Goyens
Cluster: Nonlinear Optimization

Talk 1: Higher-Order Newton Methods with Polynomial Work per Iteration
Speaker: Amir Ali Ahmadi
Abstract: We present generalizations of Newton’s method that incorporate derivatives of an arbitrary order $d$ but maintain a polynomial dependence on dimension in their cost per iteration. At each step, our $d$th-order method uses semidefinite programming to construct and minimize a sum of squares-convex approximation to the $d$th-order Taylor expansion of the function we wish to minimize. We prove that our $d$th-order method has local convergence of order $d$. This results in lower oracle complexity compared to the classical Newton method. We show on numerical examples that basins of attraction around local minima can get larger as $d$ increases. Under additional assumptions, we present a modified algorithm, again with polynomial cost per iteration, which is globally convergent and has local convergence of order $d$. Co-authors: Abraar Chaudhry and Jeffrey Zhang.

Talk 2: Sharp Global Guarantees for Nonconvex Low-rank Recovery in the Noisy Overparameterized Regime
Speaker: Richard Zhang
Abstract: Rank overparameterization was recently shown to make nonconvex low-rank matrix recovery increasingly benign under the restricted isometry property (RIP), by converting all local minima into global minima that exactly recover the ground truth. But this does not fully explain the practical success of overparameterization, because real algorithms can still become trapped at nonstrict saddle points (approximate second-order points with arbitrarily small negative curvature) even when all local minima are global. Moreover, the result does not accommodate for noisy measurements, but it is unclear whether such an extension is even possible, in view of the many discontinuous and unintuitive behaviors already known for the overparameterized regime. In this paper, we introduce a novel proof technique that unies, simplifies, and strengthens two previously competing approachesone based on escape directions and the other based on the inexistence of counterexampleto provide sharp global guarantees in the noisy overparameterized regime. We show, once local minima have been converted into global minima through slight overparameterization, that near-second-order points achieve the same minimax-optimal recovery bounds (up to small constant factors) as significantly more expensive convex approaches. Our results are sharp with respect to the noise level and the solution accuracy, and hold for both the symmetric parameterization $XX^T$, as well as the asymmetric parameterization $UV^T$ under a balancing regularizer; we demonstrate that the balancing regularizer is indeed necessary.

Talk 3: HALLaR: A New Solver for Solving Huge SDPs
Speaker: Arnesh Sujanani
Abstract: This talk presents a new first-order method for solving large-scale semidefinite programs (SDPs) with bounded domain. It is an inexact augmented Lagrangian (AL) method where the AL subproblems are solved by a hybrid low-rank method. In contrast to the classical low-rank method, the new method finds a near-optimal solution (with provable complexity bounds) of SDP instances satisfying strong duality. Computational results comparing the new method to state-of-the-art solvers on several large SDP instances show that the former finds higher accurate solutions in substantially less CPU time than the latter ones. For example, in less than 20 minutes, our method can solve (on a personal laptop) a maximum stable set SDP with 1 million vertices and 10 million edges within 1e-5 relative accuracy. This is joint work with Renato D.C. Monteiro and Diego Cifuentes.

Speakers
avatar for Florentin Goyens

Florentin Goyens

Postdoc, UCLouvain
Researcher in numerical optimization at UCLouvain in Belgium
AA

Amir Ali Ahmadi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RZ

Richard Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AS

Arnesh Sujanani

Postdoctoral Fellow
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10L: Optimization and Statistics at Scale
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Optimization and Statistics at Scale
Chair: Mateo Diaz
Cluster: Optimization For Data Science

Talk 1: A superlinearly convergent subgradient method for sharp semismooth problems
Speaker: Vasilis Charisopoulos
Abstract: Nonsmooth optimization problems appear throughout machine learning and signal processing. However, standard first-order methods for nonsmooth optimization can be slow for "poorly conditioned" problems. In this talk, I will present a locally accelerated first-order method that is less sensitive to conditioning and achieves superlinear (i.e., double-exponential) convergence near solutions for a broad family of problems. The algorithm is inspired by Newton's method for solving nonlinear equations.

Talk 2: Commutator Stepsize Schedule
Speaker: Henry Shugart
Abstract: TBD

Talk 3: The radius of statistical efficiency
Speaker: Mateo Diaz
Abstract: Classical results in asymptotic statistics show that the Fisher information matrix controls the difficulty of estimating a statistical model from observed data. In this work, we introduce a companion measure of robustness of an estimation problem: the radius of statistical efficiency (RSE) is the size of the smallest perturbation to the problem data that renders the Fisher information matrix singular. We compute the RSE up to numerical constants for a variety of test bed problems, including principal component analysis, generalized linear models, phase retrieval, bilinear sensing, and matrix completion. In all cases, the RSE quantifies the compatibility between the covariance of the population data and the latent model parameter. Interestingly, we observe a precise reciprocal relationship between the RSE and the intrinsic complexity/sensitivity of the problem instance, paralleling the classical Eckart–Young theorem in numerical analysis.

Speakers
VC

Vasilis Charisopoulos

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HS

Henry Shugart

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MD

Mateo Diaz

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10M: Advances in min-max optimization algorithms for machine learning
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Advances in min-max optimization algorithms for machine learning
Chair: Ahmet Alacaoglu
Cluster: Optimization For Data Science

Talk 1: How to make the gradient descent-ascent converge to local minimax optima
Speaker: Donghwan Kim
Abstract: Can we effectively train a generative adversarial network (GAN) (or equivalently, optimize a minimax problem), similarly to how we successfully train a classification neural network (or equivalently, minimize a function) using gradient methods? Currently, the answer is 'No'. The remarkable success of gradient descent in minimization is supported by theoretical results; under mild conditions, gradient descent converges to a local minimizer, and almost surely avoids strict saddle points. However, comparable theoretical support for minimax optimization is currently lacking. This talk will discuss recent progress in addressing this gap using dynamical systems theory. Specifically, this talk will present new variants of gradient descent-ascent that, under mild conditions, converge to local minimax optima, where the existing gradient descent-ascent methods fail to do so.

Talk 2: Parameter-free second-order methods for min-max optimization
Speaker: Ali Kavis
Abstract: In this talk, I will talk about an adaptive, line-search-free second-order methods with optimal rate of convergence for solving convex-concave min-max problems. By means of an adaptive step size, our algorithms feature a simple update rule that requires solving only one linear system per iteration, eliminating the need for line-search or backtracking mechanisms. Specifically, we base our algorithms on the optimistic method and appropriately combine it with second-order information. Moreover, distinct from common adaptive schemes, we define the step size recursively as a function of the gradient norm and the prediction error in the optimistic update. We first analyze a variant where the step size requires knowledge of the Lipschitz constant of the Hessian. Under the additional assumption of Lipschitz continuous gradients, we further design a parameter-free version by tracking the Hessian Lipschitz constant locally and ensuring the iterates remain bounded. We also evaluate the practical performance of our algorithm by comparing it to existing second-order algorithms for minimax optimization. Inspired by the adaptive design of the step size, we propose a heuristic initialization rule that performs competitively across different problems and scenarios and eliminates the need to fine tune the step size.

Talk 3: TBD
Speaker: Jelena Diakonikolas
Abstract: TBD

Speakers
AA

Ahmet Alacaoglu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DK

Donghwan Kim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AK

Ali Kavis

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10N: Nonsmooth PDE Constrained Optimization: Algorithms, Analysis and Applications Part 3
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Nonsmooth PDE Constrained Optimization: Algorithms, Analysis and Applications Part 3
Chair: Harbir Antil
Cluster: PDE-constrained Optimization

Talk 1: An Adaptive Inexact Trust-Region Method for PDE-Constrained Optimization with Regularized Objectives
Speaker: Robert Baraldi
Abstract: We introduce an inexact trust-region method for efficiently solving regularized optimization problems governed by PDEs. In particular, we consider the class of problems in which the objective is the sum of a smooth, nonconvex function and nonsmooth, convex function. Such objectives are pervasive in the literature, with examples being basis pursuit, inverse problems, and topology optimization. The inclusion of nonsmooth regularizers and constraints is critical, as they often perserve physical properties or promote sparsity in the control. Enforcing these properties in an efficient manner is critical when met with computationally intense nature of solving PDEs. A common family of methods that can obtain accurate solutions with considerably smaller mesh sizes are adaptive finite element routines. They are critical in reducing error in solutions as well as mitigating numerical cost of solving the PDE. Our adaptive trust-region method solves the regularized objective while automatically refining the mesh for the PDE. Our method increases accuracy of the gradient and objective via local error estimators and our criticality measure. We present our numerical results on problems in control.

Talk 2: The SiMPL method for density-based topology optimization
Speaker: Dohyun Kim
Abstract: We introduce Sigmoidal mirror descent with a projected latent variable (SiMPL), a novel first-order optimization method for density-based topology optimization. SiMPL ensures point-wise bound preserving design updates and faster convergence than other popular first-order topology optimization methods. By leveraging the (negative) Fermi-Dirac entropy, we define a non-symmetric Bregman divergence that facilitates a simple yet effective update rule with the help of so-called latent variable. SiMPL produces a sequence of pointwise-feasible iterates even when high-order finite elements are used in the discretization. Numerical experiments demonstrates that the method outperforms other popular first-order optimization algorithms. We also present mesh- and order-independent convergence along with possible extension of this method.

Talk 3: Two-level Discretization Scheme for Total Variation in Integer Optimal Control
Speaker: Paul Manns
Abstract: We advance the discretization of the dual formulation of the total variation term with Raviart-Thomas functions which is known from literature for convex problems. Due to our integrality constraints, the previous analysis is not applicable anymore because, when considering a Γ-convergence-type argument, the recovery sequences generally need to attain non-integer, that is, infeasible, values. We overcome this problem by embedding a finer discretization of the input functions. A superlinear coupling of the mesh sizes implies an averaging on the coarser Raviart-Thomas mesh, which enables to recover the total variation of integer-valued limit functions with integer-valued, discretized input functions. In turn, we obtain a Γ-convergence-type result and convergence rates under additional regularity assumptions.

Speakers
HA

Harbir Antil

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RB

Robert Baraldi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DK

Dohyun Kim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Paul Manns

Paul Manns

TU Dortmund University
Bio:Paul completed his PhD at the Institute for Mathematical Optimization at Technical University of Braunschweig in 2019. Afterwards, he joined the Mathematics and Computer Science Division of Argonne National Laboratory as James H Wilkinson Fellow in Scientific Computing. In September 2021, Paul moved to TU Dortmund University as assistant professor.Paul's research focus lies on mixed-integer optimization infinite dimensions, in particular, appropriate regularization techniques and trust-region algor... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10O: Optimization on Manifolds
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Optimization on Manifolds
Chair: Maurício Silva Louzeiro
Cluster: Optimization on Manifolds

Talk 1: Variational Problems and Duality on Manifolds
Speaker: Anton Schiela
Abstract: Variational problems play a fundamental role in many areas of applied mathematics, and form the basis of both theoretical results and numerical algorithms. Problems of this class also arise in the context of manifolds and have important applications there. However, to formulate them in an adequate way, refined concepts of differential geometry, in particular a well developed duality theory, are required. In this talk we will give an introduction into these concepts and elaborate, how they can be applied to variational problems, using the example of harmonic mappings. We will also describe some implications of these concepts to the design of numerical solution algorithms.

Talk 2: Newton's method for nonlinear mappings into vector bundles
Speaker: Laura Weigl
Abstract: We consider Newton's method for finding zeros of nonlinear mappings from a manifold $\mathcal X$ into a vector bundle $\mathcal E$. In this setting a connection on $\mathcal E$ is required to render the Newton equation well defined, and a retraction on $\mathcal X$ is needed to compute a Newton update. As applications we will discuss the solution of variational problems involving mappings between manifolds, and, in particular, the numerical computation of geodesics under force fields.

Talk 3: An Adaptive Cubic Regularization quasi-Newton Method on Riemannian Manifolds
Speaker: Maurício Silva Louzeiro
Abstract: A quasi-Newton method with cubic regularization is designed for solving Riemannian unconstrained nonconvex optimization problems. The proposed algorithm is fully adaptive with at most ${\cal O} (\epsilon_g^{-3/2})$ iterations to achieve a gradient smaller than $\epsilon_g$ for given $\epsilon_g$, and at most $\mathcal O(\max\{ \epsilon_g^{-\frac{3}{2}}, \epsilon_H^{-3} \})$ iterations to reach a second-order stationary point respectively. Notably, the proposed algorithm remains applicable even in cases of the gradient and Hessian of the objective function unknown. Numerical experiments are performed with gradient and Hessian being approximated by forward finite-differences to illustrate the theoretical results and numerical comparison.

Speakers
AS

Anton Schiela

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LW

Laura Weigl

PhD Student, University of Bayreuth
MS

Maurício Silva Louzeiro

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10P: Recent Algorithmic Advances in Multiagent Optimization
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Recent Algorithmic Advances in Multiagent Optimization
Chair: Prashant Khanduri & Haibo Yang
Cluster: Multi-agent Optimization and Games

Talk 1: Theory on Mixture-of-Experts in Continual Learning
Speaker: Sen Lin
Abstract: Continual learning (CL) has garnered significant attention because of its ability to adapt to new tasks that arrive over time. Catastrophic forgetting (of old tasks) has been identified as a major issue in CL, as the model adapts to new tasks. The Mixture-of-Experts (MoE) model has recently been shown to effectively mitigate catastrophic forgetting in CL, by employing a gating network to sparsify and distribute diverse tasks among multiple experts. However, there is a lack of theoretical analysis of MoE and its impact on the learning performance in CL. This paper provides the first theoretical results to characterize the impact of MoE in CL via the lens of overparameterized linear regression tasks. We establish the benefit of MoE over a single expert by proving that the MoE model can diversify its experts to specialize in different tasks, while its router learns to select the right expert for each task and balance the loads across all experts. Our study further suggests an intriguing fact that the MoE in CL needs to terminate the update of the gating network after sufficient training rounds to attain system convergence, which is not needed in the existing MoE studies that do not consider the continual task arrival. Furthermore, we provide explicit expressions for the expected forgetting and overall generalization error to characterize the benefit of MoE in the learning performance in CL. Interestingly, adding more experts requires additional rounds before convergence, which may not enhance the learning performance. Finally, we conduct experiments on both synthetic and real datasets to extend these insights from linear models to deep neural networks (DNNs), which also shed light on the practical algorithm design for MoE in CL.

Talk 2: Multimodal Federated Learning and Communication Compression
Speaker: Aritra Dutta
Abstract: Multi-modal transformers mark significant progress in different domains, but siloed high-quality data hinders their further improvement. To remedy this, federated learning (FL) has emerged as a promising privacy-preserving paradigm for training models without direct access to the raw data held by different clients. Despite its potential, a considerable research direction regarding the unpaired unimodal clients and the transformer architecture in FL remains unexplored. To fill this gap, first, we explore a transfer multi-modal federated learning (MFL) scenario within the vision-language domain, where clients possess data of various modalities distributed across different datasets. We systematically evaluate the performance of existing methods when a transformer architecture is utilized and introduce a novel framework called Federated modality complementary and collaboration (FedCola) by addressing the in-modality and cross-modality gaps among clients. To remedy the communication bottleneck of FedCola during the training, when we used lossy compression schemes, we realized that, unlike unimodal FL, one compression operator throughout the training shows extremely degraded performance. The second part of this talk is dedicated to designing an adaptive compressor scheduler for multimodal FL training. 

Talk 3: Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization
Speaker: Haibo Yang
Abstract: Federated Learning (FL) offers a promising framework for collaborative and privacy-preserving machine learning across distributed data sources. However, the substantial communication costs associated with FL significantly challenge its efficiency. Specifically, in each communication round, the communication costs scale linearly with the model's dimension, which presents a formidable obstacle, especially in large model scenarios. Despite various communication-efficient strategies, the intrinsic dimension-dependent communication cost remains a major bottleneck for current FL implementations. This paper proposes a novel dimension-free communication algorithm – DeComFL, which leverages the zeroth-order optimization techniques and reduces the communication cost from $\mathcal{O}(d)$ to $\mathcal{O}(1)$ by transmitting only a constant number of scalar values between clients and the server in each round, regardless of the dimension $d$ of the model parameters. Theoretically, in non-convex functions, we prove that our algorithm achieves state-of-the-art rates, which show a linear speedup of the number of clients and local steps under standard assumptions. With additional low effective rank assumption, we can further show the convergence rate is independent of the model dimension $d$ as well. Empirical evaluations, encompassing both classic deep learning training and large language model fine-tuning, demonstrate significant reductions in communication overhead. Notably, DeComFL achieves this by transmitting only around 1MB of data in total between the server and a client to fine-tune a model with billions of parameters.

Speakers
SL

Sen Lin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AD

Aritra Dutta

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Haibo Yang

Haibo Yang

Assistant Professor, Rochester Institute of Technology
Name: Dr. Haibo YangTitle: Assistant Professor of Distributed Optimization & Federated LearningAffiliation: Rochester Institute of TechnologyBio:Dr. Haibo Yang builds intelligent systems that know when to communicate, when to compute, and when to just quietly wait for the next client... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10Q: Advanced Computational Solvers 1
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Advanced Computational Solvers 1
Chair: Nai-Yuan Chiang
Cluster: Computational Software

Talk 1: Simultaneous approaches for estimation and optimization of hybrid dynamic systems
Speaker: Carl Laird
Abstract: Hybrid modeling is becoming increasingly popular in science and engineering applications where it benefits from first-principles domain knowledge and increasing availability of data to train and validate machine learning components. Notable recent methodologies include the use of neural networks within DAE (differential algebraic equation) systems, namely Neural ODEs and Universal Differential Equations (UDE), which are powerful tools that enable effective hybrid modeling of systems with complex dynamics. In this presentation, we demonstrate the integration of machine learning models with simultaneous approaches for estimation and optimization of hybrid dynamic systems. We investigate both algorithmic aspects for improving computational performance and integration with modeling languages.

Talk 2: Recent Progress in the Cardinal Optimizer
Speaker: Nils-Christian Kempke
Abstract: The Cardinal Optimizer (COPT) is a mathematical optimization solver for large-scale optimization problems. It includes high-performance solvers for LP, MIP and conic programming etc. In this talk, we present the recent advances in COPT, including our work on GPU-accelerated optimization. We highlight some key computational and theoretical improvements we have recently integrated and evaluate their impact on the performance of our solver. We provide an overview of the performance numbers of the latest COPT release for all supported problem classes.

Talk 3: Latest Developments and Improvements in the MindOpt Interior Point Optimizer
Speaker: Kuoling Huang
Abstract: We focus on practical improvements in the MindOpt interior point optimizer, with a particular emphasis on enhancements in the sparse solver. We discuss various algorithmic techniques, such as advanced Cholesky factorization and reordering strategies, that enhance numerical stability and efficiency. Key numerical results on large-scale optimization problems will be presented.

Speakers
CL

Carl Laird

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
NK

Nils-Christian Kempke

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KH

Kuoling Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10R: Quantum Methods for Optimization and Sampling
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Quantum Methods for Optimization and Sampling
Chair: Jiaqi Leng
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Optimizing random local Hamiltonians by dissipation
Speaker: Alexander Dalzell
Abstract: Markov chain Monte Carlo (MCMC) is a versatile category of numerical methods that has been fruitfully applied to many instances of classical constraint optimization problems. Recent work has developed a quantum analogue of MCMC—quantum Gibbs sampling algorithms—which can be applied to quantum optimization problems, that is, the task of preparing a low-energy quantum state of a local Hamiltonian. However, since quantum computers do not yet exist, these quantum algorithms cannot yet be tested empirically. In this work, we provide analytical guarantees for a simplified version of the Gibbs sampling algorithm on a wide class of random quantum optimization problems. Specifically, we study the problem of preparing a quantum state that optimizes a local Hamiltonian on a system of either quantum spins or fermions consisting of random all-to-all, k-local interactions. We prove that the simplified Gibbs sampling algorithm achieves a Ω(1/k)-fraction approximation of the optimum energy, giving an exponential improvement on the k-dependence over the prior best (both classical and quantum) algorithmic guarantees. Combined with existing circuit lower bound for such states, our results suggest that finding low-energy states for sparsified (quasi)local spin and fermionic models is quantumly easy but classically nontrivial. This further indicates that quantum Gibbs sampling may be a suitable metaheuristic for optimization problems. This is based on joint work with Joao Basso and Chi-Fang Chen (https://arxiv.org/pdf/2411.02578)

Talk 2: Quantum Langevin Dynamics for Optimization
Speaker: Zherui Chen
Abstract: We initiate the study of utilizing Quantum Langevin Dynamics (QLD) to solve optimization problems, particularly those non-convex objective functions that present substantial obstacles for traditional gradient descent algorithms. Specifically, we examine the dynamics of a system coupled with an infinite heat bath. This interaction induces both random quantum noise and a deterministic damping effect to the system, which nudge the system towards a steady state that hovers near the global minimum of objective functions. We theoretically prove the convergence of QLD in convex landscapes, demonstrating that the average energy of the system can approach zero in the low temperature limit with an exponential decay rate correlated with the evolution time. Numerically, we first show the energy dissipation capability of QLD by retracing its origins to spontaneous emission. Furthermore, we conduct detailed discussion of the impact of each parameter. Finally, based on the observations when comparing QLD with classical Fokker-Plank-Smoluchowski equation, we propose a time-dependent QLD by making temperature and ℏ time-dependent parameters, which can be theoretically proven to converge better than the time-independent case and also outperforms a series of state-of-the-art quantum and classical optimization algorithms in many non-convex landscapes.

Talk 3: Quantum Acceleration of Gibbs Sampling for Continuous Potentials
Speaker: Jiaqi Leng
Abstract: Realizing a random variable corresponding to a Gibbs distribution σ ∝ e(−βV(x)) with a continuous potential V(x) is a prominent task in the computer science and engineering. However, sampling from a high-dimensional Gibbs distribution is in general intractable due to the exponential blowup of computational complexity, a challenge known as the curse of dimensionality. While quantum computers can efficiently process high-dimensional data, existing quantum algorithms for Gibbs sampling exhibit sub-optimal dimension dependence and potentially slow convergence rate, leaving substantial room for quantum advantage for the classical sampling task. In this paper, by reformulating the invariant measure of certain diffusion processes as ground state of quantum Hamiltonians, we propose a novel approach to accelerate Gibbs sampling on quantum computers. In this paper, by reformulating the invariant measure of certain diffusion processes as the ground state of quantum Hamiltonians, we propose a novel approach to accelerating Gibbs sampling on quantum computers using the standard quantum numerical linear algebra toolbox. We apply our framework to two commonly used diffusion processes in the literature: Langevin Diffusion (LD) and Replica Exchange Langevin Diffusion (RELD). By quantizing LD, we obtain a quantum algorithm that matches the best-known classical methods. While RELD has been widely used in practice, its rigorous analysis remains challenging in the classical literature. Our new method bypasses the technical difficulties in the classical convergence analysis of RELD, yielding a quantum algorithm that outperforms all known results in terms of dimension dependence and convergence rate.

Speakers
AD

Alexander Dalzell

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZC

Zherui Chen

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jiaqi Leng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10S: Advances in Modeling and Optimization for MDP and Optimal Control
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Advances in Modeling and Optimization for MDP and Optimal Control
Chair: Yan Li & Minda Zhao
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Beyond absolute continuity: a new class of dynamic risk measures
Speaker: Jincheng Yang
Abstract: The modern theory of risk measures copes with uncertainty by considering multiple probability measures. While it is often assumed that a reference probability measure exists, under which all relevant probability measures are absolutely continuous, there are examples where this assumption does not hold, such as certain distributional robust functionals. In this talk, we introduce a novel class of dynamic risk measures that do not rely on this assumption. We will discuss its convexity, coherence, and time consistency properties.

Talk 2: TBD
Speaker: Yashaswini Murthy
Abstract: TBD

Talk 3: Landscape of Policy Optimization for Finite Horizon MDPs with General State and Action
Speaker: Minda Zhao
Abstract: Policy gradient methods are widely used in reinforcement learning. Yet, the nonconvexity of policy optimization imposes significant challenges in understanding the global convergence of policy gradient methods. For a class of finite-horizon Markov Decision Processes (MDPs) with general state and action spaces, we develop a framework that provides a set of easily verifiable assumptions to ensure the Kurdyka-Łojasiewicz (KŁ) condition of the policy optimization. Leveraging the KŁ condition, policy gradient methods converge to the globally optimal policy with a non-asymptomatic rate despite nonconvexity. Our results find applications in various control and operations models, including entropy-regularized tabular MDPs, Linear Quadratic Regulator (LQR) problems, stochastic inventory models, and stochastic cash balance problems, for which we show an $\epsilon$-optimal policy can be obtained using a sample size in $\tilde{\co}(\epsilon^{-1})$ and polynomial in terms of the planning horizon by stochastic policy gradient methods. Our result establishes the first sample complexity for multi-period inventory systems with Markov-modulated demands and stochastic cash balance problems in the literature.

Speakers
JY

Jincheng Yang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YM

Yashaswini Murthy

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MZ

Minda Zhao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10T: Optimization on Manifolds
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Optimization on Manifolds
Chair: Christopher Criscitiello
Cluster: Optimization on Manifolds

Talk 1: Estimation of barycenters in convex domains of metric spaces
Speaker: Victor-Emmanuel Brunel
Abstract: In this talk, we study the statistical and algorithmic aspects of barycenter estimation in convex domains of metric spaces. For data that lie in a non-linear metric space, such as a sphere, barycenters are the natural extension of means, and their estimation is a fundamental statistical problem. Here, we assume that the space satisfies a curvature upper bound in Alexandrov’s sense: Then, barycenters are minimizers of geodesically convex functions, yielding good statistical and algorithmic guarantees.

Talk 2: Non-Euclidean Motion Planning with Graphs of Geodesically-Convex Sets
Speaker: Thomas Cohn
Abstract: Mathematical optimization is a central tool for planning efficient, collision-free trajectories for robots. However, such trajectory optimization methods may get stuck in local minima due to inherent nonconvexities in the optimization landscape. The use of mixed-integer programming to encapsulate these nonconvexities and find globally optimal trajectories has recently shown great promise. One such tool is the Graph of Convex Sets framework, which yields tight convex relaxations and efficient approximation strategies that greatly reduce runtimes. These approaches were previously limited to Euclidean configuration spaces, but many robotic configuration spaces of interest are best represented as smooth manifolds. We introduce Graphs of Geodesically-Convex Sets, the natural generalization of GCS to Riemannian manifolds. We leverage a notion of "local" geodesic convexity to avoid the limitations that would otherwise arise from compact manifolds and closed geodesics, which often arise in robot configuration spaces. We demonstrate that this optimization framework encompasses the motion planning problems of interest, and examine when it can and cannot be solved efficiently. In the case of zero-curvature, we are able to reduce the problem to a mixed-integer convex program that can be solved efficiently. On the other hand, positive curvature makes the Riemannian distance function not even locally convex. In the general case, we describe a piecewise-linear approximation strategy to obtain approximate solutions to shortest path problems on manifolds of arbitrary curvature. We also specifically compare different parametrizations of the Lie groups SO(3) and SE(3). We present experiments demonstrating our methodology on various robot platforms, including producing efficient collision-free trajectories for a PR2 bimanual mobile manipulator.

Talk 3: Leveraging Manifold Structure for Fast Algorithms
Speaker: Brighton Ancelin
Abstract: In today's data-driven world, many problems require the processing of high-dimensional data. Methods like Principal Component Analysis (PCA) may help reduce the dimensionality of such problems, but may be ineffective or inefficient when data lies on non-linear manifolds. Some common manifolds even exhibit a natural Euclidean embedding dimension that far exceeds their intrinsic dimension, highlighting the need for more sophisticated approaches. In this talk, we will explore techniques for effectively handling data on such manifolds by operating on this lower intrinsic dimension. Such techniques can lead to much faster and more memory-efficient algorithms. We will first examine a specific problem on Grassmannian manifolds, then generalize to the broader class of quotient manifolds. Attendees may expect to gain a better understanding of how to view their structured data, and how to more efficiently process it.

Speakers
CC

Christopher Criscitiello

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
VB

Victor-Emmanuel Brunel

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Thomas Cohn

Thomas Cohn

PhD Candidate, Massachusetts Institute of Technology
I'm a PhD candidate at MIT, working with Professor Russ Tedrake in the Robot Locomotion Group. My research focus is robot motion planning, primarily using tools from optimization and differential geometry.
BA

Brighton Ancelin

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10U: Approximating derivatives in derivative-free optimization
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Approximating derivatives in derivative-free optimization
Chair: Geovani Grapiglia
Cluster: Derivative-free Optimization

Talk 1: TRFD: A derivative-free trust-region method based on finite differences for composite nonsmooth optimization
Speaker: Geovani Grapiglia
Abstract: In this work we present TRFD, a derivative-free trust-region method based on finite differences for minimizing composite functions of the form $f(x)=h(F(x))$, where $F$ is a black-box function assumed to have a Lipschitz continuous Jacobian, and $h$ is a known convex Lipschitz function, possibly nonsmooth. The method approximates the Jacobian of $F$ via forward finite differences. We establish an upper bound for the number of evaluations of $F$ that TRFD requires to find an $\epsilon$-approximate stationary point. For L1 and Minimax problems, we show that our complexity bound reduces to $\mathcal{O}(n\epsilon^{-2})$ for specific instances of TRFD, where $n$ is the number of variables of the problem. Assuming that $h$ is monotone and that the components of $F$ are convex, we also establish a worst-case complexity bound, which reduces to $\mathcal{O}(n\epsilon^{-1})$ for Minimax problems. Numerical results are provided to illustrate the relative efficiency of TRFD in comparison with existing derivative-free solvers for composite nonsmooth optimization.

Talk 2: Sparse gradients in derivative-free optimization
Speaker: Daniel McKenzie
Abstract: In this talk I’ll survey some of my recent work in extending DFO to the high-dimensional regime, primarily by exploiting sparsity in gradients to construct good gradient approximations cheaply.

Talk 3: How sampling distribution affect gradient-free optimization
Speaker: Liyuan Cao
Abstract: Gradient-free optimization is often used to optimize functions where only function values are accessible. It primarily refers to gradient descent methods where the "gradients" are directional derivatives estimated via finite differences using randomly sampled points. The sampling distribution is often chosen as Gaussian, uniform on sphere, Haar, or uniform along the coordinate directions. This choice has been shown to affect the algorithms' numerical efficiency. It is presented in this talk a theoretical analysis of the algorithms' performance under different sample distribution. The result provides a guidance on how to choose the distribution.

Speakers
GG

Geovani Grapiglia

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DM

Daniel McKenzie

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
LC

Liyuan Cao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10V: Modeling Oriented Software and Methods (2)
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Modeling Oriented Software and Methods (2)
Chair: Jean-Paul Watson
Cluster: Computational Software

Talk 1: SNoGloDE: Structural Nonlinear Global Decomposition for Parallel Solution of large-Scale NLPs in Python
Speaker: Georgia Stinchfield
Abstract: TBD

Talk 2: Automated Generation and Modeling of Piecewise-linear Functions in Pyomo
Speaker: Bashar Ammari
Abstract: TBD

Talk 3: TBD
Speaker: TBD TBD
Abstract: TBD

Speakers
JW

Jean-Paul Watson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
GS

Georgia Stinchfield

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BA

Bashar Ammari

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TT

TBD TBD

TBD, TBD
TBD
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10W: Parallel and Distributed Optimization
Thursday July 24, 2025 10:30am - 11:45am PDT
Session: Parallel and Distributed Optimization
Chair: Michel Lahmann
Cluster: nan

Talk 1: Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
Speaker: Artavazd Maranjyan
Abstract: Asynchronous Stochastic Gradient Descent (Asynchronous SGD) is a cornerstone method for parallelizing learning in distributed machine learning. However, its performance suffers under arbitrarily heterogeneous computation times across workers, leading to suboptimal time complexity and inefficiency as the number of workers scales. While several Asynchronous SGD variants have been proposed, recent findings by Tyurin & Richtárik (NeurIPS 2023) reveal that none achieve optimal time complexity, leaving a significant gap in the literature. In this paper, we propose Ringmaster ASGD, a novel Asynchronous SGD method designed to address these limitations and tame the inherent challenges of Asynchronous SGD. We establish, through rigorous theoretical analysis, that Ringmaster ASGD achieves optimal time complexity under arbitrarily heterogeneous and dynamically fluctuating worker computation times. This makes it the first Asynchronous SGD method to meet the theoretical lower bounds for time complexity in such scenarios.

Talk 2: GPU-Based Complete and Rigorous Search for Continuous Global Optimization and Constraint Satisfaction
Speaker: Guanglu Zhang
Abstract: Continuous optimization is of significance in many science and engineering fields. In many practical applications, it is desirable, and sometimes indispensable, to find the global optimum when solving continuous nonconvex optimization problems. However, popular gradient-based methods (e.g., interior point methods) and heuristic methods (e.g., genetic algorithms) often become trapped in local optima and do not take rounding errors into account, leading to inconsistent or incorrect assumptions and applications. Several methods using interval analysis have been introduced to enclose the guaranteed global optimum for continuous nonconvex optimization, but these interval methods are significantly more computationally expensive than gradient-based methods and heuristic methods and therefore are not commonly adopted in practical applications. Importantly, the unprecedented computational power brought by modern Graphics Processing Units (GPUs) offers new opportunities to overcome the computational bottleneck in global optimization using interval analysis. However, existing interval methods cannot utilize the computational power of modern GPUs, because they are designed for CPU-based sequential computing while modern GPUs are designed for massively parallel computing with unique compute and memory architecture. Based on the research gap, we introduce a novel GPU-based, complete, and rigorous method that can efficiently enclose the guaranteed global optimum based on user-specified tolerances for continuous nonconvex optimization problems, especially for large-scale continuous nonconvex optimization problems. Using interval analysis, coupled with the computational power and architecture of GPU, the method iteratively rules out the suboptimal regions in the search domain where the global optimum cannot exist and leaves a finite set of regions where the global optimum must exist. The method is validated by enclosing the global optimum of 10 multimodal benchmark test functions with scalable dimensions, including the Levy function, Ackley function, and Griewank function. These test functions represent grand challenges of global optimization and enclosing the guaranteed global optimum of these test functions with more than 80 variables has not been reported in the literature. Our method successfully encloses the global optimum for each of these ten test functions with up to 10,000 variables using a server with one GPU in reasonable computation time, demonstrating a linear increase in total number of iterations and approximately quadratic increase in computation time with the number of variables in these optimization problems. The method is also applied to solve several practical engineering optimization problems effectively and efficiently. Corresponding Publication Zhang, G., Feng, W., & Cagan, J. (2024). A GPU-Based Parallel Region Classification Method for Continuous Constraint Satisfaction Problems. Journal of Mechanical Design, 146(4), 041705.

Talk 3: A parallelizable ADMM approach for block-structured quadratic programs
Speaker: Michel Lahmann
Abstract: Block-structured quadratic programs (QPs) arise frequently in the context of optimal control problems. The goal of our study is to provide a fast solution to these QPs which is crucial for the successful application of optimal control algorithms to many real-world problems. Existing ADMM-based QP solvers usually split the problem into an equality-constrained QP and a projection onto a box. In general, this is a proven and widely used method. Nevertheless, splitting the problem in this way does not allow the block structure of OPs arising from optimal control problems to be optimally utilized. With our new splitting of the problem we are able to utilize this block structure and solve parts of the problem in parallel. We will introduce the resulting method and show numerical results.

Speakers
AM

Artavazd Maranjyan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Guanglu Zhang

Guanglu Zhang

Research Scientist, Carnegie Mellon University
Name: Dr. Guanglu ZhangTitle: GPU-Based Complete and Rigorous Search for Continuous Global Optimization and Constraint SatisfactionBio: Dr. Zhang is a research scientist at Carnegie Mellon University. His research in mathematical optimization employs interval analysis, coupled with... Read More →
ML

Michel Lahmann

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10X
Thursday July 24, 2025 10:30am - 11:45am PDT
Thursday July 24, 2025 10:30am - 11:45am PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

10:30am PDT

Parallel Sessions 10Y
Thursday July 24, 2025 10:30am - 11:45am PDT
Thursday July 24, 2025 10:30am - 11:45am PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

11:45am PDT

Lunch 4 (provided)
Thursday July 24, 2025 11:45am - 1:15pm PDT
American BBQ Buffet
Thursday July 24, 2025 11:45am - 1:15pm PDT
USC Founder's / Hutton Park 3551 Trousdale Pkwy, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11A: First order methods in machine learning
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: First order methods in machine learning
Chair: Yaoliang Yu
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Independently-Normalized SGD for Generalized-Smooth Nonconvex Optimization
Speaker: Yufeng Yang
Abstract: Recent studies have shown that many nonconvex machine learning problems meet a so-called generalized-smooth condition that extends beyond traditional smooth nonconvex optimization. However, the existing algorithms designed for generalized-smooth nonconvex optimization encounter significant limitations in both their design and convergence analysis. In this work, we first study deterministic generalized-smooth nonconvex optimization and analyze the convergence of normalized gradient descent under the generalized Polyak-Lojasiewicz condition. Our results provide a comprehensive understanding of the interplay between gradient normalization and function geometry. Then, for stochastic generalized-smooth nonconvex optimization, we propose an independently-normalized stochastic gradient descent algorithm, which leverages independent sampling, gradient normalization, and clipping to achieve the standard sample complexity under relaxed assumptions. Experiments demonstrate the fast convergence of our algorithm.

Talk 2: Why does the Adam algorithm work so well for training large language models?
Speaker: Mark Schmidt
Abstract: The success of the Adam optimizer on a wide array of architectures has made it the default in settings where stochastic gradient descent (SGD) performs poorly. However, it is unclear why the gap between Adam and SGD is often big for large language models (LLMs) but small for computer vision benchmarks. Recent work proposed that Adam works better for LLMs due to heavy-tailed noise in the stochastic gradients. We show evidence that the noise is not a major factor in the performance gap between SGD and Adam. Instead, we show that a key factor is the class imbalance found in language tasks. In particular, the large number of low-frequency classes causes SGD to converge slowly but has a smaller effect on Adam and sign descent. We show that a gap between SGD and Adam can be induced by adding a large number of low-frequency classes to computer vision models or even to linear models. We further prove in a simple setting that gradient descent converges slowly while sign descent does not.

Talk 3: On Games with Conflicting Interests
Speaker: Baoxiang Wang
Abstract: To investigate differentiable games (that have strategy spaces in $\reals^n$), we decompose the game into components where the dynamic is well understood. We show that any differentiable game can be decomposed as a direct sum of three parts: the first decomposition as an exact potential part, a near vector potential part, and a non-strategic part; the second as a near potential part, an exact vector potential part, and a non-strategic part. A potential part coincides with potential games described by Monderer and Shapley (1996), known as pure cooperative games. A vector potential game on the other hand represents a game with purely conflicting interests. We show that the individual gradient field is divergence-free, in which case the gradient descent dynamic may either be divergent or recurrent. When the divergence-free game is finite, including harmonic games and important classes of zero-sum games, we show that optimistic variants of classical no-regret learning algorithms converge to an $\epsilon$-approximate Nash equilibrium at a rate of $O(1/\epsilon^2).

Speakers
YY

Yaoliang Yu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MS

Mark Schmidt

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Baoxiang Wang

Baoxiang Wang

Assistant professor, The Chinese University of Hong Kong, Shenzhen
Name: Baoxiang WangTitle: Assistant professorAffiliation: The Chinese University of Hong Kong, ShenzhenBio:Baoxiang Wang is an assistant professor at School of Data Science, The Chinese University of Hong Kong, Shenzhen. Baoxiang works on reinforcement learning and learning theory... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11B: Learning, Robustness, and Fairness
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Learning, Robustness, and Fairness
Chair: Cagil Kocyigit
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Fairness in Federated Learning
Speaker: Daniel Kuhn
Abstract: We study group fairness regularizers in federated learning with the aim to find a globally fair model in a distributed fashion. Distributed training poses unique challenges because the fairness regularizers based on probability metrics popular in centralized training cannot be decomposed across clients. To circumvent this challenge, we propose a function-tracking scheme for the global fairness regularizer based on a Maximum Mean Discrepancy (MMD), which incurs a small communication overhead. The proposed function-tracking scheme can readily be incorporated into most federated learning algorithms while maintaining rigorous convergence guarantees, as we will exemplify in the context of FedAvg. When enforcing differential privacy, the kernel-based MMD regularization allows for easy analysis via a change of kernel, leveraging an intuitive interpretation of kernel convolution. Numerical experiments validate our theoretical findings.

Talk 2: Robust Offline Policy Learning Under Covariate Shifts
Speaker: Phebe Vayanos
Abstract: We study the problem of distribution shifts in offline policy learning, where the policy training distribution is different from the deployment distribution and may lead to harmful or sub-optimal policy actions at deployment. In real world applications, changes to an allocation system can cause shifts in measured covariates, such as wording changes in survey questions that elicit different responses from individuals experiencing homelessness. As a result, a non-robust allocation policy may incorrectly over or under allocate resources based on the original offline data distribution. Adopting a Wasserstein distributionally robust approach, we learn an allocation policy that is not restricted to any functional form and robust to potential covariate shifts in the population of allocatees.

Talk 3: Learning Robust Risk Scores under Unobserved Confounders
Speaker: Cagil Kocyigit
Abstract: We study the problem of learning robust risk scores from observational data in the presence of unobserved confounders. In the absence of unobserved confounders, a well-known approach to adjust for confounding is inverse probability weighting (IPW) of the data. However, in the presence of unobserved confounders, estimating these weights is challenging, even in large data regimes. We formulate a robust maximum likelihood problem with the objective of maximizing the worst-case likelihood in view of all possible weights within an uncertainty set, which we construct by drawing inspiration from sensitivity analysis in observational data settings. We formulate this problem as a convex optimization problem by leveraging duality techniques rooted in robust optimization. Numerical experiments show that our robust estimates outperform both IPW and direct estimation methods on synthetic data designed to reflect realistic scenarios.

Speakers
DK

Daniel Kuhn

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PV

Phebe Vayanos

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CK

Cagil Kocyigit

Assistant Professor, University of Luxembourg
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11C: Recent Advances in Stochastic First-Order Methods
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Recent Advances in Stochastic First-Order Methods
Chair: Tianjiao Li
Cluster: Optimization Under Uncertainty and Data-driven Optimization

Talk 1: Auto-conditioned gradient methods for nonconvex stochastic optimization
Speaker: Guanghui Lan
Abstract: We present a novel class of projected gradient methods, termed auto-conditioned projected gradient (AC-PG) method, for minimizing a smooth but not necessarily convex function over a compact convex set. AC-PG does not require the input of the Lipschitz constant of the objective function or any line search procedure, but can still achieve the best-known iteration complexity associated with the projected gradient method for finding an approximate stationary point of the problem. The key idea is to estimate the Lipschitz constant using first-order information gathered from the previous iterations, and to show that the error caused by underestimating the Lipschitz constant can be properly controlled. We then generalize AC-PG for stochastic nonconvex optimization and derive the well-known sample complexity for stochastic projected gradient descent for a class of nonconvex stochastic optimization problem.

Talk 2: A Novel Catalyst Scheme for Stochastic Minimax Optimization
Speaker: Yan Li
Abstract: We present a proximal-point-based catalyst scheme for simple first-order methods applied to convex minimization and convex-concave minimax problems. In particular, for smooth and (strongly)-convex minimization problems, the proposed catalyst scheme, instantiated with a simple variant of stochastic gradient method, attains the optimal rate of convergence in terms of both deterministic and stochastic errors. For smooth and strongly-convex-strongly-concave minimax problems, the catalyst scheme attains the optimal rate of convergence for deterministic and stochastic errors up to a logarithmic factor. To the best of our knowledge, this reported convergence seems to be attained for the first time by stochastic first-order methods in the literature. We obtain this result by designing and catalyzing a novel variant of stochastic extragradient method for solving smooth and strongly-monotone variational inequality, which may be of independent interest.

Talk 3: Bridging constrained stochastic optimization and statistical learning
Speaker: Wenlong Mou
Abstract: First-order stochastic optimization algorithms and complexity-based statistical risk bounds form the foundation of modern machine learning theory. When seeking optimal parameters within a constrained set, the achievable statistical risk depends on the complexity of the constraint set rather than on the ambient dimension. However, existing first-order stochastic optimization methods often fail to adapt to this complexity, making it challenging to attain instance-dependent optimal risk. In this talk, I will discuss recent advances in constrained stochastic optimization. We examine the complexity of stochastic gradient methods and their variance-reduced counterparts, exploring how these algorithms respond to the geometric structures of constraint sets. I will also present new instance-dependent optimality guarantees for a class of variance-reduced algorithms, highlighting how these results provide concrete guidance on the choice of parameters.

Speakers
avatar for Tianjiao Li

Tianjiao Li

PhD student, Georgia Institute of Technology
Name: Tianjiao LiTitle: PhD studentAffiliation: Georgia Institute of TechnologyBio:Tianjiao Li is a fifth-year PhD candidate in the School of Industrial and Systems Engineering at Georgia Institute of Technology, advised by Prof. George Lan and Prof. Ashwin Pananjady. His research... Read More →
GL

Guanghui Lan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Yan Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
WM

Wenlong Mou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11D: Nonmonontone inclusions and nonconvex optimization
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Nonmonontone inclusions and nonconvex optimization
Chair: Dimitri Papadimitriou
Cluster: Nonlinear Optimization

Talk 1: Proximal iterations for solving nonmonotone inclusions and applications
Speaker: Dimitri Papadimitriou
Abstract: In a real Hilbert space H, the monotone inclusion problem aims at finding a zero of a set-valued maximally monotone operator A. The term "warped proximal iteration" was recently introduced as generalization of the proximal point iterative algorithm for finding a zero point of a maximally monotone operator A acting on the space H. Nevertheless, the maximal monotonicity of the operator A restricts the applicability of this algorithm to the class of convex optimization problems as well as operator splitting methods for composite monotone inclusions. The solving of general nonmonotone inclusion, i.e., the inclusion where the operator A is nonmonotone, remains an open and challenging research problem. For this purpose, the notion of r-(co)hypomonotonicity has been introduced to guarantee the convergence of the generated sequence. From this perspective, the first objective of this paper is to extend the definition of r-hypomotonicity. The second is to determine the convergence property of the warped proximal iteration as well as its various applications for the solving of (constrained) nonconvex optimization problems. In particular, we place our attention to the finding of Karush Kuhn Tucker (KKT) points for a class of nonconvex quadratic programming (QP) problems with equality constraints. Numerical results illustrate the performance of our method against state-of-the-art QP solvers.

Talk 2: Qualitative Analysis and Boosted DCA for Generalized Multi-Source Weber Problems
Speaker: Tuyen Tran
Abstract: In this talk, we first investigate fundamental qualitative properties of the generalized multi-source Weber problem formulated using the Minkowski gauge function. Then, we apply Nesterov's smoothing and the adaptive Boosted Difference of Convex functions Algorithm (BDCA) to solve both the unconstrained and constrained versions of the generalized multi-source Weber problems. These algorithms are tested in Matlab with real and artificial data sets. We conduct a comprehensive evaluation of the new algorithms and provide insights into its efficiency.

Talk 3: A Primal-Dual Proximal Outer Approximation algorithm
Speaker: Vu Bang
Abstract: In this talk we propose a Proximal Outer Approximation (PrOA) algorithm for solving Mixed Integer Quadratically Constrained Quadratic Programs (MI-QCQP). The proposed sequential method works as follows: starting from the integer-feasible point obtained by solving the continuous relaxation of the original problem, at each iteration, it first solves by means of a primal-dual proximal algorithm an NLP problem to obtain a constraint-feasible pair that is (also) used to compute an upper-bound on the optimal value, it then obtains a new integer-feasible pair by solving an MIQP through iteratively improving first-order approximations of the nonlinear constraints and computes a lower bound using this new pair. The integer-feasible point obtained is then used to solve the NLP problem of the next iteration. Convergence and computational properties are analyzed and compared together with numerical executions on classical QCQP's such as the QKP, the congestion-constrained FLP, and the generalized multi-source Weber problem.

Speakers
DP

Dimitri Papadimitriou

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TT

Tuyen Tran

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
VB

Vu Bang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11E: Quantum Computing and discrete continuous optimization
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Quantum Computing and discrete continuous optimization
Chair: David Bernal Neira
Cluster: Interplay Between Continuous and Discrete Optimization

Talk 1: Interior-point and first-order methods for quantum entanglement detection
Speaker: Javier Pena
Abstract: We describe two different algorithmic approaches to tackle the following central problem in quantum information science: given a bipartite quantum state, detect if the state is entangled by constructing a suitable "entanglement witness". The crux for both approaches is to solve a challenging convex feasibility problem based on a widely known symmetric extension criteria for entanglement detection. We show that a judicious formulation of the convex feasibility problem can be naturally tackled via both interior-point methods and Frank-Wolfe method. We will discuss the tradeoffs of these two computational approaches.

Talk 2: Prospects for quantum speedups in convex integer programming
Speaker: Brandon Augustino
Abstract: Integer programming constitutes a fundamental class of problems in computer science with countless real-world applications, and their NP-hardness makes them an attractive candidate for large quantum speedups. The most well-known quantum approaches to solve discrete optimization problems either rely on quantum speedups for unstructured search, or rely on the use of penalty functions to incorporate constraints. Unsurprisingly, these techniques do not lead to quantum algorithms that offer speedups over the state-of-the-art classical IP algorithms, which search the feasible region in a highly structured manner. In this talk we discuss the challenges and potential associated with designing faster quantum algorithms for the convex integer programming problem. From a technical standpoint, we discuss new quantum approaches to enumerating lattice points in convex bodies, which is at the core of the quantum IP algorithm we analyze.

Talk 3: Continuous optimization for unconventional continuous computing: Accelerating Continuous Variable Coherent Ising Machines with momentum updates
Speaker: David Bernal Neira
Abstract: The Coherent Ising Machine (CIM) leverages continuous dynamics and physical annealing principles for solving complex Ising problems. Our work introduces modifications to the Continuous Variable CIM (CV-CIM) by integrating momentum and Adam-based updates, targeting accelerated convergence for continuous optimization. Standard CV-CIMs operate with gradient-based updates, which, although effective, can be trapped by local minima or suffer from poor problem conditioning. By enhancing the CIM dynamics with momentum and adaptive techniques, our approach significantly improves convergence rates and solution diversity. Through numerical experiments on benchmark quadratic programming problems, we demonstrate that momentum-CV-CIM and Adam-CV-CIM variations outperform standard CV-CIM in convergence speed and robustness, especially under challenging, poorly conditioned instances. These findings suggest that incorporating classical optimization techniques like momentum and Adam into CV-CIM systems not only enhances computational efficiency but also extends the practical applicability of CIMs for a broader class of continuous optimization problems. This work underscores the potential for hybrid approaches that marry classical optimization strategies with non-conventional computing architectures, promoting more reliable and scalable solutions across diverse fields in optimization. The results are available in https://arxiv.org/abs/2401.12135

Speakers
JP

Javier Pena

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BA

Brandon Augustino

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DB

David Bernal Neira

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11F: SDPs and their applications across mathematics
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: SDPs and their applications across mathematics
Chair: Christoph Spiegel
Cluster: Conic and Semidefinite Optimization

Talk 1: Semidefinite programming bounds for packing in geometrical graphs and hypergraphs
Speaker: Fernando Mario de Oliveira Filho
Abstract: Packing problems in geometry, like the sphere-packing problem, can be modeled as independent-set problems in infinite graphs. The Lovász theta number, a semidefinite programming upper bound for the independence number of a graph, can often be extended to such infinite graphs, providing some of the best upper bounds for several geometrical parameters. Perhaps the most famous such application is Viazovska's solution of the sphere-packing problem in dimension 8. In this talk I will discuss the problem of packing regular polygons on the surface of the 2-dimensional sphere and how the Lovász theta number can be used to give upper bounds. I will also discuss the problem of finding large almost-equiangular sets, which is a hypergraph extension of packing.

Talk 2: TBD
Speaker: Pablo A. Parrilo
Abstract: TBD

Talk 3: Shattering triples with permutations
Speaker: Jan Volec
Abstract: Given a 6-tuple of permutations of [n], we say that a triple of points T of [n] is totally shattered if each of the six possible relative orderings of T appears in exactly one of the permutations. We set F(n) to be the maximum fraction of triples that can be totally shattered by a 6-tuple of permutations of [n]. In 2023, Johnson and Wickes showed that the limit of F(n) is at least 17/42 and at most 11/14, and they asked to determine the limit. In this talk, we use the flag algebra method to prove that the limit of F(n) = 10/21. This is a joint work with Daniela Opocenska.

Speakers
FM

Fernando Mario de Oliveira Filho

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
PA

Pablo A. Parrilo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JV

Jan Volec

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11G: Bilevel Programming and applications
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Bilevel Programming and applications
Chair: Luce Brotcorne
Cluster: Optimization Applications (Communication, Energy, Health, ML, ...)

Talk 1: Optimal Electric Vehicle Charging with Dynamic Pricing, Customer Preferences and Power Peak Reduction
Speaker: Luce Brotcorne
Abstract: We consider a provider of electric vehicle charging stations that operates a network of charging stations and use time varying pricing to maximize profit and reduce the impact on the electric grid. We propose a bilevel model with a single leader and multiple disjoint followers. The provider (leader) sets the price of charging for each station at each time slot, and ensures there is enough energy to charge. The charging choice of each customer is represented by a combination of a preference list of (station, time) pairs and a reserve price. The proposed model takes thus into accounts for the heterogeneity of customers with respect to price sensitivity. We define a single-level reformulation based on a reformulation approach from the literature on product line optimization, and we report computational results that highlight the efficiency of the new reformulation and the potential impact for reducing peaks on the electricity grid.

Talk 2: A Benders decomposition algorithm for thMinimum Spanning Tree Interdiction Problem..
Speaker: Frederic Semet
Abstract: In this presentation, we consider the Minimum Spanning Tree Interdiction (MSTI) problem. This problem is a two-player game between a network operator and an interdictor. The former aims to determine a Minimum Spanning Tree (MST) in a network. Constrained by a budget, the latter seeks to change the network topology to increase the weight of a MST. Two types of interdiction are considered: total and partial interdiction. A total interdicted edge is considered absent, while the weight of a partial interdicted edge is augmented by a predefined amount. The interdictor's budget is modeled as a knapsack constraint. In the first part of the presentation a mathematical formulation is introduced and valid inequalities are proposed to strengthen the model. Since a commercial solver can only solve instances of limited size, we describe a Benders decomposition algorithm in the second part. A MSTI formulation is decomposed into a Master Problem (MP) and a subproblem. A family constraints for the MP, called optimality constraints, are introduced and lifted. A family of valid inequalities is also proposed to tighten the linear relaxation of the MP. The subproblem, parameterized by the solution of the MP, consists only in the computation of a MST. Computational results show that instances with up to 200 nodes can be solved to optimality.

Talk 3: ..
Speaker: Pablo Escalona
Abstract: ..

Speakers
LB

Luce Brotcorne

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
FS

Frederic Semet

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11H: Distributed Learning for Machine Learning
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Distributed Learning for Machine Learning
Chair: Hoi To Wai
Cluster: Multi-agent Optimization and Games

Talk 1: Why batch normalization damage federated learning on non-iid data?
Speaker: Yanmeng Wang
Abstract: As a promising distributed learning paradigm, federated learning (FL) involves training deep neural network (DNN) models at the network edge while protecting the privacy of the edge clients. To train a large-scale DNN model, batch normalization (BN) has been regarded as a simple and effective means to accelerate the training and improve the generalization capability. However, recent findings indicate that BN can significantly impair the performance of FL in the presence of non-i.i.d. data. While several FL algorithms have been proposed to address this issue, their performance still falls significantly when compared to the centralized scheme. Furthermore, none of them have provided a theoretical explanation of how the BN damages the FL convergence. In this work, we present the first convergence analysis to show that under the non-i.i.d. data, the mismatch between the local and global statistical parameters in BN causes the gradient deviation between the local and global models, which, as a result, slows down and biases the FL convergence. In view of this, we develop a new FL algorithm that is tailored to BN, called FedTAN, which is capable of achieving robust FL performance under a variety of data distributions via iterative layer-wise parameter aggregation. Comprehensive experimental results demonstrate the superiority of the proposed FedTAN over existing baselines for training BN-based DNN models.

Talk 2: Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning
Speaker: Thinh Doan
Abstract: Two-time-scale optimization is a framework that abstracts a range of policy evaluation and policy optimization problems in reinforcement learning (RL). Akin to bi-level optimization under a particular type of stochastic oracle, the two-time-scale optimization framework has an upper level objective whose gradient evaluation depends on the solution of a lower level problem, which is to find the root of a strongly monotone operator. In this work, we propose a new method for solving two-time-scale optimization that achieves significantly faster convergence than the prior arts. The key idea of our approach is to leverage an averaging step to improve the estimates of the operators in both lower and upper levels before using them to update the decision variables. These additional averaging steps eliminate the direct coupling between the main variables, enabling the accelerated performance of our algorithm. We characterize the finite-time convergence rates of the proposed algorithm under various conditions of the underlying objective function, including strong convexity, convexity, Polyak-Lojasiewicz condition, and general non-convexity. These rates significantly improve over the best-known complexity of the standard two-time-scale stochastic approximation algorithm. When applied to RL, we show how the proposed algorithm specializes to novel online sample-based methods that surpass or match the performance of the existing state of the art. Finally, we support our theoretical results with numerical simulations in RL.

Talk 3: Two-timescale Stochastic Primal-dual Algorithms for Decentralized Optimization with Communication Compression
Speaker: Hoi-To Wai
Abstract: This talk presents a two-timescale compressed primal-dual (TiCoPD) algorithm for decentralized optimization. The algorithm is built upon the primal-dual optimization framework developed by a majorization-minimization procedure. The latter naturally suggests the agents to share a compressed difference term during the iteration. Furthermore, the TiCoPD algorithm incorporates a fast timescale mirror sequence for agent consensus on nonlinearly compressed terms with noise, together with a slow timescale primal-dual recursion for optimizing the objective function. We show that the TiCoPD algorithm converges with a constant step size. It also finds an O(1/sqrt{T}) stationary solution after T iterations. Numerical experiments on decentralized training of a neural network validate the efficacy of TiCoPD algorithm.

Speakers
TD

Thinh Doan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HW

Hoi-To Wai

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11I: Privacy Preserving Collaborative Learning
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Privacy Preserving Collaborative Learning
Chair: Sai Praneeth Karimireddy
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)

Talk 1: Efficient Distributed Optimization under Heavy-Tailed Noise
Speaker: Tian Li
Abstract: In distributed learning, to mitigate communication overhead, local updates are often applied before global aggregation, resulting in a nested optimization approach with inner and outer steps. However, heavy-tailed stochastic gradient noise remains a significant challenge, particularly in attention-based models, hindering effective training. In this work, we propose TailOPT, an efficient framework designed to address heavy-tailed noise by leveraging adaptive optimization and novel clipping techniques. We establish convergence guarantees for the TailOPT framework under heavy-tailed noise with potentially unbounded gradient variance and local updates. Among its variants, we propose a memory- and communication-efficient instantiation (named Bi^2Clip) that performs coordinate-wise clipping from both above and below at both the inner and outer optimizers. Bi^2Clip brings about benefits of adaptive optimization (e.g., Adam) without the cost of maintaining or transmitting additional gradient statistics. Empirically, TailOPT, including Bi^2Clip, demonstrates superior performance on several language tasks and models compared with state-of-the-art methods.

Talk 2: Parameter-Efficient Federated Learning Algorithms for Large-Scale Models
Speaker: Kibaek Kim
Abstract: Federated learning (FL) is a new collaborative training paradigm for training large-scale models across distributed data sources without sharing raw data. However, applying FL to large models presents significant challenges in terms of communication efficiency and resource utilization. In this work, we introduce novel parameter-efficient algorithms tailored to FL with large models. Our approach optimizes model updates by reducing the number of parameters communicated across clients. We will present preliminary results based on the new algorithms.

Talk 3: Data-Centric ML need Statistics and Game Theory
Speaker: Sai Praneeth Karimireddy
Abstract: Data is the most important factor determining the quality of an AI system. Data centric ML is an emerging research direction which constructs metrics to quantify usefulness of data (data valuation / data attribution). However, we show that existing methods do not properly take into account the randomness inherent in ML training. We also show that they are not fit to be used as compensation in data markets since they may not be incentive-compatible.

Speakers
TL

Tian Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KK

Kibaek Kim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SP

Sai Praneeth Karimireddy

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11J: Advances in first-order methods for large-scale optimization
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Advances in first-order methods for large-scale optimization
Chair: Jiawei Zhang
Cluster: Nonlinear Optimization

Talk 1: Acceleration by Random Stepsizes
Speaker: Jason Altschuler
Abstract: We show that for separable convex optimization, random stepsizes fully accelerate Gradient Descent (GD). Specifically, using iid stepsizes from the Arcsine distribution improves the iteration complexity from O(k) to O(\sqrt{k}), where k is the condition number. No momentum or other modifications to GD are required. This result is incomparable to the (deterministic) Silver Stepsize Schedule, which does not require separability but only achieves partial acceleration O(k^{\log_{1+\sqrt{2}} 2}) \approx O(k^{0.78}). Our starting point is a conceptual connection to potential theory: the variational characterization for the distribution of stepsizes with fastest convergence rate mirrors the variational characterization for the distribution of charged particles with minimal logarithmic potential energy. The Arcsine distribution solves both variational characterizations due to a remarkable "equalizing property'' which in the physical context amounts to a constant potential over space, and in the optimization context amounts to an identical convergence rate for all quadratic functions. A key technical insight is that martingale arguments extend this phenomenon to all separable convex functions.

Talk 2: Last Iterate Convergence of Incremental Methods and Applications in Continual Learning
Speaker: Xufeng Cai
Abstract: Incremental gradient and incremental proximal methods are a fundamental class of optimization algorithms used for solving finite sum problems, broadly studied in the literature. Yet, without strong convexity, their convergence guarantees have primarily been established for the ergodic (average) iterate. Motivated by applications in continual learning, we obtain the first convergence guarantees for the last iterate of both incremental gradient and incremental proximal methods, in general convex smooth (for both) and convex Lipschitz (for the proximal variants) settings. Our oracle complexity bounds for the last iterate nearly match (i.e., match up to a square-root-log or a log factor) the best known oracle complexity bounds for the average iterate, for both classes of methods. We further obtain generalizations of our results to weighted averaging of the iterates with increasing weights and for randomly permuted ordering of updates. We study incremental proximal methods as a model of continual learning with generalization and argue that large amount of regularization is crucial to preventing catastrophic forgetting. Our results generalize last iterate guarantees for incremental methods compared to state of the art, as such results were previously known only for overparameterized linear models, which correspond to convex quadratic problems with infinitely many solutions.

Talk 3: Unveiling Spurious Stationarity and Hardness Results for Bregman Proximal-Type Algorithms
Speaker: Jiajin Li
Abstract: Bregman proximal-type algorithms, such as mirror descent, are popular in optimization and data science for effectively exploiting problem structures and optimizing them under tailored geometries. However, most of existing convergence results rely on the gradient Lipschitz continuity of the kernel, which unfortunately excludes most commonly used cases, such as the Shannon entropy. In this paper, we reveal a fundamental limitation of these methods: Spurious stationary points inevitably arise when the kernel is not gradient Lipschitz. The existence of these spurious stationary points leads to an algorithm-dependent hardness result: Bregman proximal-type algorithms cannot escape from a spurious stationary point within any finite number of iterations when initialized from that point, even in convex settings. This limitation is discovered through the lack of a well-defined stationarity measure based on Bregman divergence for non-gradient Lipschitz kernels. Although some extensions attempt to address this issue, we demonstrate that they still fail to reliably distinguish between stationary and non-stationary points for such kernels. Our findings underscore the need for new theoretical tools and algorithms in Bregman geometry, paving the way for further research.

Speakers
JZ

Jiawei Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JA

Jason Altschuler

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
XC

Xufeng Cai

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Jiajin Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11K: Nonsmooth PDE Constrained Optimization: Algorithms, Analysis and Applications Part 2
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Nonsmooth PDE Constrained Optimization: Algorithms, Analysis and Applications Part 2
Chair: Robert Baraldi
Cluster: PDE-constrained Optimization

Talk 1: An Inexact Trust-Region Algorithm for Nonsmooth Risk-Averse Optimization
Speaker: Drew Kouri
Abstract: Many practical problems require the optimization of systems (e.g., differential equations) with uncertain inputs such as noisy problem data, unknown operating conditions, and unverifiable modeling assumptions. In this talk, we formulate these problems as infinite-dimensional, risk-averse stochastic programs for which we minimize a quantification of risk associated with the system performance. For many popular risk measures, the resulting risk-averse objective function is not differentiable, significantly complicating the numerical solution of the optimization problem. Unfortunately, traditional methods for nonsmooth optimization converge slowly (e.g., sublinearly) and consequently are often intractable for problems in which the objective function and any derivative information is expensive to evaluate. To address this challenge, we introduce a novel trust-region algorithm for solving large-scale nonsmooth risk-averse optimization problems. This algorithm is motivated by the primal-dual risk minimization algorithm and employs smooth approximate risk measures at each iteration. In addition, this algorithm permits and rigorously controls inexact objective function value and derivative (when available) computations, enabling the use of inexpensive approximations such as adaptive discretizations. We discuss convergence of the algorithm under mild assumptions and demonstrate its efficiency on various examples from PDE-constrained optimization.

Talk 2: Variational problems with gradient constraints: A priori and a posteriori error identities
Speaker: Rohit Khandelwal
Abstract: Nonsmooth variational problems are ubiquitous in science and engineering, for e.g., fracture modeling and contact mechanics. This talk presents a generic primal-dual framework to tackle these types of nonsmooth problems. Special attention is given to variational problems with gradient constraints. The key challenge here is how to project onto the constraint set both at the continuous and discrete levels. In fact, both a priori and a posteriori error analysis for such nonsmooth problems has remained open. In this talk, on the basis of a (Fenchel) duality theory at the continuous level, an a posteriori error identity for arbitrary conforming approximations of primal-dual formulations is derived. In addition, on the basis of a (Fenchel) duality theory at the discrete level, an a priori error identity for primal (Crouzeix–Raviart) and dual (Raviart–Thomas) formulations is established. The talk concludes by deriving the optimal a priori error decay rates.

Talk 3: Optimal Insulation: Numerical Analysis and Spacecraft
Speaker: Keegan Kirk
Abstract: Given a fixed amount of insulating material, how should one coat a heat-conducting body to optimize its insulating properties? A rigorous asymptotic analysis reveals this problem can be cast as a convex variational problem with a non-smooth boundary term. As this boundary term is difficult to treat numerically, we consider an equivalent (Fenchel) dual variational formulation more amenable to discretization. We propose a numerical scheme to solve this dual formulation on the basis of a discrete duality theory inherited by the Raviart-Thomas and Crouzeix-Raviart finite elements and show that the solution of the original primal problem can be reconstructed locally from the discrete dual solution. We discuss the a posteriori and a priori error analysis of our scheme, derive a posteriori estimators based on convex optimality conditions, and present numerical examples to verify theory. As an application, we consider the design of an optimally insulated spacecraft heat shield.

Speakers
RB

Robert Baraldi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DK

Drew Kouri

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RK

Rohit Khandelwal

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KK

Keegan Kirk

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11L: Recent Advances in Shape and Topology Optimization
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Recent Advances in Shape and Topology Optimization
Chair: Volker Schulz
Cluster: PDE-constrained Optimization

Talk 1: Approaches to Boundary-Effect-Dominated Topology Optimization
Speaker: Eddie Wadbro
Abstract: In classical design optimization using the material distribution method (density-based topology optimization), a material indicator function represents the presence or absence of material within the domain. To use this approach for boundary-effect-dominated problems, it is necessary to identify the boundary of the design at each iteration; this talk explores two methods to achieve this. The first involves using a boundary strip indicator function defined on the elements of the computational mesh. The second involves using a boundary indicator function defined on the mesh faces (edges in 2D and facets in 3D). The first method is applied to model a coated structure in a minimum heat compliance problem. The second method optimizes a heat sink, modeled by the Poisson equation with a Newtonian cooling boundary condition. The talk will cover the main ideas behind both approaches and showcase results from both model problems.

Talk 2: Topology optimization of the first truly wave focusing sonic black hole
Speaker: Martin Berggren
Abstract: We apply density-based topology optimization to design an acoustic waveguide possessing broadband wave focusing properties. Although effective as absorbers, previous suggestions of such devices — labeled sonic or acoustic black holes — have been unable to create the focusing effect. The optimization objective is a broadband maximization of the dissipated energy in a small absorbing area towards the end of the waveguide. A challenge with this application is that it is necessary to accurately model viscothermal boundary-layer losses associated with material boundaries. Here we rely on recent progress in the use of density-based topology optimization approaches for boundary-effect-dominated problems. Using such a procedure, we have been able to successfully design what seems to be the first truly broadband wave-focusing sonic black hole.

Talk 3: Shape Optimization Using the Total Generalized Variation of the Normal as Prior
Speaker: Stephan Schmidt
Abstract: The talk discusses how to use a total variation semi-norm on shapes as a prior. The idea is to concentrate curvature changes to edges, which is of great help when non-smooth shapes are to be reconstructed in inverse problems. Unsurprisingly, classical total variation keeps all typical downsides such as stair-casing when applied to manifolds. To this end, the concept of total generalized variation (TGV) by Bredies/Kunisch/Pock is extended to shapes. To implement TGV for shapes, the required separation of the linear component of a geometric variables necessitates non-standard finite elements on the tangent space of manifolds. The methodology is exemplified with applications stemming from geo-electric impedance tomography and mesh inpainting as well as texture denoising.

Speakers
EW

Eddie Wadbro

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Martin Berggren

Martin Berggren

Professor, Umeå University
Name: Martin BerggrenTitle: Professor of Scientific ComputingAffiliation: Department of Computing Science, Umeå UniversityBio:My main area of interest is Computational Design Optimization, in which numerical optimization algorithms are employed in the engineering design process... Read More →
SS

Stephan Schmidt

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11M: Derivative-free optimization for large scale problems
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Derivative-free optimization for large scale problems
Chair: Zaikun Zhang
Cluster: Derivative-free Optimization

Talk 1: Scalable derivative-free optimization algorithms with low-dimensional subspace techniques
Speaker: Zaikun Zhang
Abstract: We re-introduce a derivative-free subspace optimization framework originating from Chapter 5 the thesis [Z. Zhang, On Derivative-Free Optimization Methods, PhD thesis, Chinese Academy of Sciences, Beijing, 2012] of the author under the supervision of Ya-xiang Yuan. At each iteration, the framework defines a (low-dimensional) subspace based on an approximate gradient, and then solves a subproblem in this subspace to generate a new iterate. We sketch the global convergence and worst-case complexity analysis of the framework, elaborate on its implementation, and present some numerical results on solving problems with dimension as high as 10,000. The same framework was presented during ICCOPT 2013 in Lisbon under the title "A Derivative-Free Optimization Algorithm with Low-Dimensional Subspace Techniques for Large-Scale Problems", although it remains nearly unknown to the community until very recently. An algorithms following this framework named NEWUOAs was implemented by Zhang in MATLAB in 2011 (https://github.com/newuoas/newuoas), ported to Module-3 by Nystroem (Intel) in 2017, and included in cm3 in 2019 (https://github.com/modula3/cm3/blob/master/caltech-other/newuoa/src/NewUOAs.m3).

Talk 2: High-dimensional DFO: Stochastic subspace descent and improvements
Speaker: Stephen Becker
Abstract: We describe and analyze a family of algorithms which we call "stochastic subspace descent" which use projections of the gradient onto random subspaces, in a slightly similar spirit to well-known work by Nesterov and Spokoiny. We explain the benefits of subspace projection compared to Gaussian directional derivatives. We present a complete convergence analysis, and show that the method is well suited for high-dimensional problems. We also focus on our very recent work for cheap and automatic stepsize selection, as well as some preliminary results on biased sampling which requires leaving the "projected" paradigm.

Talk 3: Blockwise direct search methods
Speaker: Haitian Li
Abstract: Direct search methods are one class of derivative-free optimization algorithms that evaluate the objective function only based on the comparison of function values. We introduce a new framework of direct search method called blockwise direct search (BDS), which divides the searching set into blocks. For every iteration, each block has its step size. We develop this framework into a solver, which is open-source and easy to use. In numerical experiments, we observe that BDS can also be compared with model based methods in some cases. In addition, our solver shows its efficiency and stability under noise without introducing specific techniques. BDS can also be used to tune the hyperparameters itself to improve the performance.

Speakers
ZZ

Zaikun Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HL

Haitian LI

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11N: Riemannian geometry, optimization, and applications
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Riemannian geometry, optimization, and applications
Chair: Wen Huang
Cluster: Optimization on Manifolds

Talk 1: A Riemannian Proximal Newton-CG Method
Speaker: Wen Huang
Abstract: The proximal gradient method and its variants have been generalized to Riemannian manifolds for solving optimization problems in the form of $f + g$, where $f$ is continuously differentiable and $g$ may be nonsmooth. However, most of them do not have local superlinear convergence. Recently, a Riemannian proximal Newton method has been developed for optimizing problems in this form with $\mathcal{M}$ being a compact embedded submanifold and $g(x)= \lambda \|x\|_1$. Although this method converges superlinearly locally, global convergence is not guaranteed. The existing remedy relies on a hybrid approach: running a Riemannian proximal gradient method until the iterate is sufficiently accurate and switching to the Riemannian proximal Newton method. This existing approach is sensitive to the switching parameter. In this talk, we propose a Riemannian proximal Newton-CG method that merges the truncated conjugate gradient method with the Riemannian proximal Newton method. The global convergence and local superlinear convergence are proven. Numerical experiments show that the proposed method outperforms other state-of-the-art methods.

Talk 2: Manifold Identification and Second-Order Algorithms for l1-Regularization on the Stiefel Manifold
Speaker: Shixiang Chen
Abstract: In this talk, we will discuss manifold identification for the l1-regularization problem on the Stiefel manifold. First, we will demonstrate that the intersection of the identified manifold with the Stiefel manifold forms a submanifold. Building on this, we will propose a novel second-order retraction-based algorithm specifically designed for the intersected submanifold. Numerical experiments confirm that the new algorithm exhibits superlinear convergence.

Talk 3: A Riemannian Accelerated Proximal Gradient Method
Speaker: Shuailing Feng
Abstract: Riemannian accelerated gradient methods have been widely studied for smooth problem, but whether accelerated proximal gradient methods for nonsmooth composite problem on Riemannian manifolds can achieve theoretically acceleration remains unclear. Moreover, existing Riemannian accelerated gradient methods address geodesically convex and geodesically strongly convex cases separately. In this work, we introduce a unified Riemannian accelerated proximal gradient method with a rigorous convergence rate analysis for optimization problem of the form $F(x) = f(x) + h(x)$ on manifolds, where $f$ is either geodesically convex or geodesically strongly convex, and $h$ is weakly retraction-convex. Our analysis shows that the proposed method achieves acceleration under appropriate conditions. Additionally, we introduce a safeguard mechanism to ensure the convergence of the Riemannian accelerated proximal gradient method in non-convex settings. Numerical experiments demonstrate the effectiveness and theoretical acceleration of our algorithms.

Speakers
WH

Wen Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SC

Shixiang Chen

Assistant Professor, University of Science and Technology of China
I work on nonconvex optimizaiton.
SF

Shuailing Feng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11O: Riemannian geometry, optimization, and applications
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Riemannian geometry, optimization, and applications
Chair: Wen Huang
Cluster: Optimization on Manifolds

Talk 1: Strong Convexity of Sets in Riemannian Manifolds
Speaker: Damien Scieur
Abstract: Convex curvature properties are important in designing and analyzing convex optimization algorithms in the Hilbertian or Riemannian settings. In the case of the Hilbertian setting, strongly convex sets are well studied. Herein, we propose various definitions of strong convexity for uniquely geodesic sets in a Riemannian manifold. We study their relationship, propose tools to determine the geodesic strongly convex nature of sets, and analyze the convergence of optimization algorithms over those sets. In particular, we demonstrate that the Riemannian Frank-Wolfe algorithm enjoys a global linear convergence rate when the Riemannian scaling inequalities hold

Talk 2: Velocity-Based Karcher Mean on the Special Orthogonal Group: A Generalized Karcher Mean for Riemannian Manifolds
Speaker: Zhifeng Deng
Abstract: Special orthogonal group SOn consists of orthogonal matrices with the determinant 1 that realize rotations on n variables. It arises in many important applications, especially from the computer vision background where the rotations of a coordinates are used to described spacial motions of an object. Averaging a given data set is one of the most important computational tasks and the Karcher mean formulation in a metric space with the associated algorithms are naturally adapted for the averaging task on SOn as the Riemannian structure equips SOn with a metric space. Unfortunately, the distance induced by the Riemannian metric is not differentiable at cut loci which makes the Karcher mean formulation on SOn, and in general manifold, a non-convex optimization problem. In this presentation, we investigate the consequences of the non-convex optimization, with the SO2 circle as an example, and address the key issue in the discontinuous Riemannian logarithm that retrieves velocities in the tangent space from the points in the manifold. We propose a generalized Karcher mean formulation for Riemannian manifolds, namely the velocity-based Karcher mean, to overcome the discontinuity issue of retrieving velocities. This generalized Karcher mean subsumes the original Karcher mean and includes extra points as velocity-based Karcher mean if they emanate (non-minimal) geodesics to that data set with velocities summed up to zero. Then, we develop the algorithmic primitives, including a nearby matrix logarithm, to solve the velocity-based Karcher mean on SOn in a smooth manner such that the computed velocity-based Karcher mean smoothly depends on the data set and the convergence towards a computed mean can be controlled by the velocities emanated from the initial guess. Finally, numerical experiments demonstrate the advantages of the velocity-based Karcher mean on SOn in applications with smooth constraints.

Talk 3: Riemannian Federated Learning via Averaging Gradient Stream
Speaker: Zhenwei Huang
Abstract: In recent years, federated learning has garnered significant attention as an efficient and privacy-preserving distributed learning paradigm. In the Euclidean setting, federated averaging (FedAvg) and its variants are a class of efficient algorithms for expected (empirical) risk minimization. In this talk, we introduces and analyzes a Riemannian federated averaging gradient stream (RFedAGS) algorithm, which is a generalization of FedAvg, to problems defined on a Riemannian manifold. Under standard assumptions, the convergence rate of RFedAGS with fixed step sizes is proven to be sublinear for an approximate stationary solution. If decaying step sizes are used, the global convergence is established. Furthermore, assuming that the objective obeys the Riemannian Polyak-{\L}ojasiewicz property, the optimal gaps generated by RFedAGS with fixed step size are linearly decreasing up to a tiny upper bound, meanwhile, if decaying step sizes are used, then the gaps sublinearly vanish. Unlike the existing Riemannian federated learning whose convergence analysis only allows one agent or one step in local update, the proposed RFedAGS allows multiple agents and multiple-step local updates. Thus, RFedAGS theoretically guarantees a reduction of outer iterations and therefore reduces communication costs. Numerical simulations conducted on synthetic and real-world data demonstrate the performance of the proposed RFedAGS.

Speakers
WH

Wen Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
DS

Damien Scieur

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZD

Zhifeng Deng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZH

Zhenwei Huang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11P: Advanced Computational Solver 2
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Advanced Computational Solver 2
Chair: Jingyi Wang
Cluster: Computational Software

Talk 1: Implicit Scenario Reduction for Superquantile-Constrained Optimization: Applications to Energy Systems
Speaker: Jake Roth
Abstract: We discuss an efficient and scalable second-order solver for solving large-scale optimization problems with superquantile (aka conditional value at risk) constraints based on the augmented Lagrangian and semismooth Newton method framework. Unlike empirical risk models, superquantile models have non-separable constraints that make typical first-order algorithms difficult to scale. In contrast, our computational approach scales well in terms of the number of training data due to an implicit first- and second-order sparsity associated with the superquantile operator. In particular, only a fraction of the set of scenarios contributes second-order information, resulting in computations that can be performed in a reduced space. When evaluating the risk of each scenario is expensive, the relative cost of second-order information is diminished. Our solver is expected to help control the risk of adverse events for safety-critical applications in the power grid.

Talk 2: Exponential cone optimization with COPT
Speaker: Joachim Dahl
Abstract: Recently there has been increasing interest in conic optimization over non-symmetric cones. One important example is the exponential cone, which has recently been implemented in leading commercial solvers including COPT. In this talk we give an overview of the algorithm implemented in COPT and present numerical results showing the advantage and feasibility of embedding conic representable sets involving exponentials into a conic algorithm.

Talk 3: Randomized Algorithms for Bayesian Inversion and Data Acquisition in Predictive Digital Twins
Speaker: Vishwas Rao
Abstract: A digital twin couples computational models with a physical counterpart to create a system that is dynamically updated through bidirectional data flows as conditions change. Data Assimilation and Optimal Experimental Design (OED) provide a systematic means of updating the computational model and acquiring information as the physical system evolves. This talk will describe scalable preconditioners and solvers for Bayesian inversion using different randomization techniques. The proposed techniques are amenable to parallelization and drastically reduce the required number of model evaluations. We also develop theoretical guarantees on the condition number. Additionally, the talk will describe connections between OED for Bayesian linear inverse problems and the column subset selection problem (CSSP) in matrix approximation and derive bounds, both lower and upper, for the D-optimality criterion via CSSP for the independent and colored noise cases. We demonstrate the performance and effectiveness of our methodology on a variety of test problems such as Burgers and quasi-geostrophic equations

Speakers
JR

Jake Roth

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JD

Joachim Dahl

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
VR

Vishwas Rao

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11Q: Online Learning and Optimization over Symmetric Cones
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Online Learning and Optimization over Symmetric Cones
Chair: Antonios Varvitsiotis
Cluster: Conic and Semidefinite Optimization

Talk 1: SEMIDEFINITE NETWORK GAMES: MULTIPLAYER MINIMAX AND COMPLEMENTARITY PROBLEMS
Speaker: Constantin Ickstadt
Abstract: Network games provide a powerful framework for modeling agent in- teractions in networked systems, where players are represented by nodes in a graph, and their payoffs depend on the actions taken by their neighbors. Extending the framework of network games, in this work we introduce and study semidefinite net- work games. In this model, each player selects a positive semidefinite matrix with trace equal to one, known as a density matrix, to engage in a two-player game with every neighboring node. The player’s payoff is the cumulative payoff acquired from these edge games. Network semidefinite games are of interest because they provide a simplified framework for representing quantum strategic interactions. Initially, we focus on the zero-sum setting, where the sum of all players’ payoffs is equal to zero. We establish that in this class of games, Nash equilibria can be characterized as the projection of a spectrahedron. Furthermore, we demonstrate that determining whether a game is a semidefinite network game is equivalent to deciding if the value of a semidefinite program is zero. Beyond the zero-sum case, we characterize Nash equilibria as the solutions of a semidefinite linear complementarity problem.

Talk 2: Symmetric Cone Eigenvalue Optimization: Expressivity and Algorithms through Equilibrium Computation
Speaker: Jiaqi Zheng
Abstract: We investigate eigenvalue optimization problems over symmetric cones, a broad generalization of the matrix eigenvalue optimization problems. We show that symmetric cone eigenvalue optimization problems are highly expressive, capable of modeling a wide range of problems including nearest point problems, the fastest mixing Markov processes, robust regression, and computing the diameter of a convex set. From an algorithmic perspective, we show that these problems can be reformulated as two-player zero-sum or common-interest games over symmetric cones, enabling us to design algorithms for solving them through the lens of equilibrium computation. We implement these algorithms and assess their effectiveness in the aforementioned applications.

Talk 3: Optimistic Online Learning for Symmetric Cone Games
Speaker: Anas Bakarat
Abstract: Optimistic online learning algorithms have led to significant advances in equilibrium computation, particularly for two-player zero-sum games, achieving an iteration complexity of O(1/ϵ) to reach an ϵ-saddle point. These advances have been established in normal-form games, where strategies are simplex vectors, and quantum games, where strategies are trace-one positive semidefinite matrices. We extend optimistic learning to symmetric cone games (SCGs), a class of two-player zero-sum games where strategy spaces are generalized simplices—trace-one slices of symmetric cones. A symmetric cone is the cone of squares of a Euclidean Jordan Algebra; canonical examples include the nonnegative orthant, the second-order cone, the cone of positive semidefinite matrices, and their direct sums—all fundamental to convex optimization. SCGs unify normal-form and quantum games and, as we show, offer significantly greater modeling flexibility, allowing us to model applications such as distance metric learning problems and the Fermat-Weber problem. To compute approximate saddle points in SCGs, we introduce the Optimistic Symmetric Cone Multiplicative Weights Update algorithm and establish an iteration complexity of O(1/ϵ) to reach an ϵ-saddle point. Our analysis builds on the Optimistic Follow-the-Regularized-Leader framework, with a key technical contribution being a new proof of the strong convexity of the symmetric cone negative entropy with respect to the trace-one norm—a result that may be of independent interest.

Speakers
AV

Antonios Varvitsiotis

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11R: Communication-efficient distributed optimization
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Communication-efficient distributed optimization
Chair: Laurent Condat
Cluster: Nonlinear Optimization

Talk 1: Local Exact Diffusion for Decentralized Stochastic Optimization
Speaker: Sulaiman Alghunaim
Abstract: Distributed optimization methods featuring local updates have gained significant attention for their potential to cut down the communication costs in distributed systems. In these algorithms, nodes perform multiple local updates based on their data before exchanging estimation information. Although many studies have explored distributed local methods with centralized network connections, fewer have focused on decentralized networks. In this study, we introduce and examine a locally updated decentralized method named Local Exact-Diffusion (LED). This method allows each node to perform either fixed or random local updates, with communication occurring randomly at each iteration. We establish the convergence of LED in both convex and nonconvex settings for the stochastic online environment. Our convergence rate surpasses those of existing decentralized methods. Specializing the network to a centralized setup, we achieve the state-of-the-art bound for centralized methods. Additionally, we connect LED to various other independently studied distributed methods, including Scaffnew, FedGate, and VRL-SGD. Our analysis extends to these methods, and notably, we demonstrate that Scaffnew achieves a linear speedup property, previously unattained, where the communication complexity is inversely proportional with the number of nodes. This finding shows that Scaffnew can achieve linear speedup and potentially reduce communication overhead. In the strongly convex setting, we further prove that Scaffnew can achieve linear speedup with network-independent step sizes. Lastly, we numerically explore the advantages of local updates in decentralized networks and validate the effectiveness of our proposed method.

Talk 2: In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting
Speaker: Constantin Philippenko
Abstract: We analyze a distributed algorithm to compute a low-rank matrix factorization on several clients, each holding a local dataset. Considering a power initialization, we rewrite the previous smooth non-convex problem into a smooth strongly-convex problem that we solve using a parallel Nesterov gradient descent potentially requiring a single step of communication at the initialization step. We obtain a global-local matrix factorization: one part contains information on features shared by all clients (item embedding), while the other part captures the unique characteristics of each client (user embeddings). We provide a linear rate of convergence of the excess loss which depends on the truncated condition number, this result improves the rates of convergence given in the literature, which depend on the condition number. We provide an upper bound on the Frobenius-norm error of reconstruction under the power initialization strategy. We complete our analysis with experiments on both synthetic and real data.

Talk 3: LoCoDL: Communication-Efficient Distributed Optimization with Local Training and Compression
Speaker: Laurent Condat
Abstract: In distributed optimization, and even more in federated learning, communication is the main bottleneck. We introduce LoCoDL, a communication-efficient algorithm that leverages the two techniques of Local training, which reduces the communication frequency, and Compression with a large class of unbiased compressors that includes sparsification and quantization strategies. LoCoDL provably benefits from the two mechanisms and enjoys a doubly-accelerated communication complexity, with respect to the condition number of the functions and the model dimension, in the general heterogenous regime with strongly convex functions.

Speakers
SA

Sulaiman Alghunaim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CP

Constantin Philippenko

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Laurent Condat

Laurent Condat

Senior Research Scientist, King Abdullah University of Science and Technology (KAUST)
Laurent Condat received a PhD in applied mathematics in 2006 from Grenoble Institute of Technology, Grenoble, France. After a postdoc in the Helmholtz Zentrum Muenchen, Munich, Germany, he was hired in 2008 as a permanent researcher by the French National Center for Scientific Research... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11S: Interplay between Online Convex Optimization and Statistics
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Interplay between Online Convex Optimization and Statistics
Chair: Jun-Kun Wang
Cluster: Optimization For Data Science

Talk 1: On the near-optimality of betting confidence sets for bounded means
Speaker: Shubhanshu Shekhar
Abstract: Constructing nonasymptotic confidence intervals (CIs) for the mean of a univariate distribution from independent and identically distributed (i.i.d.) observations is a fundamental task in statistics. For bounded observations, a classical nonparametric approach proceeds by inverting standard concentration bounds, such as Hoeffding's or Bernstein's inequalities. Recently, an alternative betting-based approach for defining CIs and their time-uniform variants called confidence sequences (CSs), has been shown to be empirically superior to the classical methods. In this paper, we provide theoretical justification for this improved empirical performance of betting CIs and CSs. Our main contributions are as follows: (i) We first compare CIs using the values of their first-order asymptotic widths (scaled by ), and show that the betting CI of Waudby-Smith and Ramdas (2023) has a smaller limiting width than existing empirical Bernstein (EB)-CIs. (ii) Next, we establish two lower bounds that characterize the minimum width achievable by any method for constructing CIs/CSs in terms of certain inverse information projections. (iii) Finally, we show that the betting CI and CS match the fundamental limits, modulo an additive logarithmic term and a multiplicative constant. Overall these results imply that the betting CI~(and CS) admit stronger theoretical guarantees than the existing state-of-the-art EB-CI~(and CS); both in the asymptotic and finite-sample regimes.

Talk 2: Confidence sequences via online learning
Speaker: Kwang-Sung Jun
Abstract: Confidence sequence provides ways to characterize uncertainty in stochastic environments, which is a widely-used tool for interactive machine learning algorithms and statistical problems including A/B testing, Bayesian optimization, reinforcement learning, and offline evaluation/learning. In these problems, constructing confidence sequences that are tight without losing correctness is crucial since it has a dramatic impact on the performance of downstream tasks. In this talk, I will present how to leverage results from online learning to derive confidence sequences that are provably and numerically tight. First, I will present an implicitly-defined confidence sequence for bounded random variables, which induces an empirical Bernstein-style confidence bound (thus adapts to the variance) and is provably better than the KL divergence-based confidence bound simultaneously, unlike the standard empirical Bernstein bound. Our confidence bound is never vacuous, can be efficiently computed, and provides state-of-the-art numerical performance. Second, I will turn to generalized linear models and how leveraging online learning helps develop (i) improved confidence sets for logistic linear models and (ii) noise-adaptive confidence sets for linear models with sub-Gaussian and bounded noise respectively. These lead to improved regret bounds in (generalized) linear bandit problems. I will conclude with open problems in confidence sequences and the role that online learning plays for them.

Talk 3: Sequential Hypothesis Testing via Online Learning and Optimization
Speaker: Jun-Kun Wang
Abstract: Online convex optimization (a.k.a. no-regret learning) concerns a scenario where an online learner commits to a point at each round before receiving the loss function. The learner's goal is to minimize the regret, defined as the gap between the cumulative losses and that of a clairvoyant who knows the sequence of the loss functions in advance. In this talk, I will first review a very neat known result in the literature that casts non-parametric sequential hypothesis testing as an online convex optimization problem, where an online learner tries to bet whether the null hypothesis is true or false, and a tighter regret bound suggests a faster stopping time to reject the null when the alternative is true. Then, I will show the relevant techniques can be used to design algorithms with strong statistical guarantees with applications such as online detecting LLM-generated texts, auditing fairness, and detecting distribution shifts. After that, I will introduce a new algorithm that overcomes the limitations of the existing methods and potentially leads to a faster rejection time under the alternative while controlling the false positive rate.

Speakers
SS

Shubhanshu Shekhar

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
KJ

Kwang-Sung Jun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JW

Jun-Kun Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11T: Matrix optimization and geometry of curved spaces
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Matrix optimization and geometry of curved spaces
Chair: Florentin Goyens
Cluster: Optimization on Manifolds

Talk 1: Wrapped Gaussian distributions on SDP matrices and their estimation
Speaker: Florian Yger
Abstract: A common assumption for Euclidean data is that the underlying distribution is Gaussian, as the Gaussian distribution is both well-studied and it enables efficient parameter estimation from samples. In this work, we extend the concept of Gaussian distributions to the Riemannian manifold of Symmetric Positive Definite (SPD) matrices, with a focus on generalizing non-isotropic Gaussian distributions for complex statistical modeling in this space. We propose a wrapped Gaussian model, constructed by mapping a Euclidean Gaussian in a tangent space onto the SPD manifold via the exponential map. After defining this wrapped Gaussian distribution, we address the issue of non-identifiability of our model by establishing an equivalence relation among parameter sets that yield the same distribution. We then show that the parameters of a wrapped Gaussian can be estimated from sample data using a maximum likelihood estimator, optimized on a product manifold. Additionally, we reinterpret existing classifiers on the SPD manifold through a probabilistic framework and introduce new probabilistic classifiers based on wrapped Gaussian models. Finally, we present experimental results, both synthetic and real, to evaluate the parameter estimation accuracy and classification performance of our wrapped classifiers.

Talk 2: The ultimate upper bound on the injectivity radius of the Stiefel manifold
Speaker: Simon Mataigne
Abstract: We exhibit conjugate points on the Stiefel manifold endowed with any member of the family of Riemannian metrics introduced by Hüper et al. (2021). This family contains the well-known canonical and Euclidean metrics. An upper bound on the injectivity radius of the Stiefel manifold in the considered metric is then obtained as the minimum between the length of the geodesic along which the points are conjugate and the length of certain geodesic loops. Numerical experiments support the conjecture that the obtained upper bound is in fact equal to the injectivity radius. Authors: Pierre-Antoine Absil, Simon Mataigne

Talk 3: Deterministic and Randomized Direct Search on Riemannian Manifolds with Complexity Guarantees
Speaker: Florentin Goyens
Abstract: In this work, we investigate the problem of minimizing a nonconvex objective function defined on a Riemannian manifold, where derivative information is unavailable or impractical to compute. To address this, we consider the direct search methodology—a class of derivative-free optimization algorithms—in the Riemannian setting. We present a Riemannian adaptation of the deterministic direct search method and analyze its performance, deriving a global complexity bound on the number of function evaluations required to reach an approximate critical point. Building on this, we introduce a randomized variant of the Riemannian direct search algorithm, which operates by generating search directions in a random lower-dimensional subspace of the tangent space. Finally, we present numerical experiments to illustrate the effectiveness and computational advantages of both the deterministic and randomized Riemannian direct search methods. Co-authors: Bastien Cavarretta, Clément Royer and Florian Yger.

Speakers
FY

Florian Yger

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SM

Simon Mataigne

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Florentin Goyens

Florentin Goyens

Postdoc, UCLouvain
Researcher in numerical optimization at UCLouvain in Belgium
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11U: Grid Optimization (GO) Competition
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Grid Optimization (GO) Competition
Chair: Jesse Holzer
Cluster: Computational Software

Talk 1: Multi-Period Reserve-Constrained AC Optimal Power Flow on High Performance Computing
Speaker: Yong Fu
Abstract: The power grid is becoming more diverse and integrated with high-level distributed energy resources, creating new grid management challenges of large-scale, nonlinear, and non-convex problem modeling, complex and time-consuming computation. This work focuses on solutions for integrating operating reserves into system operation scheduling to co-optimize power generation, delivery, and reserve allocation over multiple time periods. The presented multi-period reserve constrained AC optimal power flow (M-ROPF) problem is challenged by the expanded model scale with linked decision variables across time periods, additional and comprehensive reserve requirements, and the inherent nonlinearity of power systems. We propose a parallel method for a fast, efficient, and reliable solution to M-ROPF. To decompose the problem, we reformulate the original problem by introducing auxiliary bounding variables for both producing and consuming devices, converting the hard and direct ramping up and down constraints to the soft and indirect real power dispatching boundaries. This reformulation yields a decomposable problem structure that can be solved by augmented Lagrangian relaxation-based decomposition methodologies. We use the alternating direction of method of multipliers to decompose the problem into two modules: the single period reserve constrained AC optimal power flow (S-RCOPF) module, and the single device ramping limit (S-DRL) module. The S-RCOPF module co-optimizes power dispatch and reserve allocation of devices by period, while satisfying AC network constraints, system reserve requirements, as well as the auxiliary real power dispatching boundaries for devices. It can be efficiently solved by an accelerated primal-dual interior point method that we developed. The S-DRL module determines the optimal auxiliary real power dispatching boundaries for a device which meets its intertemporal coupling constraints. It can be rapidly solved by quadratic programming. These solver modules can be processed in parallel, ensuring scalability across time periods. The solver guarantees feasibility through the whole iterative process, and achieves optimality in a limited time. The proposed parallel method is implemented and verified on the HPC platform. We will discuss multiple technical issues to enhance computational efficiency on multicore resources, such as task mapping strategy, communication and synchronization of tasks, and computational efficiency with increased computing processors. The effectiveness of the proposed solution is shown on datasets from the DOE ARPA-E Grid Optimization (GO) Competition Challenge 3, ranging from a small 73-bus system to a large-scale 23,643-bus system, and with different dispatch horizons including the real-time market with 8-hour look ahead, day-ahead market with 48-hour look ahead, and week-ahead advisory with 7-day look ahead. The results show a potential to meet the growing and diverse demands of the future electricity grid.

Talk 2: A Parallelized, Adam-Based Solver for Reserve and Security Constrained AC Unit Commitment
Speaker: Samuel Chevalier
Abstract: Power system optimization problems which include the nonlinear AC power flow equations require powerful and robust numerical solution algorithms. Within this sub-field of nonlinear optimization, interior point methods have come to dominate the solver landscape. Over the last decade, however, a number of efficient numerical optimizers have emerged from the field of Machine Learning (ML). One algorithm in particular, Adam, has become the optimizer-of-choice for a massive percentage of ML training problems (including, e.g., the training of GPT-3), solving some of the largest unconstrained optimization problems ever conceived of. Inspired by such progress, this talk presents a parallelized Adam-based numerical solver to overcome one of the most challenging power system optimization problems: security and reserve constrained AC Unit Commitment. The resulting solver, termed QuasiGrad, recently competed in the third ARPA-E Grid Optimization (GO3) competition. In the day-ahead market clearing category (with systems ranging from 3 to 23,643 buses over 48 time periods), QuasiGrad's aggregated market surplus scores were within 5% of the winningest market surplus scores. The QuasiGrad solver is now released as an open-source Julia package, and the internal gradient-based solver (Adam) can easily be substituted for other ML-inspired solvers (e.g., AdaGrad, AdaDelta, RMSProp, etc.).

Talk 3: Grid Optimization Competition and AC Unit Commitment
Speaker: Jesse Holzer
Abstract: The Grid Optimization (GO) Competition has posed challenge problems combining AC optimal power flow (ACOPF) and unit commitment (UC) into a single problem. UC typically is posed as a mixed integer linear programming (MILP) problem to determine the startup and shut down schedules along with real power output profiles of a set of generators in a power system so as to meet a given load over a planning horizon partitioned into multiple intervals. The physics of power flow through the network is represented by a linear model. In ACOPF, the network physics is modeled with a more accurate but nonlinear AC formulation, permitting the resolution of voltage and reactive power, but the discrete variables of generator commitment are either fixed to prescribed values or relaxed to continuous variables. Thus ACOPF is typically a nonlinear programming (NLP) problem in continuous variables. A day ahead UC with AC power flow modeling uses a more accurate power flow physics model to get a better day ahead commitment schedule. A real time ACOPF with unit commitment for fast start generators uses a more accurate representation of the actual capabilities and costs of the generators at the five to fifteen minute time scale. We describe two classes of solver approaches for a combined UC and ACOPF problem in the illustrative example of a single period UC model with AC power flow and balance constraints, voltage bounds, and limits on apparent power flow over transmission lines. In an NLP-based approach, we model the AC physics through nonlinear equations and use quadratic constraints to force the commitment variables to take integer values. In an MILP-based approach, we model the commitment decisions with binary variables and iteratively apply linear constraints derived from AC power flow subproblems to enforce the AC physics aspects of the model. We show computational results from these two approaches and discuss their advantages and disadvantages.

Speakers
YF

Yong Fu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
SC

Samuel Chevalier

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JH

Jesse Holzer

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11V: Nonsmooth Dynamical Systems
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Nonsmooth Dynamical Systems
Chair: Emilio Vilches
Cluster: Nonsmooth Optimization

Talk 1: Understanding accelerated gradient methods through high-order ODEs
Speaker: Samir Adly
Abstract: We study convex optimization problems with smooth objectives by looking at how damped inertial dynamics, driven by the gradient, can be discretized in time. This leads to three well-known accelerated algorithms: Nesterov’s method (NAG), Ravine Accelerated Gradient (RAG), and the IGAHD method introduced by Attouch, Chbani, Fadili, and Riahi. The IGAHD method uses Hessian-driven damping to reduce oscillations that often appear in inertial methods. By analyzing continuous-time models (ODEs) of these algorithms at different levels of resolution (orders $p=0,1,2$), we gain a better understanding of their behavior. All three methods share the same low-resolution model (order 0), which corresponds to the Su–Boyd–Candès ODE for NAG. However, when we go to higher resolution (order 2), we show that NAG and RAG follow different dynamics. This is a new result and shows that NAG and RAG should not be considered equivalent. We also present numerical experiments. In terms of number of iterations, IGAHD performs best. RAG is slightly better than NAG on average. When looking at CPU-time, NAG and RAG are faster than IGAHD. All three methods show similar results when comparing gradient norms.

Talk 2: A Newton-Like Dynamical System for Nonsmooth and Nonconvex Optimization
Speaker: Juan Guillermo Garrido
Abstract: This work investigates a dynamical system functioning as a nonsmooth adaptation of the continuous Newton method, aimed at minimizing the sum of a primal lower-regular and a locally Lipschitz function, both potentially nonsmooth. The classical Newton method’s second-order information is extended by incorporating the graphical derivative of a locally Lipschitz mapping. Specifically, we analyze the existence and uniqueness of solutions, along with the asymptotic behavior of the system's trajectories. Conditions for convergence and respective convergence rates are established under two distinct scenarios: strong metric subregularity and satisfaction of the Kurdyka-Łojasiewicz inequality.

Talk 3: (Cancelled)
Speaker: Emilio Vilches
Abstract: Cancelled

Speakers
JG

Juan Guillermo Garrido

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
EV

Emilio Vilches

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11W: Distributionally Robust Optimization (DRO) I
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Distributionally Robust Optimization (DRO) I
Chair: Man Yiu Tsang
Cluster: nan

Talk 1: On the trade-off between distributional belief and ambiguity: Conservatism, finite-sample guarantees, and asymptotic properties
Speaker: Man Yiu Tsang
Abstract: We present a new data-driven trade-off (TRO) approach for modeling uncertainty that serves as a middle ground between the optimistic approach, which adopts a distributional belief, and the pessimistic distributionally robust optimization approach, which hedges against distributional ambiguity. We equip the TRO model with a TRO ambiguity set characterized by a size parameter controlling the level of optimism and a shape parameter representing distributional ambiguity. We first show that constructing the TRO ambiguity set using a general star-shaped shape parameter with the empirical distribution as its star center is necessary and sufficient to guarantee the hierarchical structure of the sequence of TRO ambiguity sets. Then, we analyze the properties of the TRO model, including quantifying conservatism, quantifying bias and generalization error, and establishing asymptotic properties. Specifically, we show that the TRO model could generate a spectrum of decisions, ranging from optimistic to conservative decisions. Additionally, we show that it could produce an unbiased estimator of the true optimal value. Furthermore, we establish the almost-sure convergence of the optimal value and the set of optimal solutions of the TRO model to their true counterparts. We exemplify our theoretical results using stylized optimization problems.

Talk 2: Generalization Bound Analysis of Nonconvex Minimax Optimization and Beyond
Speaker: Siqi Zhang
Abstract: In this work, we systematically investigate the generalization bounds of algorithms that solve nonconvex–(strongly)–concave (NC-SC/NC-C) stochastic minimax optimization, measured by the stationarity of primal functions. We first establish algorithm-agnostic generalization bounds via uniform convergence between the empirical and population minimax problems, thereby deriving sample complexities for achieving generalization. We then explore algorithm-dependent generalization bounds using algorithmic stability arguments. In particular, we introduce a novel stability notion for minimax problems and build its connection to generalization bounds. Consequently, we establish algorithm-dependent generalization bounds for stochastic gradient descent ascent (SGDA) and more general sampling-based algorithms. We will also discuss some extensions of these results to more general settings.

Talk 3: Optimized Dimensionality Reduction for Moment-based Distributionally Robust Optimization
Speaker: Kai Pan
Abstract: Moment-based distributionally robust optimization (DRO) provides an optimization framework to integrate statistical information with traditional optimization approaches. Under this framework, one assumes that the underlying joint distribution of random parameters runs in a distributional ambiguity set constructed by moment information and makes decisions against the worst-case distribution within the set. Although most moment-based DRO problems can be reformulated as semidefinite programming (SDP) problems that can be solved in polynomial time, solving high-dimensional SDPs is still time-consuming. Unlike existing approximation approaches that first reduce the dimensionality of random parameters and then solve the approximated SDPs, we propose an optimized dimensionality reduction (ODR) approach by integrating the dimensionality reduction of random parameters with the subsequent optimization problems. Such integration enables two outer and one inner approximations of the original problem, all of which are low-dimensional SDPs that can be solved efficiently, providing two lower bounds and one upper bound correspondingly. More importantly, these approximations can theoretically achieve the optimal value of the original high-dimensional SDPs. As these approximations are nonconvex SDPs, we develop modified Alternating Direction Method of Multipliers (ADMM) algorithms to solve them efficiently. We demonstrate the effectiveness of our proposed ODR approach and algorithm in solving multiproduct newsvendor and production-transportation problems. Numerical results show significant advantages of our approach regarding computational time and solution quality over the three best possible benchmark approaches. Our approach can obtain an optimal or near-optimal (mostly within 0.1%) solution and reduce the computational time by up to three orders of magnitude. Paper reference: Jiang, S., Cheng, J., Pan, K., & Shen, Z. J. M. (2023). Optimized dimensionality reduction for moment-based distributionally robust optimization. arXiv preprint arXiv:2305.03996.

Speakers
MY

Man Yiu Tsang

Name: Man Yiu (Tim) TsangTitle: Ph.D. CandidateAffiliation: Lehigh UniversityBio:Man Yiu (Tim) Tsang is a Ph.D. candidate in Industrial and Systems Engineering at Lehigh University under the supervision of Prof. Karmel S. Shehadeh. He obtained his BSc and MPhil in Risk Management... Read More →
SZ

Siqi Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Kai Pan

Kai Pan

Associate Professor, The Hong Kong Polytechnic University
Kai Pan is currently an Associate Professor in Operations Management at the Faculty of Business of The Hong Kong Polytechnic University (PolyU) and the Director of the MSc Program in Operations Management (MScOM). He serves as a Secretary/Treasurer for the INFORMS Computing Socie... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11X: Nonsmooth Optimization - Algorithms
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Session: Nonsmooth Optimization - Algorithms
Chair: Dânâ Davar
Cluster: nan

Talk 1: TRFD: A derivative-free trust-region method based on finite differences for composite nonsmooth optimization
Speaker: Dânâ Davar
Abstract: In this work we present TRFD, a derivative-free trust-region method based on finite differences for minimizing composite functions of the form $f(x)=h(F(x))$, where $F$ is a black-box function assumed to have a Lipschitz continuous Jacobian, and $h$ is a known convex Lipschitz function, possibly nonsmooth. The method approximates the Jacobian of $F$ via forward finite differences. We establish an upper bound for the number of evaluations of $F$ that TRFD requires to find an $\epsilon$-approximate stationary point. For L1 and Minimax problems, we show that our complexity bound reduces to $\mathcal{O}(n\epsilon^{-2})$ for specific instances of TRFD, where $n$ is the number of variables of the problem. Assuming that $h$ is monotone and that the components of $F$ are convex, we also establish a worst-case complexity bound, which reduces to $\mathcal{O}(n\epsilon^{-1})$ for Minimax problems. Numerical results are provided to illustrate the relative efficiency of TRFD in comparison with existing derivative-free solvers for composite nonsmooth optimization. This is a joint work with Geovani Grapiglia. The arXiv manuscript can be found at https://arxiv.org/abs/2410.09165

Talk 2: AN AUGMENTED LAGRANGIAN PRIMAL-DUAL SEMISMOOTH NEWTON METHOD FOR MULTI-BLOCK COMPOSITE OPTIMIZATION
Speaker: Zhanwang Deng
Abstract: In this talk, we develop a novel primal-dual semismooth Newton method for solving linearly constrained multi-block convex composite optimization problems which include commonly used models in image processing and conic programming. First, a differentiable augmented Lagrangian (AL) function is constructed by utilizing the Moreau envelopes of the nonsmooth functions. It enables us to derive an equivalent saddle point problem and establish the strong AL duality under the Slater’s condition. Consequently, a semismooth system of nonlinear equations is formulated to characterize the optimality of the original problem instead of the inclusion-form KKT conditions. We then develop a semismooth Newton method, called ALPDSN, which uses purely second-order steps and a nonmonotone line search based globalization strategy. Through a connection to the inexact first-order steps when the regularization parameter is sufficiently large, the global convergence of ALPDSN is established. Under the regularity conditions, partial smoothness, the local error bound, and the strict complementarity, we show that both the primal and the dual iteration sequences possess a superlinear convergence rate and provide concrete examples where these regularity conditions are met. Numerical results on the image restoration with two regularization terms, the corrected tensor nuclear norm problem and semidefinite programming on Mittelmann benchmark are presented to demonstrate the high efficiency and robustness of our ALPDSN. Furthermore, we will also design a software to solve a class of convex composite optimization in the future.

Talk 3: Anderson acceleration in nonsmooth problems: local convergence via active manifold identification
Speaker: Hao Wang
Abstract: Anderson acceleration is an effective technique for enhancing the efficiency of fixed- point iterations; however, analyzing its convergence in nonsmooth settings presents significant challenges. In this paper, we investigate a class of nonsmooth optimization algorithms characterized by the active manifold identification property. This class includes a diverse array of methods such as the proximal point method, proximal gradient method, proximal linear method, proximal coordinate descent method, Douglas-Rachford splitting (or the alternating direction method of multipliers), and the iteratively reweighted L1 method, among others. Under the assumption that the optimization problem possesses an active manifold at a stationary point, we establish a local R-linear convergence rate for the Anderson-accelerated algorithm. Our extensive numerical experiments further highlight the robust performance of the proposed Anderson-accelerated methods.

Speakers
ZD

Zhanwang Deng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HW

Hao Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

1:15pm PDT

Parallel Sessions 11Y
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Thursday July 24, 2025 1:15pm - 2:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

2:30pm PDT

Coffee & Snack Break (Provided)
Thursday July 24, 2025 2:30pm - 3:00pm PDT
Thursday July 24, 2025 2:30pm - 3:00pm PDT
TBA

3:00pm PDT

Parallel Semi-Plenary Talk 4A
Thursday July 24, 2025 3:00pm - 4:00pm PDT
Speakers
TH

Tim Hoheisel

Tim Hoheisel received his doctorate of mathematics from Julius-Maximilians University (Würzburg) under the supervision of Christian Kanzow in 2009. He was a postdoctoral researcher there until 2016. During this time, he was a visiting professor at Heinrich-Heine University (Düsseldorf... Read More →
Thursday July 24, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

3:00pm PDT

Parallel Semi-Plenary Talk 4B
Thursday July 24, 2025 3:00pm - 4:00pm PDT
Speakers
MH

Mingyi Hong

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 3:00pm - 4:00pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:00pm PDT

Break
Thursday July 24, 2025 4:00pm - 4:15pm PDT
Thursday July 24, 2025 4:00pm - 4:15pm PDT
TBA

4:15pm PDT

Parallel Sessions 12A: Distributionally Robust Optimization (DRO) II
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Distributionally Robust Optimization (DRO) II
Chair: Yiling Zhang
Cluster: nan

Talk 1: Distributionally robust standard quadratic optimization with Wasserstein ambiguity
Speaker: Daniel de Vicente
Abstract: The standard quadratic optimization problem (StQP) consists of minimizing a quadratic form over the standard simplex. If the quadratic form is neither convex nor concave, the StQP is NP-hard. This problem has many interesting applications ranging from portfolio optimization to machine learning. Sometimes, the data matrix is uncertain but some information about its distribution can be inferred, e.g. the first two moments or else a reference distribution (typically, the empirical distribution after sampling). In distributionally robust optimization, the goal is to minimize over all possible distributions in an ambiguity set defined based upon above mentioned characteristics. We will explore two versions: the distributionally robust stochastic StQP focussing on expectations, and the distributionally robust chance constrained StQP, both with an ambiguity set based upon maximal Wasserstein distance to the sampled distribution.

Talk 2: A Primal Perspective on Distributionally Robust Two-Stage Problems with Integer Recourse
Speaker: Yiling Zhang
Abstract: In this talk, we introduce and study a two-stage distributionally two-stage linear problems with integer recourse, where the objective coefficients are random. The random parameters follow the worst-case distribution belonging to an a second-order conic representable ambiguity set of probability distributions. We show that the worst-case recourse objective, under various risk measures, can be formulated as a conic program from a primal perspective. This method also provides additional information on the probability of attaining an integer recourse solution, extending the concept of persistency studied in Bertsimas et al. (2006). Unlike the marginal moment sets used in Bertsimas et al. 2006), the second-order conic representable ambiguity sets in our method offers greater flexibility by incorporating more distributional information. Furthermore, this method enables column constraint generation methods for solving two-stage problems with integer recourse.

Talk 3: Distributionally Robust Nonlinear Optimization
Speaker: Judith Brugman
Abstract: Distributionally robust optimization (DRO) provides a powerful framework for handling uncertainty when only partial information, such as mean, variance and support, is available. Instead of assuming full knowledge of the probability distribution, DRO seeks solutions that perform well under the worst-case distribution within an ambiguity set. While DRO problems can be reformulated as robust optimization (RO) problems, making them more tractable while maintaining theoretical guarantees, solving the resulting RO problem remains challenging. Wiesemann et al. (2014) address this problem for a very rich class of ambiguity sets, but relies on a max-of-linear-functions assumption on the cost function, limiting its applicability. In our work, we extend this approach to a much broader class of cost functions, including all convex and twice continuously differentiable functions. By leveraging the Reformulation-Perspectification Technique with Branch and Bound (RPT-BB) for RO, which combines relaxation-based partitioning with branch-and-bound techniques, we show that DRO problems can be efficiently solved even for highly nonlinear functions. To demonstrate the practical relevance of this approach, I will focus on the appointment scheduling problem, where our method not only generalizes existing results but also improves computational efficiency. I will conclude with a discussion on broader applications of our framework in other domains.

Speakers
DD

Daniel de Vicente

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YZ

Yiling Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JB

Judith Brugman

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 101 3501 Trousdale Pkwy, 101, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12B: Optimization on Manifolds and Geometric Approaches
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Optimization on Manifolds and Geometric Approaches
Chair: Ian McPherson
Cluster: nan

Talk 1: Convergence Rates for Riemannian Proximal Bundle Methods
Speaker: Ian McPherson
Abstract: We propose a novel class of Riemannian proximal bundle methods for optimization on Hadamard manifolds, providing convergence rates for this new approach. Our assumptions are weak and we are able to relax the reliance on exponential maps and parallel transports, requiring only first-order retractions and vector transports. To our knowledge, these are the first non-asymptotic convergence rates for Riemannian proximal bundle methods, extending arguments that achieve optimal rates in the Euclidean case. Moreover, we show given exponential maps and parallel transports we recover the exact same rates as the Euclidean setting.

Talk 2: Incremental minimization in nonpositively curved geodesic spaces
Speaker: Ariel Goodwin
Abstract: Subgradient methods for minimizing geodesically convex functions on Hadamard manifolds have gained interest in recent years, but fall short in two respects: their complexity relies unavoidably on a lower curvature bound for the space, and they do not generalize well to metric spaces in the absence of local linearity. Complete geodesic metric spaces of nonpositive curvature, called Hadamard spaces, prove useful in modelling many applications and have a rich geometric structure enabling theoretical and computational aspects of convex optimization. It has recently been shown that a restricted class of functions on Hadamard spaces can be effectively minimized using an iteration resembling a subgradient method, with the same complexity result as the classical Euclidean subgradient method. In this work we propose a related class of functions which we call Busemann convex, admitting a notion of subgradient that is attuned to the geometry of the space. Many functions defined in terms of basic metric quantities are Busemann convex, and their subgradients are readily computed in terms of geodesics. We address the minimization of sums of Busemann convex functions with an incremental subgradient-style method and associated complexity result. To illustrate the algorithm, we numerically compute medians of trees in the BHV phylogenetic tree space. This is joint work with Adrian Lewis, Genaro López-Acedo, and Adriana Nicolae. A preprint is available at https://arxiv.org/abs/2412.06730.

Talk 3: Interior Riemannian subgradient flow over manifold with boundary
Speaker: Kuangyu Ding
Abstract: We study a nonsmooth nonconvex optimization problem defined over a manifold with boundary, where the feasible set is given by the intersection of the closure of an open set and a smooth manifold. By endowing the open set with a Riemannian metric induced by a barrier function, we obtain a Riemannian subgradient flow—formulated as a differential inclusion—that remains strictly within the interior of the feasible set. This continuous dynamical system unifies two classes of iterative optimization methods, namely the Hessian barrier method and Bregman-type methods, by revealing that these methods can be interpreted as discrete approximations of the continuous flow. We analyze the long-term behavior of the trajectories generated by this dynamical system and show that many properties of the Hessian barrier and Bregman-type methods can be more insightfully understood through these of the continuous trajectory. For instance, the notorious spurious stationary points observed in Bregman-type methods are interpreted as stable equilibria of the dynamical system that do not correspond to true stationary points of the original problem. We prove that these spurious stationary points can be avoided if the strict complementarity condition holds. In the absence of this regularity condition, we propose a random perturbation strategy that ensures the trajectory converges (subsequentially) to an approximate stationary point. Building on these insights, we introduce an iterative Riemannian subgradient method—a form of interior point approach—that generalizes the existing Hessian barrier method for solving nonsmooth nonconvex optimization problems.

Speakers
IM

Ian McPherson

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
AG

Ariel Goodwin

Cornell University Center for Applied Mathematics, Ph.D. Student.
KD

Kuangyu Ding

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 201 3501 Trousdale Pkwy, 201, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12C: Convex Relaxations for Discrete & Combinatorial Optimization
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Convex Relaxations for Discrete & Combinatorial Optimization
Chair: Soobin Choi
Cluster: nan

Talk 1: Rank-one Convexification for Convex Quadratic Optimization with Sign Indicators
Speaker: Soobin Choi
Abstract: We investigate convexification in convex quadratic optimization with sign indicators, where the sign of each continuous variable in the quadratic objective is controlled by a binary variable. First, we derive the convex hull for the epigraph of a quadratic function defined by a rank-one matrix. Using this rank-one convexification, we develop copositive and semi-definite relaxations for general convex quadratic functions. Leveraging these findings, we reformulate the support vector machine problem with 0--1 loss and conduct numerical experiments on synthetic and real instances.

Talk 2: Tightening Quadratic Convex Relaxations for the AC Optimal Transmission Switching Problem
Speaker: Cheng Guo
Abstract: The Alternating Current Optimal Transmission Switching (ACOTS) problem incorporates line switching decisions into the AC Optimal Power Flow framework, offering benefits in reducing costs and enhancing reliability. ACOTS optimization models contain discrete variables and nonlinear, non-convex constraints, which make it difficult to solve. We develop strengthened quadratic convex (QC) relaxations for ACOTS, where we tighten the relaxation with several new valid inequalities, including a novel kind of on/off cycle-based polynomial constraints by taking advantage of the network structure. We demonstrate theoretical tightness, and efficiently incorporate on/off cycle-based polynomial constraints through disjunctive programming-based cutting planes. Our method results in the tightest QC-based ACOTS relaxation to date. We additionally propose a novel maximum spanning tree-based heuristic to improve the computational performance by fixing certain lines to be switched on. Tests on large-scale instances with up to 2,312 buses demonstrate substantial performance gains. This work in under minor revision at INFORMS Journal on Computing, and is available on arXiv: https://arxiv.org/abs/2212.12097

Talk 3: Solving Large Flag Algebra Problems
Speaker: Bernard Lidický
Abstract: The Flag Algebra Method is a tool in extremal combinatorics, which has been very useful to tackle open problems for graphs, hypergraphs, permutations, pointsets and others. The method establishes combinatorial inequalities via sums of squares, which are usually obtained by a semidefinite relaxation of the original combinatorial problem. A typical application of the method in graph theory starts with fixing a parameter $n$ followed by constructing a semidefinite program based on the (subgraph) structure of all $n$-vertex graphs. In particular, the number of constraints in the resulting program is at least the number of all $n$-vertex graphs upto isomorphism. These numbers grow very quickly; for example, they are 12346, 274668 and 12005168 for $n$ being 8, 9 and 10, respectively. While solving the resulting program for $n=8$ is an easy task on any modern computer, the program for $n \geq 10$ seems out of reach, mainly because the SDP solvers need too much RAM. Yet solving such programs for as large $n$ as possible is desirable because they typically yield better bounds for the original combinatorial problems. We bypass the aforementioned memory requirements by linear programming with cutting-planes. Although solving semidefinite programs using this approach has no optimality guarantee, it turned out that in many flag algebra instances much less memory is needed to significantly improve bounds on numerous well-known hypergraph Turán problems. This is a joint work with Jan Volec.

Speakers
SC

Soobin Choi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CG

Cheng Guo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
BL

Bernard Lidický

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 208 3501 Trousdale Pkwy, 208, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12D: Sparsity Optimization
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Sparsity Optimization
Chair: Jiachang Liu
Cluster: nan

Talk 1: (Canceled) A PATH-BASED APPROACH TO CONSTRAINED SPARSE OPTIMIZATION
Speaker: Nadav Hallak
Abstract: This talk presents a path-based approach for the minimization of a continuously differentiable function over sparse symmetric sets, which is a hard problem that exhibits a restrictiveness-hierarchy of necessary optimality conditions. To achieve the more restrictive conditions in the hierarchy, contemporary methods require a support optimization oracle that must exactly solve the problem in smaller dimensions. The presented path-based approach achieves a path-based optimality condition, which is well placed in the restrictiveness-hierarchy, and a method to achieve it that does not require a support optimization oracle and, moreover, is projection-free. New results for the regularized linear minimization problem over sparse symmetric sets and their implications are also presented.

Talk 2: Scalable First-order Method for Certifying Optimal k-Sparse GLMs
Speaker: Jiachang Liu
Abstract: This paper investigates the problem of certifying optimality for sparse generalized linear models (GLMs), where sparsity is enforced through an l0 cardinality constraint. While branch-and-bound (BnB) frameworks can certify optimality by pruning nodes using dual bounds, existing methods for computing these bounds are either computationally intensive or exhibit slow convergence, limiting their scalability to large-scale problems. To address this challenge, we propose a first-order proximal gradient algorithm designed to solve the perspective relaxation of the problem within a BnB framework. Specifically, we formulate the relaxed problem as a composite optimization problem and demonstrate that the proximal operator of the non-smooth component can be computed exactly in log-linear time complexity, eliminating the need to solve a computationally expensive second-order cone program. Furthermore, we introduce a simple restart strategy that enhances convergence speed while maintaining low per-iteration complexity. Extensive experiments on synthetic and real-world datasets show that our approach significantly accelerates dual bound computations and is highly effective in providing optimality certificates for large-scale problems. This is joint work with Andrea Lodi and Soroosh Shafiee. A preprint of this work can be found at https://arxiv.org/abs/2502.09502.

Talk 3: Sparse quadratic optimization via cardinality constraints
Speaker: Ademir Ribeiro
Abstract: We study the sparse regularized quadratic optimization problem where the sparsity level is controlled by means of the cardinality constraints. However, due to the existence of this kind of constraint (which is not continuous neither convex), it is very difficult to directly solve the problem. Recently, several researchers have addressed these type of problems using a common strategy, a continuous relaxation reformulation of the problem. In this work we propose a different approach in which the feasible set of the problem is decomposed into simpler sets, all of them meeting the cardinality constraint. Then, in order to overcome the combinatorial difficulty induced by this decomposition, we make use of an auxiliary and simple problem of maximizing a concave piecewise quadratic function. The solution of this problem, obtained by a subgradient method, is then used to find the solution of the original problem. Numerical experiments show that this strategy may be successful for a significant number of problems. Throughout this work we establish nonlinear optimization results in order to provide a rigorous mathematical analysis of the ideas involving the reformulation of problem, the proposed method and its convergence properties. Beck A, Eldar YC (2013) Sparsity constrained nonlinear optimization: Optimality conditions and algorithms. SIAM J. Optim. 23(3):1480–1509. Nesterov Y (2004) Introductory Lectures on Convex Optimization – Basic Course (Kluwer Academic Publishers). Vreugdenhil R, Nguyen VA, Eftekhari A, Esfahani PM (2021) Principal component hierarchy for sparse quadratic programs. Meila M, Zhang T, eds., Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, 10607–10616 (PMLR).

Speakers
NH

Nadav Hallak

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Jiachang Liu

Jiachang Liu

Assistant Research Professor, Cornell University
I am an assistant research professor (postdoc) at the Center for Data Science for Enterprise and Society (CDSES) at Cornell University. My hosts are Professor Andrea Lodi and Professor Soroosh Shafiee.Prior to joining Cornell, I completed my Ph.D. in Electrical and Computer Engineering... Read More →
AR

Ademir Ribeiro

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 210 3501 Trousdale Pkwy, 210, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12E: Sparse and Low-rank Optimization
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Sparse and Low-rank Optimization
Chair: Ishy Zagdoun
Cluster: nan

Talk 1: On solving a rank regularized minimization problem via equivalent factorized column-sparse regularized models
Speaker: Wenjing Li
Abstract: Rank regularized minimization problem is an ideal model for the low-rank matrix completion/recovery problem. The matrix factorization approach can transform the high-dimensional rank regularized problem to a low-dimensional factorized column-sparse regularized problem. The latter can greatly facilitate fast computations in applicable algorithms, but needs to overcome the simultaneous non-convexity of the loss and regularization functions. In this talk, we consider the factorized column-sparse regularized model. Firstly, we optimize this model with bound constraints, and establish a certain equivalence between the optimized factorization problem and rank regularized problem. Further, we strengthen the optimality condition for stationary points of the factorization problem and define the notion of strong stationary point. Moreover, we establish the equivalence between the factorization problem and its a nonconvex relaxation in the sense of global minimizers and strong stationary points. To solve the factorization problem, we design two types of algorithms and give an adaptive method to reduce their computation. The first algorithm is from the relaxation point of view and its iterates own some properties from global minimizers of the factorization problem after finite iterations. We give some analysis on the convergence of its iterates to the strong stationary point. The second algorithm is designed for directly solving the factorization problem. We improve the PALM algorithm introduced by Bolte et al. (Math Program Ser A 146:459-494, 2014) for the factorization problem and give its improved convergence results. Finally, we conduct numerical experiments to show the promising performance of the proposed model and algorithms for low-rank matrix completion. The corresponding publication is Math Program Ser A, 2024, doi: 10.1007/s10107-024-02103-1.

Talk 2: Convergence of accelerated singular value shrinkage algorithm for nonconvex low-rank matrix regularization problem
Speaker: LIU Yanyan
Abstract: This paper develops a new framework to design and analyse singular value shrinkage algorithm and its variants built on the notion of singular value shrinkage operator for the nonconvex low-rank regularization problem. The truncation technique, Nesterov's acceleration and heavy-ball method are chosen to accelerate traditional singular value shrinkage algorithm. We establish their convergence to an apporixmate true low-rank solution of nonconvex regularization problem under restricted isometry condition and some mild parameter assumptions. Numerical results based on sythetical data and real data show that the proposed algorithms are competitive to the state-of-the-art algorithms in terms of efficiency and accuracy.

Talk 3: Projecting onto a Capped Rotated Second-Order Cone for Sparse Optimization
Speaker: Ishy Zagdoun
Abstract: This paper presents a closed-form expression for the projection onto a capped rotated second-order cone, a convex set that emerges as part of the Boolean relaxation in sparse regression problems and, more broadly, in the perspective relaxation of mixed-integer nonlinear programs (MINLP) with binary indicator variables. The closed-form expression is divided into three cases, one of which simplifies to the projection onto a standard second-order cone (a widely used projection with a well-known solution involving three additional cases). The nontrivial solutions for the other two cases include the necessary and sufficient conditions for when the projection lies along the intersection of the rotated cone and a facet of a box. The ability to rapidly compute the projection onto a capped rotated second-order cone facilitates the development of efficient methods for solving the continuous relaxation of sparse optimization problems. These problems typically involve a Cartesian product of many such sets. Important machine learning tasks that benefit from this closed form expression include solving the continuous relaxation of a sparse regression using projected gradient methods and the accelerated variant of this method (FISTA). Since the closed-form expression is derived in a general manner, it can be seamlessly extended to regression problems with group sparsity constraints, involving cones of a dimension beyond the three-dimensional cone used in standard sparsity penalties and constraints.

Speakers
avatar for Wenjing Li

Wenjing Li

Associate Researcher, Harbin Institute of Technology
Name: Dr. Wenjing LiTitle: Associate ResearcherAffiliation: Harbin Institute of TechnologyBio:Dr. Wenjing Li is an associate researcher at Harbin Institute of Technology. Under the guidance of Prof. Wei Bian, she obtained her Master's and PhD degrees from the School of Mathematics... Read More →
LY

LIU Yanyan

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 212 3501 Trousdale Pkwy, 212, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12F: PDE-Constrained Optimization and Optimal Control
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: PDE-Constrained Optimization and Optimal Control
Chair: Henrik Wyschka
Cluster: nan

Talk 1: Numerical Solution of p-Laplace Problems for Shape Optimization
Speaker: Henrik Wyschka
Abstract: Shape optimization constrained to partial differential equations is a vibrant field of research with high relevance for industrial-grade applications. Recent developments suggest that using a p-harmonic approach to determine descent directions is superior to classical Hilbert space methods. This applies in particular to the representation of kinks and corners in occurring shapes. However, the approach requires the efficient solution of a p-Laplace problem in each descent step. Therefore, we extend an algorithm based on an interior-point method without resorting to homotopy techniques for high orders p. Further, we discuss modifications for the limit setting. A key challenge in this regard is that the Lipschitz deformations obtained as solutions in limit setting are in general non-unique. Thus, we focus on solutions which are in a sense limits to solutions for finite p and aim to preserve mesh quality throughout the optimization. Building upon this work, we also aim to reduce the number of outer iterations and thus calls of the algorithm by proposing a trust-region method. Due to the structure of the algorithm for finite p, we are able to introduce a constraint on the gradient of the solution naturally. Consequently, the obtained deformation fields also fulfill a trust-radius in terms of the Lipschitz topology.

Talk 2: A Variational and Adjoint Calculus for Optimal Control of the Generalized Riemann Problem for Hyperbolic Systems of Conservation Laws
Speaker: Jannik Breitkopf
Abstract: In this talk, we analyze optimal control problems for quasilinear strictly hyperbolic systems of conservation laws where the control is the initial state of the system. The problem is interesting, for example, in the context of fluid mechanics or traffic flow modelling. Similar problems for scalar conservation laws have already been studied. However, the case of hyperbolic systems is more involved due to the coupling of the characteristic fields. We begin our analysis by considering the Generalized Riemann Problem, which has a piecewise smooth initial state with exactly one discontinuity. This is a natural choice since it is well known that solutions to hyperbolic conservation laws generally develop discontinuities even for smooth data. For piecewise $C^1$ initial data we obtain the existence, uniqueness and stability of an entropy solution by a careful fixed point argument built on the associated Riemann Problem with piecewise constant initial states. The construction yields insights into the structure and regularity of the solution and provides a foundation to derive differentiability results of the control-to-state mapping. The entropy solution is piecewise $C^1$. Its smooth parts are separated by $C^2$ curves which are either shock curves or boundaries of rarefaction waves. In a subsequent step, we show that these curves depend differentiably on the initial state. This allows the transformation to a fixed space-time domain as a reference space. In this reference space, we can show that the transformed solution depends differentiably on the initial state in the topology of continuous functions. For this, a detailed knowledge of the structure of the solution and the behaviour of the shock curves is crucial. As an immediate consequence, the differentiability of tracking type functionals for the optimal control problem follows. Finally, we investigate the adjoint problem as an efficient way to compute the gradient of the objective functional. The adjoint problem is a linear system of transport equations with a discontinuous coefficient and possibly discontinuous terminal data. In general, problems of this kind do not admit unique solutions. We derive interior boundary conditions to characterize the correct reversible solution. This is a joint work with Stefan Ulbrich.

Talk 3: A Variational Calculus for Optimal Control of Networks of Scalar Conservation or Balance Laws
Speaker: Marcel Steinhardt
Abstract: Networks of scalar conservation or balance laws provide models for vehicular traffic flow, supply chains or transmission of data. These networks usually consist of initial boundary value problems (IBVPs) of scalar conservation or balance laws on every edge coupled by node conditions. For the optimal control of solutions a variational calculus is desirable that implies differentiability of objective functionals w.r.t. controls. In the last decade research on IBVPs successfully introduced a variational calculus which implies differentiability of objective functionals of tracking type and also yields an adjoint based gradient representation for the functional. This talk presents recent progress in an extension of these results to networks of scalar conservation or balance laws. Regarding node conditions we introduce a framework for their representation compatible with the known approach on single edges which allows us to extend the results such as continuous Fréchet differentiability for functionals of tracking-type and an adjoint based gradient representation on the network. Joint work with Stefan Ulbrich

Speakers
avatar for Henrik Wyschka

Henrik Wyschka

University of Hamburg
JB

Jannik Breitkopf

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MS

Marcel Steinhardt

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 156 3518 Trousdale Pkwy, 156, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12G: PDE-Constrained Optimization
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: PDE-Constrained Optimization
Chair: Zhexian Li
Cluster: nan

Talk 1: Robust Topology Optimisation of Electric Machines using Topological Derivatives
Speaker: Theodor Komann
Abstract: Designing high-performance electrical machines that remain efficient and reliable under uncertain material and operating conditions is important for industrial applications. In this paper, we present a robust topology optimization framework with PDE constraints to address this challenge. We formulate the robust optimisation problem as a min-max problem, where the inner maximisation considers the worst-case under predefined uncertainties, and the outer minimisation searchs for a design that remains robust to these variations using the topological derivative. A level set function represents the shape of the domain, allowing for arbitary perturbations. We use a theorem of Clarke to compute subgradients of the worst-case function, thereby ensuring efficient solution of the min-max problem. Finally, numerical results on a two-material permanent magnet synchronous machine illustrate both the effectiveness of the method and the improved performance of designs that account for uncertainty. Theodor Komann, TU Darmstadt Joint work with Peter Gangl and Nepomuk Krenn, RICAM Linz, and Stefan Ulbrich, TU Darmstadt

Talk 2: Generalized Nash equilibrium problems for linear hyperbolic PDE-constrained games
Speaker: Marcelo Bongarti
Abstract: The concept of Nash equilibrium is fundamental to a wide range of applications, spanning fields from particle mechanics to micro and macroeconomics. However, much of the existing literature focuses on finite-dimensional settings. In this talk, we draw on energy markets coupled with transport dynamics to motivate the study of multi-objective optimization problems with linear hyperbolic PDE constraints. We present recent results on the existence and characterization of Nash equilibria, emphasizing optimality conditions as a framework for understanding such solutions.

Talk 3: Bilinear optimal control of advection-diffusion-reaction equations
Speaker: Zhexian Li
Abstract: We consider bilinear optimal control of advection-diffusion-reaction equations, where control arises as the velocity in the advection term. Existing works approached the problem by discretizing the ADR equation in space and time and solving the resulting finite-dimensional optimization. In this study, we treat the advection term as a forcing term and the equation becomes a reaction-diffusion equation. Then, we apply the Fourier transform to reformulate the PDE as infinitely many decoupled ODEs for which optimal control can be derived. The resulting optimal control is in an integral representation in the complex plane and can be numerically solved efficiently. Numerical experiments on canonical examples show that the optimality gap under our approach is smaller than existing approaches that discretize the equation.

Speakers
TK

Theodor Komann

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MB

Marcelo Bongarti

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZL

Zhexian Li

Graduate Research Assistant, University of Southern California
Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 114 3501 Trousdale Pkwy, 114, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12H: Neural Networks and Optimization
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Neural Networks and Optimization
Chair: Yiping Lu
Cluster: nan

Talk 1: Towards Quantifying the Hessian structure of Neural Networks
Speaker: Yushun Zhang
Abstract: Empirical studies reported that the Hessian of neural networks exhibits a near-block-diagonal structure, yet its theoretical foundation remains unclear. In this work, we provide a rigorous theoretical analysis of the Hessian structure of NNs at random initialization. We study linear models and 1-hidden-layer networks with the mean-square (MSE) loss and the Cross-Entropy (CE) loss for classification tasks. By leveraging random matrix theory, we compare the limit distributions of the diagonal and off-diagonal Hessian blocks and find that the block-diagonal structure arises as $C \rightarrow \infty$, where $C$ denotes the number of classes. Our findings reveal that $C$ is a primary driver of the block-diagonal structure. These results may shed new light on the Hessian structure of large language models (LLMs), which typically operate with a large $C$, often exceeding $10^4$ or $10^5$.

Talk 2: Multiscale Behavior of Gradient Descent at the Edge of Stability: Central Flow as a Boundary Layer
Speaker: Yiping Lu
Abstract: Understanding optimization in deep learning is challenging due to complex oscillatory dynamics known as the “edge of stability.” In this regime, gradient flow no longer serves as an accurate proxy for gradient descent. In this talk, we adopt a fast-slow differential equation approach to characterize both the oscillatory dynamics and the self-stabilizing behavior of gradient descent when operating at a large learning rate. Using singular perturbation theory, we describe the behavior near stationary manifolds as a boundary layer—analogous to the thin layer of fluid flowing immediately adjacent to a bounding surface. This boundary layer approximation captures the essential dynamics of gradient descent at the edge of stability.

Talk 3: Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Structured Neural Network
Speaker: Ching-pei Lee
Abstract: We propose a Regularized Adaptive Momentum Dual Averaging (RAMDA) algorithm for training neural networks with a regularization term for promoting desired structures. Similar to existing regularized adaptive methods that adopt coordinate-wise scaling, the subproblem for computing the update direction of RAMDA involves a nonsmooth regularizer and a diagonal preconditioner, and therefore does not possess a closed-form solution in general. We thus also carefully devise an implementable inexactness condition that retains convergence guarantees similar to the exact versions, and show that this proposed condition can be quickly satisfied by applying standard proximal gradient to the subproblem of RAMDA. We show asymptotic variance reduction for RAMDA, and further leverage the theory of manifold identification to prove that, even in the presence of such inexactness, after a finite number of steps, the iterates of RAMDA attain the ideal structure induced by the partly smooth regularizer at the stationary point of asymptotic convergence. This structure is locally optimal near the point of convergence, making RAMDA the first regularized adaptive method outputting models that are locally optimally structured. Extensive numerical experiments in training state-of-the-art modern neural network models in computer vision, language modeling, and speech tasks show that the proposed RAMDA is efficient and consistently outperforms state of the art for training structured neural network in terms of both the structuredness and the predictive power of the model. This is a joint work with Zih-Syuan Huang.

Speakers
YZ

Yushun Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Yiping Lu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
CL

Ching-pei Lee

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 116 3501 Trousdale Pkwy, 116, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12I: Machine Learning - Data Handling and Task Learning
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Machine Learning - Data Handling and Task Learning
Chair: Zeman Li
Cluster: nan

Talk 1: PiKE: Adaptive Data Mixing for Multi-Task Learning Under Low Gradient Conflicts
Speaker: Zeman Li
Abstract: Modern machine learning models are trained on diverse datasets and tasks to improve generalization. A key challenge in multitask learning is determining the optimal data mixing and sampling strategy across different data sources. Prior research in this multi-task learning setting has primarily focused on mitigating gradient conflicts between tasks. However, we observe that many real-world multitask learning scenarios-such as multilingual training and multi-domain learning in large foundation models-exhibit predominantly positive task interactions with minimal or no gradient conflict. Building on this insight, we introduce PiKE (Positive gradient interaction-based K-task weights Estimator), an adaptive data mixing algorithm that dynamically adjusts task contributions throughout training. PiKE optimizes task sampling to minimize overall loss, effectively leveraging positive gradient interactions with almost no additional computational overhead. We establish theoretical convergence guarantees for PiKE and demonstrate its superiority over static and non-adaptive mixing strategies. Additionally, we extend PiKE to promote fair learning across tasks, ensuring balanced progress and preventing task underrepresentation. Empirical evaluations on large-scale language model pretraining show that PiKE consistently outperforms existing heuristic and static mixing strategies, leading to faster convergence and improved downstream task performance. Li, Z., Deng, Y., Zhong, P., Razaviyayn, M., & Mirrokni, V. (2025). PiKE: Adaptive Data Mixing for Multi-Task Learning Under Low Gradient Conflicts. arXiv preprint arXiv:2502.06244.

Talk 2: Sample Reweighting for Large Models by Leveraging Weights and Losses of Smaller Models
Speaker: Mahdi Salmani
Abstract: Sample reweighting is an effective approach for mitigating the impact of noisy data by adjusting the importance of individual samples, enabling the model to focus on informative examples to improve performance. This is especially valuable for Large Language Models (LLMs), which leverage vast datasets to capture complex language patterns and drive advancements in AI applications. However, finding optimal sample weights may not be feasible due to the high cost of using conventional methods, such as those in [1], during the pre-training of larger models. One potential solution is to use weights obtained by a smaller model directly as weights for data in a larger model. However, as we will see in this talk, this may not be effective, as the optimal weight distribution for the smaller model can be too distant from that of the larger model, leading to suboptimal results. There are also papers [2] that use losses from a small model to prioritize training data. In this talk, we explore using both the weights and losses of the smaller model together as an alternative for training the larger model. References [1] Ren, M., et al. (2018). Learning to Reweight Examples for Robust Deep Learning. Proceedings of the 35th International Conference on Machine Learning, PMLR 80:4342-4350. [2] Mindermann, S., et al. (2022). Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt. Proceedings of the 39th International Conference on Machine Learning, PMLR 162:15520-15542.

Talk 3: Learning Optimal Robust Policies under Observational Data with Causal Transport
Speaker: Ruijia Zhang
Abstract: We propose a causal distributionally robust learning framework that accounts for potential distributional shifts in observational data. To hedge against uncertainty, we introduce a novel ambiguity set based on a two-stage nested transport distance, which characterizes the similarity between the empirical distribution of observational data and the true distribution of potential treatment outcomes. It penalizes deviations in covariates and treatment-specific conditional distributions while preserving the underlying causal structure. We derive a dual reformulation and establish conditions under which the robust optimization problem admits a linear programming representation, ensuring computational tractability.

Speakers
ZL

Zeman Li

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MS

Mahdi Salmani

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
RZ

Ruijia Zhang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 100 3518 Trousdale Pkwy, 100, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12J: Optimization in Power Systems and Energy
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Optimization in Power Systems and Energy
Chair: Paprapee Buason
Cluster: nan

Talk 1: Beyond Traditional Linearizations: Enhancing Power Flow Approximations with Second-Order Sensitivities
Speaker: Paprapee Buason
Abstract: The power flow equations are fundamental to power system planning, analysis, and control, but their nonlinear and nonconvex nature presents significant challenges for optimization and decision-making. While linear approximations offer computational efficiency, they often rely on broad assumptions that may not hold across diverse operating conditions, leading to inaccuracies. In particular, these methods often fail to consistently over- or under-estimate critical quantities, such as voltage magnitudes, which can result in constraint violations when applied to optimization problems. Furthermore, they do not account for the varying curvature of the power flow equations, limiting their ability to provide reliable approximations In this talk, we introduce advanced techniques to enhance the accuracy and reliability of power flow linearizations. Specifically, we focus on conservative linear approximations (CLAs), which systematically over- or under-estimate key quantities to ensure constraint satisfaction and deliver safer, more reliable solutions. Our approach leverages a sample-based framework combined with constrained linear regression while maintaining tractability and parallelizability. Additionally, we employ second-order sensitivity analysis to assess curvature and guide the selection of high-accuracy approximations. Building on these insights, we develop conservative piecewise linear approximations (CPLAs), which selectively apply piecewise linear functions in directions exhibiting significant nonlinearities, further improving accuracy beyond what standard linear approximations can achieve. Through extensive evaluations, we demonstrate that these methods enhance both accuracy and reliability, broadening their applicability to power system optimization. References: P. Buason, S. Misra, J.P. Watson, and D.K. Molzahn, "Adaptive Power Flow Approximations with Second-Order Sensitivity Insights," to appear in IEEE Transactions on Power Systems. P. Buason, S. Misra, and D.K. Molzahn, "Sample-Based Piecewise Linear Power Flow Approximations Using Second-Order Sensitivities," submitted.

Talk 2: Learning to Optimize: An Accelerated Deep Learning Framework for AC Optimal Power Flow Problem
Speaker: Yu Zhang
Abstract: The Alternating Current Optimal Power Flow (AC-OPF) problem plays a critical role in ensuring efficient and reliable power grid operations, especially with the increasing penetration of renewable energy. However, traditional solvers struggle to meet the real-time requirements due to their computational complexity. This work presents a novel semi-supervised learning framework that leverages physics-informed gradient estimation techniques to accelerate deep learning-based AC-OPF solutions. By integrating data augmentation and developing batch-mean gradient estimators with a reduced branch set, we achieve significant improvements in both feasibility and optimality. Numerical simulations on benchmark systems demonstrate that the proposed method consistently delivers near-optimal solutions with minimal constraint violations while achieving substantial speed-ups compared to conventional nonlinear programming solvers. These results highlight the potential of deep learning to transform real-time energy market operations and support the growing demand for renewable integration. Reference: K. Chen, S. Bose, Y. Zhang, "Physics-Informed Gradient Estimation for Accelerating Deep Learning based AC-OPF," IEEE Transactions on Industrial Informatics, Feb. 2025 (accepted)

Talk 3: Stationary battery energy management problem: a comparative study of several optimization models
Speaker: Daniel Mimouni
Abstract: Energy Management Systems (EMS) are crucial in optimizing energy production and minimizing costs in modern power networks. The inherent complexity of EMS problems arises from their multiperiod nature, where decisions at each stage are interplayed with outcomes of a random vector representing fluctuations in both production and consumption over time. In this paper, we focus on the EMS of a stationary battery, using ground truth measurements of electricity consumption and production from a predominantly commercial building in France. We compare several optimization models tailored to the problem for this particular EMS. Classical approaches such as MPC and risk-free multi-stage stochastic programming with recourse rely on specific assumptions (e.g. knowing the probability distribution). Therefore, they often lack robustness to distributional shifts. To enhance robustness, we explore other models. By introducing a policy variance penalty into the multi-stage stochastic model, inspired by regularization techniques in machine learning, we mitigate sensitivity to distributional shifts. Furthermore, we consider a distributionally robust optimization that offers a middle ground between robust and risk-neutral models, improving robustness by optimizing over an ambiguity set. Reinforcement learning, in contrast, offers a data-driven approach that bypasses explicit scenario generation but introduces challenges related to stability and convergence. Through numerical experiments, we evaluate these models in terms of cost efficiency, computational scalability, and out-of-sample robustness, offering a comprehensive comparison and insights into their practical interest for real-world EMS problems.

Speakers
avatar for Paprapee Buason

Paprapee Buason

Postdoctoral research associate, Los Alamos National Laboratory
Name: Dr. Paprapee BuasonTitle: Beyond Traditional Linearizations: Enhancing Power Flow Approximations with Second-Order SensitivitiesAffiliation: Los Alamos National LaboratoryBio:
avatar for Yu Zhang

Yu Zhang

Assistant Professor, UC SANTA CRUZ
Name: Dr.Yu ZhangTitle: Assistant ProfessorAffiliation: University of California, Santa CruzBio:Yu Zhang is an Assistant Professor in the ECE Department at the University of California, Santa Cruz. He earned my Ph.D. in Electrical Engineering from the University of Minnesota and pursued... Read More →
DM

Daniel Mimouni

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 102 3501 Trousdale Pkwy, 102, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12K: Decomposition Methods
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Decomposition Methods
Chair: Matthew Viens
Cluster: nan

Talk 1: A Dantzig-Wolfe Decomposition Method for Quasi-Variational Inequalities
Speaker: Manoel Jardim
Abstract: We propose an algorithm to solve quasi-variational inequality problems, based on the Dantzig-Wolfe decomposition paradigm. Our approach solves in the subproblems variational inequalities, which is a simpler problem, while restricting quasi-variational inequalities in the master subproblems, making them generally (much) smaller in size when the original problem is large-scale. We prove global convergence of our algorithm, assuming the the mapping of the quasi-variational inequality is either single-valued and continuous or it is set-vaued maximally monotone. Quasi-variational inequalities serve as a framework for several equilibrium problems, and we illustrate our algorithm on an important example in the field of economics, namely the Walrasian equilibrium problem formulated as a generalized Nash equilibrium problem. We show that the proposed method demonstrates good performance for the large-scale cases. Our numerical section tackles big problems in the theory of abstract economy, as well as some academic examples that have been previously employed in the literature.

Talk 2: Treating Uncertainty in Modeling with Multiple Solutions
Speaker: Matthew Viens
Abstract: Optimization problems can have multiple solutions that achieve an optimal or near-optimal objective value. We provide a theoretical foundation for characterizing multiple solutions combining sublevel set, epigraph, and KKT representations. We discuss how this theory enables generation of multiple solutions in two different problems: quadratic programming and Benders decomposition. We demonstrate how multiple solutions can provide additional insights into solution structure and tradeoffs for both problems. Further, we show how this additional insight is of especial value in models with data uncertainty and held-out objectives.

Talk 3: A general-purpose approach to multi-agent Bayesian optimization across decomposition methods
Speaker: Dinesh Krishnamoorthy
Abstract: Multi-agent decision-making problems, formulated as optimization problems, arise in a wide range of applications where multiple local decision-making agents collaborate to achieve a common goal. Examples include sensor and communication networks, where nodes coordinate to optimize performance and resource allocation. Distributed optimization techniques play a crucial role in these settings, enabling each agent to solve its local optimization problem while coordinating with others to achieve a system-wide optimum. Such coordination can be either decentralized (peer-to-peer) or facilitated by a central coordinator. However, traditional approaches typically require explicit analytical models linking local decision variables to objectives, which are often difficult or impractical to obtain in engineering applications. This necessitates black-box optimization methods, such as Bayesian optimization (BO), which can optimize unknown objective functions based on sampled observations. In multi-agent systems with interdependencies through shared variables or coupling constraints, standard BO methods fail to account for subsystem interactions effectively. Moreover, local objective function observations are often inaccessible to other agents, limiting the information available for updating probabilistic models and acquisition functions. Consequently, while BO has proven effective for single-agent optimization with unknown objectives, its extension to multi-agent settings remains underdeveloped. This talk will address this research gap by presenting a general-purpose multi-agent Bayesian optimization (MABO) framework that is compatible with a wide array of decomposition methods, with both centralized coordination and peer-to-peer coordination. whereby we augment traditional BO acquisition functions with suitably derived coordinating terms to facilitate coordination among subsystems without sharing local data. Regret analysis reveals that the cumulative regret of MABO is the sum of individual regrets and remains unaffected by the coordinating terms, thus bridging advancements in distributed optimization and Bayesian optimization methodologies. Numerical experiments on vehicle platooning and offshore oil production optimization examples validate the effectiveness of the proposed MABO framework for different classes of decomposition methods. This talk is based on the paper published in Optimizationa and Engineering. Krishnamoorthy, D. A general-purpose approach to multi-agent Bayesian optimization across decomposition methods. Optim Eng (2025). https://doi.org/10.1007/s11081-024-09953-w

Speakers
MJ

Manoel Jardim

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MV

Matthew Viens

PhD Student/PhD Student Intern, UW-Madison Department of Computer Sciences/Sandia National Labs
Name:Matthew ViensTitle: PhD Student/PhD Student InternAffiliation: University of Wisconsin-Madison/Sandia National LabsBio:PhD Student at University of Wisconsin-Madison in Computer Sciences advised by Michael Ferris. Also a PhD Intern for the Discrete Math & Optimization team at... Read More →
avatar for Dinesh Krishnamoorthy

Dinesh Krishnamoorthy

Associate Professor, Norwegian University of Science and Technology
Name: Dr. Dinesh KrishnamoorthyTitle: Associate Professor Affiliation: Norwegian University of Science and Technology, TrondheimBio:Dinesh Krishnamoorthy is currently an Associate Professor at the Department of Engineering Cybernetics, Norwegian University of Science and Technology... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 118 3501 Trousdale Pkwy, 118, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12L: Polynomial Optimization & Tensor Methods
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Polynomial Optimization & Tensor Methods
Chair: Yang Liu
Cluster: nan

Talk 1: On the convergence of critical points on real varieties and applications to polynomial optimization
Speaker: Ali Mohammad Nezhad
Abstract: Let $F \in \mathrm{R}[X_1,\ldots,X_n]$ and the zero set $V=\mathrm{zero}(\{P_1,\ldots,P_s\},\mathrm{R}^n)$ be given with the canonical Whitney stratification, where $\{P_1,\ldots,P_s\} \subset \mathrm{R}[X_1,\ldots,X_n]$ and $\mathrm{R}$ is a real closed field. We explore isolated trajectories that result from critical points of $F$ on $V_{\xi}=\mathrm{zero}(\{P_1-\xi_1,\ldots,P_s-\xi_s\},\mathrm{R}^n)$ when $\xi \downarrow 0$, in the sense of stratified Morse theory. Our main motivation is the limiting behavior of log-barrier functions in polynomial optimization which leads to a central path, an underlying notion behind the theory of interior point methods. We prove conditions for the existence, convergence, and smoothness of a central path. We also consider the cases where $F$ and $P_i$ are definable functions in a (polynomially bounded) o-minimal expansion of $\mathbb{R}^n$. Joint work with Saugata Basu, Purdue University

Talk 2: APPROXIMATION OF A MOMENT SEQUENCE BY MOMENT-S.o.S HIERARCHY
Speaker: Hoang Anh Tran
Abstract: The moment-S.o.S hierarchy is a widely applicable framework to address polynomial optimization problems over basic semi-algebraic sets based on positivity certificates of polynomial. Recent works show that the convergence rate of this hierarchy over certain simple sets, namely, the unit ball, hypercube, and standard simplex, is of the order $\mathrm{O}(1/r^2)$, where $r$ denotes the level of the moment-S.o.S hierarchy. This paper aims to provide a comprehensive understanding of the convergence rate of the moment-S.o.S hierarchy by estimating the Hausdorff distance between the set of pseudo truncated moment sequences and the set of truncated moment sequences specified by Tchakaloff’s theorem. Our results provide a connection between the convergence rate of the moment-S.o.S hierarchy and the \L{}ojasiewicz exponent of the domain under the compactness assumption. Consequently, we obtain the convergence rate of $\mathrm{O}(1/r)$ for polytopes, $\mathrm{O}(1/\sqrt{r})$ for domains that either satisfy the Polyak-Łojasiewicz condition or are defined by locally strongly convex polynomials, and extends the convergence rate of $\mathrm{O}(1/r^2)$ for general polynomial over the sphere.

Talk 3: Efficient Adaptive Regularized Tensor Methods
Speaker: Yang Liu
Abstract: High-order tensor methods employing local Taylor approximations have attracted considerable attention for convex and nonconvex optimization. The pth-order adaptive regularization (ARp) approach builds a local model comprising a pth-order Taylor expansion and a (p+1)th-order regularization term, delivering optimal worst-case global and local convergence rates. However, for p≥2, subproblem minimization can yield multiple local minima, and while a global minimizer is recommended for p=2, effectively identifying a suitable local minimum for p≥3 remains elusive. This work extends interpolation-based updating strategies, originally proposed for p=2, to cases where p≥3, allowing the regularization parameter to adapt in response to interpolation models. Additionally, it introduces a new prerejection mechanism to discard unfavorable subproblem minimizers before function evaluations, thus reducing computational costs for p≥3. Numerical experiments, particularly on Chebyshev-Rosenbrock problems with p=3, indicate that the proper use of different minimizers can significantly improve practical performance, offering a promising direction for designing more efficient high-order methods.

Speakers
AM

Ali Mohammad Nezhad

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
HA

Hoang Anh Tran

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
YL

Yang Liu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 119 3501 Trousdale Pkwy, 119, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12M: Games and Variational Inequalities
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Games and Variational Inequalities
Chair: Chuangyin Dang
Cluster: nan

Talk 1: A Characterization of Nash Equilibrium in Behavioral Strategies through $(\gamma,\varepsilon)$-Nash Equilibrium with Local Sequential Rationality under Kreps and Wilson's Consistency and Its Computation
Speaker: Chuangyin Dang
Abstract: This paper develops a characterization of Nash equilibrium in behavioral strategies (NashEBS) through the introduction of $(\gamma,\varepsilon)$-Nash equilibrium with local sequential rationality under Kreps and Wilson's consistency, capitalizing on an extra behavioral strategy profile. For any given $\gamma>0$, we generate a perfect $\gamma$-Nash equilibrium as a limit point of a sequence of $(\gamma,\varepsilon_k)$-Nash equilibrium with $\varepsilon_k\to 0$. We acquire a NashEBS from a limit point of a sequence of perfect $\gamma_q$-Nash equilibrium with $\gamma_q\to 0$. This characterization allows one to analytically find all NashEBSs for small extensive-form games. An application of the characterization yields a polynomial system as a necessary and sufficient condition for determining whether a totally mixed assessment is a $(\gamma,\varepsilon)$-Nash equilibrium. Exploiting the system, we devise differentiable path-following methods to compute a NashEBS by establishing the existence of smooth paths, which are secured from the equilibrium systems of logarithmic-barrier, entropy-barrier, square-root-barrier, and convex-quadratic-penalty extensive-form games. Comprehensive numerical results further confirm the efficiency of the methods.

Talk 2: A Bandit Learning Approach to Continuous Multi-Agent Games
Speaker: Jianghai Hu
Abstract: A variety of practical problems can be modeled by multi-player games where a group of self-interested players aim at optimizing their own local objectives, where the objectives depend on the (continuous) actions taken by other players. In many cases, the local gradient information of each player's objective, critical for many existing algorithms, is not available. In this talk, we will focus on designing solution algorithms for multi-player games using bandit feedback, i.e., the only available feedback for each player is the realized objective values. Our proposed solution algorithms are based on the residual feedback scheme and the extra-gradient methods, and require only one query of objective function per iteration for each player. We show that the actual sequence of play generated by the algorithms converges to the game's critical points and characterize the convergence rates for certain classes of games. We also discuss extensions when there are delays in receiving feedback and when a multi-point estimation scheme is used. Numerical examples will be given to illustrate the effectiveness of the proposed algorithms.

Talk 3: PEPit-Assisted Primal-Dual Coordinate Descent Method for Weak-MVI Problems
Speaker: Iyad WALWIL
Abstract: We introduce two novel primal-dual algorithms for tackling non-convex, non-concave, and non-smooth saddle point problems characterized by the weak Minty variational inequality (MVI). The first algorithm generalizes the well-known Primal-Dual Hybrid Gradient (PDHG) method to address this challenging problem class. The second algorithm introduces a randomly extrapolated primal-dual coordinate descent approach, extending the Stochastic Primal-Dual Hybrid Gradient (SPDHG) algorithm. Designing a coordinated algorithm to solve non-convex, non-concave saddle point problems is unprecedented, and proving its convergence posed significant difficulties. This challenge motivated us to utilize PEPit, a Python-based tool for computer-assisted worst-case analysis of first-order optimization methods. By integrating PEPit with automated Lyapunov function techniques, we successfully derived our second novel algorithm. Both methods are effective under a mild condition on the weak-MVI parameter, achieving linear convergence with constant step sizes that adapt to the problem’s structure. Numerical experiments on sigmoid and perceptron-regression problems validate our theoretical findings.

Speakers
CD

Chuangyin Dang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JH

Jianghai Hu

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
IW

Iyad WALWIL

Ph.D student, Télécom Paris
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 157 3518 Trousdale Pkwy, 157, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12N: Convergence Analysis and Rates
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Convergence Analysis and Rates
Chair: Tianyu Wang
Cluster: nan

Talk 1: Breaking a logarithmic barrier in the stopping time convergence rate of stochastic first-order methods.
Speaker: Tianyu Wang
Abstract: We focus on the convergence rate for stochastic optimization in terms of stopping times. Such problems are of great importance, since in most real-world cases, the time to stop the stochastic algorithm is a stopping time. To this end, we improve the state-of-the-art stopping-time convergence rate by removing a logarithmic factor. Our analysis leverages a novel framework building on a refined Lyapunov analysis and a new Gronwall-type argument.

Talk 2: Uniformly Optimal and Parameter-free First-order Methods for Convex and Function-constrained Optimization
Speaker: Zhenwei Lin
Abstract: This paper presents new first-order methods for achieving optimal oracle complexities in convex optimization with convex function constraints. Oracle complexities are measured by the number of function and gradient evaluations. To achieve this, we develop first-order methods that can utilize computational oracles for solving diagonal quadratic programs in subproblems. For problems where the optimal value $f^*$ is known, such as those in overparameterized models and feasibility problems, we propose an accelerated first-order method that incorporates a modified Polyak step size and Nesterov's momentum. Notably, our method does not require knowledge of smoothness levels, H\"{o}lder continuity parameter of the gradient, or additional line search, yet achieves the optimal oracle complexity bound of $\mathcal{O}(\varepsilon^{-2/(1+3\rho)})$ under H\"{o}lder smoothness conditions. When $f^*$ is unknown, we reformulate the problem as finding the root of the optimal value function and develop inexact fixed-point iteration and secant method to compute $f^*$. These root-finding subproblems are solved inexactly using first-order methods to a specified relative accuracy. We employ the accelerated prox-level (APL) method, which is proven to be uniformly optimal for convex optimization with simple constraints. Our analysis demonstrates that APL-based root-finding also achieves the optimal oracle complexity of $\mathcal{O}(\varepsilon^{-2/(1+3\rho)})$ for convex function-constrained optimization, without requiring knowledge of any problem-specific structures. Through experiments on various tasks, we demonstrate the advantages of our methods over existing approaches in function-constrained optimization.

Talk 3: Stochastic Halpern iteration in normed spaces and applications to reinforcement learning
Speaker: Mario Bravo
Abstract: We analyze the oracle complexity of the stochastic Halpern iteration with minibatching, where we aim to approximate fixed-points of nonexpansive operators in a normed finite-dimensional space. We show that if the underlying stochastic oracle is with uniformly bounded variance, our method exhibits an overall oracle complexity of $\tilde{O}(\varepsilon^{-5})$ to reduce the expected error residual below $\varepsilon$. Also, we establish a lower bound of $\Omega(\varepsilon^{-3})$, applicable to a broad class of algorithms, including all averaged iterations, even with minibatching.  As an application, we introduce a new model-free algorithm for weakly communicating average-reward MDPs, requiring no prior knowledge, and analyze its sample complexity. This is joint work with Juan Pablo Contreras (UDP, Chile)

Speakers
TW

Tianyu Wang

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZL

Zhenwei Lin

PhD candidate, Shanghai University of Finance and Economics
Name: Zhenwei LinTitle:Accelerated Bundle Level Method for Convex OptimizationAffiliation: Shanghai University of Finance and Economics and Purdue UniversityBio:This paper presents new first-order methods for achieving optimal oracle complexities in convex optimization with convex... Read More →
MB

Mario Bravo

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 256 3518 Trousdale Pkwy, 256, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12O: Numerical Methods
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Numerical Methods
Chair: Masaru Ito
Cluster: nan

Talk 1: Proximal gradient-type method with generalized distances for nonconvex composite optimization
Speaker: Masaru Ito
Abstract: In this talk, we consider a composite optimizaton problem minimizing the sum of two functions f and g. Typical proximal gradient methods rely on descent lemma and the convexity of g, for which the choice of distance-like functions to define the proximal subproblems is constrained by the structure of both f and g. We propose a proximal gradient-type method when f has locally Lipschitz gradient and g is nonconvex. We discuss conditions of distance-like functions allowing their broader choices and ensuring convergence results.

Talk 2: A fixed-point algorithm with matrix splitting for nonlinear second-order cone complementarity problems
Speaker: Shunsuke Hayashi
Abstract: The Second-Order Cone Complementarity Problem (SOCCP) is a wide class of problems containing the nonlinear complementarity problem (NCP) and the second-order cone programming problem (SOCP). Recently, Li et al. reformulated the linear SOCCP into a fixed-point problem by using matrix splitting, and constructed the Anderson-accelerated preconditioned modulus approach. In this study, we extend their approach to nonlinear SOCCPs. To solve such problems, we combine a matrix splitting with a fixed-point algorithm. We also present an approach with Anderson acceleration to enhance the convergence performance. We further show the convergence property under appropriate assumptions. Finally, we report some numerical results to evaluate the effectiveness of the algorithm.

Talk 3: A Simple yet Highly Accurate Prediction-Correction Algorithm for Time-Varying Optimization
Speaker: Tomoya Kamijima
Abstract: Time-varying optimization problems arise in various applications such as robotics, signal processing, and electronics. We propose SHARP, a simple yet highly accurate prediction-correction algorithm for unconstrained time-varying problems. The prediction step is based on Lagrange interpolation of past solutions, allowing for low computational cost without requiring Hessian matrices or gradients. To enhance stability, especially in non-convex settings, an acceptance condition is introduced to reject excessively large updates. We provide theoretical guarantees for a small tracking error and demonstrate the superior performance of SHARP through numerical experiments.

Speakers
MI

Masaru Ito

Associate Professor, Nihon University
SH

Shunsuke Hayashi

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Tomoya Kamijima

Tomoya Kamijima

Doctor student, The University of Tokyo
Name: Tomoya KamijimaAffiliation: The University of TokyoBio:I completed a master's degree at the University of Tokyo in 2025.Now I am pursuing a Ph.D.I am interested in continuous optimization, especially,time-varying optimization,online optimization,ODE approach.
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 258 3518 Trousdale Pkwy, 258, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12P: Learning and Optimization Interaction
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Learning and Optimization Interaction
Chair: Christopher Yeh
Cluster: nan

Talk 1: Learning Decision-Focused Uncertainty Representations for Robust and Risk-Constrained Optimization
Speaker: Christopher Yeh
Abstract: Machine learning can significantly improve performance for decision-making under uncertainty in a wide range of domains. However, ensuring robustness guarantees and satisfaction of risk constraints requires well-calibrated uncertainty estimates, which can be difficult to achieve with neural networks. Moreover, in high-dimensional settings, there may be many valid uncertainty estimates, each with their own performance profile—i.e., not all uncertainty is equally valuable for downstream decision-making. To address this problem, we developed an end-to-end framework to learn uncertainty representations for robust and risk-constrained optimization in a way that is informed by the downstream decision-making loss, with robustness guarantees and risk constraint satisfaction provided by conformal methods. In addition, we propose to represent arbitrary convex uncertainty sets with partially input-convex neural networks, which are learned as part of our framework. Our approach consistently improves upon two-stage estimate-then-optimize baselines on concrete applications in energy storage arbitrage and portfolio optimization.

Talk 2: Learning Algorithm Hyperparameters for Fast Parametric Convex Optimization
Speaker: Rajiv Sambharya
Abstract: We introduce a machine-learning framework to learn the hyperparameter sequence of first-order methods (e.g., the step sizes in gradient descent) to quickly solve parametric convex optimization problems. Our computational architecture amounts to running fixed-point iterations where the hyperparameters are the same across all parametric instances and consists of two phases. In the first step-varying phase the hyperparameters vary across iterations, while in the second steady-state phase the hyperparameters are constant across iterations. Our learned optimizer is flexible in that it can be evaluated on any number of iterations and is guaranteed to converge to an optimal solution. To train, we minimize the mean square error to a ground truth solution. In the case of gradient descent, the one-step optimal step size is the solution to a least squares problem, and in the case of unconstrained quadratic minimization, we can compute the two and three-step optimal solutions in closed-form. In other cases, we backpropagate through the algorithm steps to minimize the training objective after a given number of steps. We show how to learn hyperparameters for several popular algorithms: gradient descent, proximal gradient descent, and two ADMM-based solvers: OSQP and SCS. We use a sample convergence bound to obtain generalization guarantees for the performance of our learned algorithm for unseen data, providing both lower and upper bounds. We showcase the effectiveness of our method with many examples, including ones from control, signal processing, and machine learning. Remarkably, our approach is highly data-efficient in that we only use 10 problem instances to train the hyperparameters in all of our examples. [https://arxiv.org/pdf/2411.15717]

Talk 3: An Operator Learning Approach to Nonsmooth Optimal Control of Nonlinear Partial Differential Equations
Speaker: Tianyou Zeng
Abstract: Optimal control problems with nonsmooth objectives and nonlinear PDE constraints pose significant numerical challenges to traditional numerical methods, due to their nonconvexity, nonsmoothness, and expensive computational cost of iteratively solving high-dimensional and ill-conditioned systems introduced by mesh-based discretization. We present an operator learning approach for these problems. We implement a primal-dual idea in the optimization context and solve the resulting PDEs with pre-trained neural solution operators. Compared with traditional algorithms and existing deep learning methods, our approach avoids re-solving linear systems or retraining networks across iterations. Additionally, the pre-trained neural networks can be readily applied without retraining even for different problem parameters, like the desired state or the PDE source term. We demonstrate the effectiveness and efficiency of the proposed method through validation on some benchmark nonsmooth optimal control problems with nonlinear PDE constraints.

Speakers
avatar for Christopher Yeh

Christopher Yeh

PhD candidate (on the faculty job market), California Institute of Technology
Chris Yeh is a 5th-year PhD candidate in the Computing and Mathematical Science Department at Caltech, co-advised by Professors Yisong Yue and Adam Wierman. His research focus is on the problem of co-designing uncertainty quantification methods with decision-making algorithms, especially... Read More →
RS

Rajiv Sambharya

Postdoc, University of Pennsylvania
Name: Rajiv SambharyaTitle: PostdocAffiliation: University of PennsylvaniaBio:I am a Postdoctoral Researcher in the Electrical and Systems Engineering department at Penn hosted by George Pappas. I obtained my PhD from the Operations Research and Financial Engineering department a... Read More →
TZ

Tianyou Zeng

PhD Candidate, The University of Hong Kong
Name: Tianyou ZengTitle: PhD CandidateAffiliation: The University of Hong KongBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing in algorithms that eventuallyconverge—if given infinite time. His groundbreaking research in gradient descent... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 106 3501 Trousdale Pkwy, 106, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12Q: Cloud Computing, Transport, and AI
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Cloud Computing, Transport, and AI
Chair: Julius Lohmann
Cluster: nan

Talk 1: An ϵ outer linear approximation optimization model for geo-distributed cloud applications
Speaker: Julio C Góez
Abstract: Cloud computing has become key for software providers aiming to serve a geographically distributed user base. As application demand changes and users become more latency sensitive, software providers need to adapt their deployments to reach latency-based service-level objectives while keeping costs low. In this paper, we propose a method to provision Cloud resources considering latency-based service-level constraints while minimizing operating costs. Our method is based on a latency model to capture the load balancing among resources. To solve the resulting mixed integer non-linear optimization model, we propose an outer linear approximation that is able to find feasible solutions faster than solving the non-linear problem. Experiments based on realistic deployment data reveal how the proposed method is able to deliver timely solutions to the provisioning problem. Further, the solutions adapt to key elements of the problem, such as the service-level objective defined and the characteristics of the software application deployed.

Talk 2: Integral representation of the h-mass
Speaker: Julius Lohmann
Abstract: The multi-material transport problem [1, 2] is convex optimization problem on normal 1-currents in ℝⁿ with coefficients in ℝᵐ. The prescribed boundary can be written μ⃗₋−μ⃗₊, where μ⃗₊ and μ⃗₋ are compactly supported ℝᵐ-valued Radon measures whose components μ⃗±ʲ are nonnegative and satisfy μ⃗₊ʲ(ℝⁿ) = μ⃗₋ʲ(ℝⁿ). For each j the measure μ⃗₊ʲ can be interpreted as the source distribution of some material j which has to be transported to the sink μ⃗₋ʲ. The objective in the above Plateau problem is the so-called h-mass ℳₕ with norm h on the coefficient group ℝᵐ. The value h(θ⃗) represents the transportation cost to move a material bundle θ⃗ per unit distance. The triangle inequality h(θ⃗+θ⃗) ≤ h(θ⃗)+h(θ⃗) implies that the joint transport of different materials may be more efficient. The h-mass ℳₕ(F) then indicates the total transportation cost of an admissible candidate F, that is ∂F = μ⃗₋−μ⃗₊, with respect to h (also called mass flux in this context). A candidate mass flux F for the Plateau problem inf ℳₕ(G), subject to ∂G=μ⃗₋−μ⃗₊, can alternatively be interpreted as a finite mass flat 1-chain with coefficients in ℝᵐ and boundary μ⃗₋−μ⃗₊ or as a matrix-valued Radon measure (row j being a mass flux describing the transport of material j) whose distributional divergence equals μ⃗₊−μ⃗₋. Each interpretation of F, as a current, flat chain, or measure, comes with its own natural notion of an h-mass ℳₕ, 𝕄ₕ, or |·|ₕ. In [3] it is shown that all these notions are equivalent. In particular, this result yields an integral representation of the h-mass which, so far, was only known for rectifiable currents [2, 4]. In addition, the proof motivates a new definition of so-called calibrations for multi-material transport. Calibrations are a classical tool in the study of Plateau problems. They can be used to certify optimality of candidate minimizers. The new definition of a calibration in [3] provides a sufficient and necessary criterion for optimality (opposed to just sufficient as in the classical case). Since, in addition, this notion ‘includes’ the classical definition, a sharp characterization of the regularity of calibrations for multi- material transport is obtained. In my talk, I will motivate the study of multi-material transport, explain the different h-masses, sketch the argument yielding their equality, and give examples of calibrations. [1] A. Marchese, A. Massaccesi, R. Tione. A multi-material transport problem and its convex relaxation via rectifiable G-currents. SIAM J. Math. Anal., 51(3):1965–1998, 2019. [2] A. Marchese, A. Massaccesi, S. Stuvard, R. Tione. A multi-material transport problem with arbitrary marginals. Calc. Var. Partial Differential Equations, 60(3):Paper No. 88, 2021. [3] J. Lohmann, B. Schmitzer, B. Wirth. Formulas for the h-mass on 1-currents with coefficients in Rm. Preprint: arXiv:2407.10158 [math.OC], 2024. [4] B. White. Rectifiability of flat chains. Ann. of Math., 150(1):165–184, 1999.

Talk 3: Detecting AI Generated Images through Texture and Frequency Analysis of Patches
Speaker: Maryam Yashtini
Abstract: The significant improvement in AI image generation in recent years poses serious threats to social security, as AI generated misinformation may infringe upon political stability, personal privacy, and digital copy rights of artists. Building an AI generated image detector that accurately identifies generated image is crucial to maintain the social security and property rights of artists. This paper introduces preprocessing pipeline that uses positional encoded azimuthal integrals for image patches to create fingerprints that encapsulate distinguishing features. We then trained a multi-head attention model with 97.5% accuracy on classification of the fingerprints. The model also achieved 80% accuracy on images generated by AI models not presented in the training dataset, demonstrating the robustness of our pipeline and the potential of broader application of our model.

Speakers
JC

Julio C Góez

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
JL

Julius Lohmann

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
MY

Maryam Yashtini

Tenure Track Professor, Georgetown University
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 214 3501 Trousdale Pkwy, 214, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12R: Envelopes in Optimization
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Envelopes in Optimization
Chair: Yankun Huang
Cluster: nan

Talk 1: Majorization Minorization, Moreau Envelopes, and Deweighting Weighted Least Squares
Speaker: Qiang Heng
Abstract: This paper deals with tactics for fast computation in least squares regression in high dimensions. These tactics include: (a) the majorization-minimization (MM) principle, (b) smoothing by Moreau envelopes, and (c) the proximal distance principal for constrained estimation. In iteratively reweighted least squares, the MM principle can create a surrogate function that trades case weights for adjusted responses. Reduction to ordinary least squares then permits the reuse of the Gram matrix and its Cholesky decomposition across iterations. This tactic is pertinent to estimation in L2E regression and generalized linear models. For problems such as quantile regression, non-smooth terms of an objective function can be replaced by their Moreau envelope approximations and majorized by spherical quadratics. Finally, penalized regression with distance-to-set penalties also benefits from this perspective. Our numerical experiments validate the speed and utility of deweighting and Moreau envelope approximations. Julia software implementing these experiments is available on our web page.

Talk 2: Computing the convex envelope of bivariate piecewise linear-quadratic (PLQ) functions
Speaker: Tanmaya Karmarkar
Abstract: We introduce a linear-time algorithm for computing the convex envelope of bivariate piecewise linear-quadratic (PLQ) functions and establish that the biconjugate is piecewise rational defined over a polyhedral subdivision. Our approach consists of the following steps: (1) compute the convex envelope of each quadratic piece obtaining piecewise rational functions (quadratic divided by linear function) defined over a polyhedral subdivision; (2) compute the conjugate of each resulting piece to obtain piecewise quadratic functions defined over a parabolic subdivision; (3) compute the maximum of all those functions to obtain the conjugate of the original PLQ function as a piecewise quadratic function defined on a parabolic subdivision; (4) compute the conjugate of each resulting piece; and finally (5) compute the maximum over all those functions to obtain the biconjugate as rational functions (quadratic divided by linear function) defined over a polyhedral subdivision.

Talk 3: Inexact Moreau Envelope Lagrangian Method for Non-Convex Constrained Optimization under Local Error Bound Conditions on Constraint Functions
Speaker: Yankun Huang
Abstract: In this paper, we study the inexact Moreau envelope Lagrangian (iMELa) method for solving smooth non-convex optimization problems over a simple polytope with additional convex inequality constraints. By incorporating a proximal term into the traditional Lagrangian function, the iMELa method approximately solves a convex optimization subproblem over the polyhedral set at each main iteration. Under the assumption of a local error bound condition for subsets of the feasible set defined by subsets of the constraints, we establish that the iMELa method can find an $\epsilon$-Karush-Kuhn-Tucker point with $\tilde O(\epsilon^{-2})$ gradient oracle complexity. Paper preprint link: https://arxiv.org/abs/2502.19764.

Speakers
QH

Qiang Heng

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
TK

Tanmaya Karmarkar

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
avatar for Yankun Huang

Yankun Huang

Postdoctoral Scholar, Arizona State University
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 154 3518 Trousdale Pkwy, 154, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12S: Zeroth-Order and Derivative-Free Optimization
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Zeroth-Order and Derivative-Free Optimization
Chair: Anjie Ding
Cluster: nan

Talk 1: Sequentially testing a decrease condition in stochastic derivative-free optimization
Speaker: Anjie Ding
Abstract: In derivative-free optimization algorithms, the progress made by each trial step is measured by the potential decrease attained in the objective function, and in many algorithms one requires a decrease that must be sufficiently large when compared to a forcing function of the size of the step, typically a multiple of the square of the step size. When the function is stochastic, the estimation of the decrease needed to ensure the desired rate of convergence of the algorithms requires a sample size of the order of 4 of the inverse of the step size. In this talk, we introduce a new way of testing the satisfaction of a sufficient decrease condition in stochastic derivative-free optimization by framing it as a hypothesis test problem and solving it through the means of a sequential hypothesis test. The test makes a decision between two hypotheses, essentially corresponding to accepting or rejecting the sufficient decrease, outputting the probabilities of making these decisions incorrectly. For the purpose of establishing a standard (non-convex) convergence rate, such probabilities need to satisfy certain upper bounds to ensure enough correctness when the step size approaches zero. When the function noise is Gaussian, we will show that the size of the sample required to estimate the decrease drops to an order of 2 of the inverse of the step size when the decrease is far from the forcing function and the step size is not yet close to zero. In this case, this test sequentially estimates the potential decrease by observing the function (at the current and trial points) until their sum crosses either a lower or an upper bound selectively chosen as a function of the variation and the step size.

Talk 2: Which is faster: Linear minimization or Projection?
Speaker: Zev Woodstock
Abstract: It is already known that, for several important sets arising in applications, performing linear minimization can be faster than projection. However, if we consider the class of all nonempty compact convex sets, can we directly compare the computational complexity of linear minimization to that of projection? This talk provides two modest results in this direction: (1) high-precision linear minimization is no slower than projection; and (2) Exact linear minimization is no slower than projection under the additional assumption of polyhedrality.

Speakers
AD

Anjie Ding

PhD Student, Lehigh University
Name: Dr. Slothington "Slow Convergence" McNapfaceTitle: Distinguished Professor of Continuous Optimization & Energy MinimizationAffiliation: The Lush Canopy Institute of Sluggish AlgorithmsBio:Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
ZW

Zev Woodstock

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 155 3518 Trousdale Pkwy, 155, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12T: Distributional Shift and Systemic Risk
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Distributional Shift and Systemic Risk
Chair: Lai Tian
Cluster: nan

Talk 1: Sample-average approximations for systemic risk measures
Speaker: Çağın Ararat
Abstract: We investigate the convergence properties of sample-average approximations (SAA) for set-valued systemic risk measures. We assume that the systemic risk measure is defined using a general aggregation function with some continuity properties and value-at-risk applied as a monetary risk measure. Our focus is on the theoretical convergence of its SAA under Wijsman and Hausdorff topologies for closed sets. After building the general theory, we provide an in-depth study of an important special case where the aggregation function is defined based on the Eisenberg-Noe network model. In this case, we provide mixed-integer programming formulations for calculating the SAA sets via their weighted-sum and norm-minimizing scalarizations. When value-at-risk is replaced with expectation, we also provide lower bounds on the required sample size for a sufficiently close SAA with high probability. Based on joint works with Wissam AlAli and Nurtai Meimanjan.

Talk 2: Mixed-feature Logistic Regression Robust to Distribution Shifts
Speaker: Qingshi Sun
Abstract: Logistic regression models are widely used in the social and behavioral sciences and in high-stakes domains, due to their simplicity and interpretability properties. At the same time, such domains are permeated by distribution shifts, where the distribution generating the data changes between training and deployment. In this paper, we study a distributionally robust logistic regression problem that seeks the model that will perform best against adversarial realizations of the data distribution drawn from a suitably constructed Wasserstein ambiguity set. Our model and solution approach differ from prior work in that we can capture settings where the likelihood of distribution shifts can vary across features, significantly broadening the applicability of our model relative to the state-of-the-art. We propose a graph-based solution approach that can be integrated into off-the-shelf optimization solvers. We evaluate the performance of our model and algorithms on numerous publicly available datasets. Our solution achieves a 408x speed-up relative to the state-of-the-art. Additionally, compared to the state-of-the-art, our model reduces average calibration error by up to 36.19% and worst-case calibration error by up to 41.70%, while increasing the average area under the ROC curve (AUC) by up to 18.02% and worst-case AUC by up to 48.37%.

Talk 3: Stabilizing Stochastic Programs with Rockafellian Relaxation: Theoretical Results and Chance-Constrained Applications
Speaker: Lai Tian
Abstract: Solutions of a stochastic optimization problem tend to change disproportionally under small changes to its probability distribution. This sensitivity is particularly concerning, as it is virtually impossible to identify the “correct” distribution in real-world applications. In this talk, we demonstrate how Rockafellian relaxations provide a principled and effective approach to improving the stability of solutions under distributional changes. Unlike previous efforts that primarily focus on finite or discrete distributions, our framework accommodates general Borel probability measures and handles discontinuous integrands, such as those arising in chance-constrained formulations. These advances broaden the scope of robust solution techniques in modern stochastic optimization.

Speakers
avatar for Çağın Ararat

Çağın Ararat

Name: Çağın AraratAffiliation: Bilkent University, Ankara, TurkeyBio:Çağın Ararat is an Associate Professor in the Department of Industrial Engineering at Bilkent University. He received his BSc degree in 2010 from the same department, followed by a PhD degree in 2015 from the... Read More →
QS

Qingshi Sun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 158 3518 Trousdale Pkwy, 158, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12U: Distributional Shift and Systemic Risk
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Session: Distributional Shift and Systemic Risk
Chair: Lai Tian
Cluster: nan

Talk 1: Sample-average approximations for systemic risk measures
Speaker: Çağın Ararat
Abstract: We investigate the convergence properties of sample-average approximations (SAA) for set-valued systemic risk measures. We assume that the systemic risk measure is defined using a general aggregation function with some continuity properties and value-at-risk applied as a monetary risk measure. Our focus is on the theoretical convergence of its SAA under Wijsman and Hausdorff topologies for closed sets. After building the general theory, we provide an in-depth study of an important special case where the aggregation function is defined based on the Eisenberg-Noe network model. In this case, we provide mixed-integer programming formulations for calculating the SAA sets via their weighted-sum and norm-minimizing scalarizations. When value-at-risk is replaced with expectation, we also provide lower bounds on the required sample size for a sufficiently close SAA with high probability. Based on joint works with Wissam AlAli and Nurtai Meimanjan.

Talk 2: Mixed-feature Logistic Regression Robust to Distribution Shifts
Speaker: Qingshi Sun
Abstract: Logistic regression models are widely used in the social and behavioral sciences and in high-stakes domains, due to their simplicity and interpretability properties. At the same time, such domains are permeated by distribution shifts, where the distribution generating the data changes between training and deployment. In this paper, we study a distributionally robust logistic regression problem that seeks the model that will perform best against adversarial realizations of the data distribution drawn from a suitably constructed Wasserstein ambiguity set. Our model and solution approach differ from prior work in that we can capture settings where the likelihood of distribution shifts can vary across features, significantly broadening the applicability of our model relative to the state-of-the-art. We propose a graph-based solution approach that can be integrated into off-the-shelf optimization solvers. We evaluate the performance of our model and algorithms on numerous publicly available datasets. Our solution achieves a 408x speed-up relative to the state-of-the-art. Additionally, compared to the state-of-the-art, our model reduces average calibration error by up to 36.19% and worst-case calibration error by up to 41.70%, while increasing the average area under the ROC curve (AUC) by up to 18.02% and worst-case AUC by up to 48.37%.

Talk 3: Stabilizing Stochastic Programs with Rockafellian Relaxation: Theoretical Results and Chance-Constrained Applications 
Speaker: Lai Tian
Abstract: Solutions of a stochastic optimization problem tend to change disproportionally under small changes to its probability distribution. This sensitivity is particularly concerning, as it is virtually impossible to identify the “correct” distribution in real-world applications. In this talk, we demonstrate how Rockafellian relaxations provide a principled and effective approach to improving the stability of solutions under distributional changes. Unlike previous efforts that primarily focus on finite or discrete distributions, our framework accommodates general Borel probability measures and handles discontinuous integrands, such as those arising in chance-constrained formulations. These advances broaden the scope of robust solution techniques in modern stochastic optimization.

Speakers
avatar for Çağın Ararat

Çağın Ararat

Name: Çağın AraratAffiliation: Bilkent University, Ankara, TurkeyBio:Çağın Ararat is an Associate Professor in the Department of Industrial Engineering at Bilkent University. He received his BSc degree in 2010 from the same department, followed by a PhD degree in 2015 from the... Read More →
QS

Qingshi Sun

Name: Dr. Slothington "Slow Convergence" McNapface Title: Distinguished Professor of Continuous Optimization & Energy Minimization Affiliation: The Lush Canopy Institute of Sluggish Algorithms Bio: Dr. Slothington McNapface is a leading expert in continuous optimization, specializing... Read More →
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 108 3501 Trousdale Pkwy, 108, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12V
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 110 3501 Trousdale Pkwy, 110, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12W
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 112 3501 Trousdale Pkwy, 112, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12X
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Taper Hall (THH) 215 3501 Trousdale Pkwy, 215, Los Angeles, CA 90089

4:15pm PDT

Parallel Sessions 12Y
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Thursday July 24, 2025 4:15pm - 5:30pm PDT
Joseph Medicine Crow Center for International and Public Affairs (DMC) 200 3518 Trousdale Pkwy, 200, Los Angeles, CA 90089

5:30pm PDT

End of Conference
Thursday July 24, 2025 5:30pm - 5:30pm PDT
Thursday July 24, 2025 5:30pm - 5:30pm PDT
TBA
 
  • Filter By Date
  • Filter By Venue
  • Filter By Type
  • Timezone


Filter sessions
Apply filters to sessions.