Session: First-order methods for nonsmooth and constrained optimization - II
Chair: Zhe (Jimmy) Zhang
Cluster: Nonlinear Optimization
Talk 1: A Single-Loop Spider-Type Stochastic Subgradient Method for Nonconvex Nonsmooth Expectation Constrained Optimization
Speaker: Wei Liu
Abstract: Many real-world problems involve complex nonconvex functional constraints and large datasets, necessitating efficient stochastic methods for solving stochastic optimization problems. A majority of existing works assume no constraints or easy-to-project constraints. In this paper, we consider nonconvex stochastic optimization problems with nonconvex expectation constraints. We construct an unconstrained exact penalty model with a new penalized function for the expectation constraints that share the same stationary points as the original problem. To solve this problem, we present a single-loop spider-type stochastic subgradient first-order method, which utilizes the derivatives of f and g and the function value of g at each iteration. Under certain regularity conditions (weaker than Slater-type constraint qualification in existing works), we establish an oracle complexity result of O(ϵ−4) to reach a point that is ϵ\epsilonϵ-close to an ϵ\epsilonϵ-KKT point of the original problem in expectation, matching the lower bound for such tasks. As important applications, we apply our method to two fairness-constrained problems, demonstrating that it is at least 9 times faster than state-of-the-art algorithms, including switching subgradient methods and inexact proximal point methods.
Talk 2: Monotone Variational Inequality problem under Generalized Conditions
Speaker: Digvijay Boob
Abstract: We consider the well-known variational inequality (VI) problem on monotone operators under a novel Lipschitz type condition and possibly unbounded feasible set. This new class of problems cover various class of problems, including convex function constrained problem, convex semi-infinite constrained problem, general robust optimization problem, and function-constrained variational inequalities among others - covering wide class of problems as a specific case. We show that when the problem is generalized smooth, our method converges at the rate of O(1/K) in terms of Minty-gap criterion appropriately defined for unbounded sets. For strongly monotone generalized smooth problems, we show linear convergence. For generalized nonsmooth and stochastic VI problems, we show convergence at the rate of O(1/\sqrt{K}). To our best knowledge, this is the first time such problems are addressed, especially in the context of variational inequality problems.
Talk 3: Stochastic Block-Wise Iterative Methods for Training Feasibility-Based Neural Architectures
Speaker: Manish Krishan
Abstract: We present a unifying framework for finitely convergent stochastic block-wise iterative methods, tailored for training set-feasibility-based neural network architectures. We apply this disciplined approach to selecting both the architecture and its corresponding training algorithm for classification and regression tasks.
