Block-wise Alternating Direction Method of Multipliers for Multiple-block Convex Programming and Beyond

Bingsheng He, Xiaoming Yuan

Research output: Contribution to journalJournal articlepeer-review

30 Citations (Scopus)

Abstract

The alternating direction method of multipliers (ADMM) is a benchmark for solving a linearly constrained convex minimization model with a two-block separable objective function; and it has been shown that its direct extension to a multiple-block case where the objective function is the sum of more than two functions is not necessarily convergent. For the multiple-block case, a natural idea is to artificially group the objective functions and the corresponding variables as two groups and then apply the original ADMM directly —- the block-wise ADMM is accordingly named because each of the resulting ADMM subproblems may involve more than one function in its objective. Such a subproblem of the block-wise ADMM may not be easy as it may require minimizing more than one function with coupled variables simultaneously. We discuss how to further decompose the block-wise ADMM’s subproblems and obtain easier subproblems so that the properties of each function in the objective can be individually and thus effectively used, while the convergence can still be ensured. The generalized ADMM and the strictly contractive Peaceman-Rachford splitting method, two schemes closely relevant to the ADMM, will also be extended to the block-wise versions to tackle the multiple-block convex programming cases. We present the convergence analysis, including both the global convergence and the worst-case convergence rate measured by the iteration complexity, for these three block-wise splitting schemes in a unified framework.

Original languageEnglish
Pages (from-to)145-174
Number of pages30
JournalSMAI Journal of Computational Mathematics
Volume1
DOIs
Publication statusPublished - Jan 2015

Scopus Subject Areas

  • Statistics and Probability
  • Numerical Analysis
  • Modelling and Simulation
  • Computational Mathematics

User-Defined Keywords

  • Alternating direction method of multipliers
  • Convergence rate
  • Convex programming
  • Douglas-Rachford splitting method
  • Iteration complexity
  • Operator splitting methods
  • Peaceman-Rachford splitting method
  • proximal point algorithm

Fingerprint

Dive into the research topics of 'Block-wise Alternating Direction Method of Multipliers for Multiple-block Convex Programming and Beyond'. Together they form a unique fingerprint.

Cite this