The incremental aggregated Douglas-Rachford splitting method for large-scale optimization problem

Abstract

This paper concentrates on the large-scale composite optimization problems, whose objective function is the aggregation of a finite number of nonconvex smooth functions and a nonconvex nonsmooth regularization term. While the existing proximal-like incremental aggregated gradient (PLIAG) algorithm performs well in solving large-scale optimization problems, it requires computing the proximal operators of the sum of two functions, which typically results in the absence of a closed-form solution or even hard to solve. Motivated by the Douglas-Rachford splitting method, we propose a implementing proximal incremental aggregated Douglas-Rachford splitting method, named as the Prox2-IAG method, which computes the proximal operators of the two functions respectively. The Prox2-IAG method includes many existing algorithms as special cases, such as the Douglas-Rachford method, the incremental aggregated gradient method and the incremental aggregated proximal method. Under the Kurdyka-Łojasiewicz property, we present the global convergence of the sequence generated by the Prox2-IAG method. Additionally, the convergence rate is established under the KL framework. Experiments are conducted on sparse logistic regression problem to demonstrate the efficiency of the Prox2-IAG algorithm.