A Sufficient Descent Dai-Liao Type Conjugate Gradient Update Parameter
Abstract
In recent years, conjugate gradient methods have gained popularity as efficient iterative techniques for unconstrained optimization problems without the need for matrix storage. Based on the Dai-Laio conjugacy condition, this article presents a new hybrid conjugate gradient method that combines features of the Dai-Yuan and Dai-Laio methods. The proposed method addresses the numerical instability and slow convergence of the Dai-Yuan method as well as the potential poor performance of the Dai-Laio method in highly non-linear optimization problems. The hybrid method solves optimization problems with faster convergence rates and greater stability by combining the advantages of both methods. The resulting algorithm is shown to be more effective and reliable, and theoretical analysis reveals that it has sufficient descent properties. The proposed method's competitive performance is shown through a number of benchmark tests and comparisons with other approaches, indicating that it has the potential to be an effective approach for complex, unconstrained optimization.
References
D.C. Luenberger, Linear and Nonlinear Programming, 2nd ed., Addition-Wesley, Reading MA, 1989.
O.L. Mangasarian, Nonlinear Programming, McGraw-Hill, New-York, 1969.
M.R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand. 49 (1952), 409-436. https://doi.org/10.6028/jres.049.044
R. Fletcher and C.M. Reeves, Function minimization by conjugate gradients, Comput. J. 7 (1964), 149-154. https://doi.org/10.1093/COMJNL/7.2.149
E. Polak and G. Ribiere, Note Sur la Convergence de directions conjugees, ESAIM: Math. Model. Numer. Anal. 3 (1969), 35-43. https://doi.org/10.1051/M2AN/196903R100351
B.T. Polyak, The conjugate gradient method in extreme problems, Comput. Math. Math. Phys. 9 (1969), 94-112. https://doi.org/10.1016/0041-5553(69)90035-4
R. Fletcher, Practical Method of Optimization, 2nd ed., John Wiley, New York, 1987. https://doi.org/10.1002/9781118723203
Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms part 1, theory, J. Optim. Theory Appl. 69 (1991), 322-340. https://doi.org/10.1007/BF00940464
W.W. Hager, H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim. 16 (1) (2005), 170-192. https://doi.org/10.1137/030601880
P. Wolfe, Convergence conditions for ascent methods, SIAM J. Optim. (1969), 226-235. https://doi.org/10.1137/1011036
A.A. Goldstein, On steepest descent, J. Soc. Ind. Appl. Math. Ser. A Control 3 (1) (1965), 147-151.
L. Armijo, Minimization of function having Lipschitz continuous first partial derivative, Pacific J. Math. 16 (1966), 1-3. https://doi.org/10.2140/pjm.1966.16.1
X. Xu, F. Kong, New hybrid conjugate gradient method with the generalized wolfe line search, SpringerPlus 5 (2016), 1-10. https://doi.org/10.1186/s40064-016-2522-9
S.S. Djordjevic, New hybrid conjugate gradient method as a convex combination of LS and CD methods, Filomat 31 (6) (2017), 1813-1825. https://doi.org/10.2298/fil1706813d
X. Dong, D. Han, R. Ghanbari, X. Li, Z. Dai, Some new three-term Hestenes-Stiefel conjugate gradient methods with affine combination, Optim. 66 (5) (2017), 1-18. https://doi.org/10.1080/02331934.2017.1295242
I.A. Osinuga, I.O. Olofin, Extended hybrid conjugate gradient method for unconstrained optimization, J. Comput. Sci. Appl. 25 (1) (2018), 25-33.
S.S. Djordjevic, New hybrid conjugate gradient method as a convex combination of LS and FR methods, Acta Math. Sci. 39 (1) (2019), 214-228. https://doi.org/10.1007/s10473-019-0117-6
N. Salihu, M.R. Odekunle, A.M. Saleh, S. Salihu, A Dai-Liao hybrid Hestenes-Stiefel and Fletcher-Reeves methods for unconstrained optimization, Int. J. Ind. Optim. 2 (1) (2021), 33-50. https://doi.org/10.12928/IJIO.V2I1.3054
M. Rivaie, M. Mamat, W.J. Leong, M. Ismail, A new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput. 218 (2012), 11323-11332.
A.V. Mandara, M. Mamat, M.Y. Waziri, M.A. Mohammed, U.A. Yakubu, A new conjugate gradient coefficient with exact line search for unconstrained optimization, Far East J. Math. Sci. 105 (2) (2018), 193-206. https://doi.org/10.17654/ms105020193
A.B. Abubakar, M. Malik, P. Kumam, H. Mohammad, M. Sun, A.H. Ibrahim, A.I. Kiri, A Liu-Storey-type conjugate gradient method for unconstrained minimization problem with application in motion control, J. King Saud Univ.-Sci. 34 (2022), 1-11. https://doi.org/10.1016/j.jksus.2022.101923
M. Fang, M. Wang, M. Sun, R. Chen, A modified hybrid conjugate gradient method for unconstrained optimization, Hindawi J. Math., Article ID 5597863 (2021), 1-9. https://doi.org/10.1155/2021/5597863
J. Sabíu, K. Muangchoo, A. Shah, A.B. Abubakah, K.O. Aremu, An inexact optimal hybrid conjugate gradient method for solving symmetric nonlinear equations, Symmetry 13 (2021), 1829. https://doi.org/10.3390/sym13101829
O.B. Akinduko, A new conjugate gradient method with sufficient descent property, Earthline J. Math. Sci. 6 (2021), 163-174. https://doi.org/10.34198/EJMS.6121.163174
T. Diphofu, P. Kaelo, A.R. Tufa, A convergent hybrid three-term conjugate gradient method with sufficient descent property for unconstrained optimization, Topol. Algebra Appl. 10 (2022), 47-60. https://doi.org/10.1515/taa-2022-0112
Y.H. Dai, L.Z. Liao, New conjugacy conditions and related nonlinear conjugate gradient method, Appl. Math. Optim. 43 (2001), 87-101. https://doi.org/10.1007/s002450010019
Y.H. Dai, Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim. 10 (1999), 177-182. https://doi.org/10.1137/s1052623497318992
I. Bongartz, A.R. Conn, N.I.M. Gould, P.L. Toint, CUTE: Constrained and Unconstrained Testing Environments, ACM Trans. Math. Softw. 21 (1995), 123-160. https://doi.org/10.1145/200979.201043
N. Andrei, An unconstrained optimization test functions collection, Adv. Model. Optim. 10(1) (2008a), 147-161.
E.D. Dolan and J.J. More, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), 201-213.
This work is licensed under a Creative Commons Attribution 4.0 International License.
.jpg){kind=logo}
