Volume 16, Issue 1 (4-2021)
IJMSI 2021, 16(1): 15-33 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
kimiaei M, esmaeili H, rahpeymaii F. A Trust-region Method using Extended Nonmonotone Technique for Unconstrained Optimization. IJMSI 2021; 16 (1) :15-33
URL: http://ijmsi.ir/article-1-1188-en.html
URL: http://ijmsi.ir/article-1-1188-en.html
Abstract:
In this paper, we present a nonmonotone trust-region algorithm for unconstrained optimization. We first introduce a variant of the nonmonotone strategy proposed by Ahookhosh and Amini cite{AhA 01} and incorporate it into the trust-region framework to construct a more efficient approach. Our new nonmonotone strategy combines the current function value with the maximum function values in some prior successful iterates. For iterates far away
from the optimizer, we give a very strong nonmonotone strategy. In the vicinity of the optimizer, we have a weaker nonmonotone strategy. It leads to a medium nonmonotone strategy when iterates are not far away from or close to the optimizer. Theoretical analysis indicates that the new approach converges globally to a first-order critical point under classical assumptions. In addition, the local convergence is also studied. Extensive numerical experiments for unconstrained optimization problems are reported.
from the optimizer, we give a very strong nonmonotone strategy. In the vicinity of the optimizer, we have a weaker nonmonotone strategy. It leads to a medium nonmonotone strategy when iterates are not far away from or close to the optimizer. Theoretical analysis indicates that the new approach converges globally to a first-order critical point under classical assumptions. In addition, the local convergence is also studied. Extensive numerical experiments for unconstrained optimization problems are reported.
Keywords: Unconstrained optimization, Trust-region framework, Nonmonotone technique, Theoretical convergence
Type of Study: Research paper |
Subject:
General
References
1. M. Ahookhosh, K. Amini, An efficient nonmonotone trust-region method for unconstrained optimization,textit{Numerical Algorithms} textbf{59}(4), (2012), 523--540. [DOI:10.1007/s11075-011-9502-5]
2. M. Ahookhosh, K. Amini, A nonmonotone trust region method with adaptive radius for unconstrained optimization problems, textit{Computers and Mathematics with Applications}, textbf{60}, (2010), 411--422. [DOI:10.1016/j.camwa.2010.04.034]
3. M. Ahookhosh, K. Amini, M. Kimiaei, A globally convergent trust-region method for large-scale symmetric nonlinear systems, textit{Numerical Functional Analysis and Optimization}, textbf{36}, (2015), 830--855. [DOI:10.1080/01630563.2015.1046080]
4. M. Ahookhosh, K. Amini, H., Nosratipour, An inexact line search approach using modified nonmonotone strategy for unconstrained optimization, textit{Numerical Algorithms}, textbf{66}, (2014), 49--78. [DOI:10.1007/s11075-013-9723-x]
5. M. Ahookhosh, K. Amini, M.R. Peyghami, A nonmonotone trust-region line search method for large-scale unconstrained optimization, textit{Applied Mathematical Modelling}, textbf{36}, (2012), 478--487. [DOI:10.1016/j.apm.2011.07.021]
6. M. Ahookhosh, H. Esmaeili, M. Kimiaei, An effective trust-region-based approach for symmetric nonlinear systems, textit{International Journal of Computer Mathematics}, textbf{90}, (2013), 671--690. [DOI:10.1080/00207160.2012.736617]
7. M. Ahookhosh, S. Ghaderi, Two globally convergent nonmonotone trust-region methods for unconstrained optimization, textit{Journal of Applied Mathematics and Computing}, textbf{50}(1-2), (2016), 529--555. [DOI:10.1007/s12190-015-0883-9]
8. N. Andrei, An unconstrained optimization test functions collection, textit{Advanced Modeling and Optimization}, textbf{10}(1), (2008), 147--161.
9. R. Byrd, J. Nocedal, R. Schnabel, Representation of quasi-Newton matrices and their use in limited memory methods, textit{Mathematical Programming}, textbf{63}, (1994), 129--156. [DOI:10.1007/BF01582063]
10. A.R. Conn, N.I.M. Gould, Ph.L. Toint, textit{Trust-Region Methods}, Society for Industrial and Applied Mathematics SIAM, Philadelphia, 2000. [DOI:10.1137/1.9780898719857]
11. N.Y. Deng, Y. Xiao, F.J. Zhou, Nonmonotonic trust region algorithm, textit{Journal of Optimization Theory and Applications}, textbf{26}, (1993), 259--285. [DOI:10.1007/BF00939608]
12. E.D. Dolan, J.J. Mor'{e}, Benchmarking optimization software with performance profiles, textit{Mathematical Programming}, textbf{91}, (2002), 201--213. [DOI:10.1007/s101070100263]
13. H. Esmaeili, M. Kimiaei, An improved adaptive trust-region method for unconstrained optimization, textit{Mathematical Modelling and Analysis}, textbf{19}, (2014), 469--490. [DOI:10.3846/13926292.2014.956237]
14. G. Fasano, F. Lampariello, M. Sciandrone, A truncated nonmonotone Gauss-Newton method for large-scale nonlinear least-squares problems, textit{Computational Optimization and Applications}, textbf{34}(3), 343--358, (2006). [DOI:10.1007/s10589-006-6444-2]
15. A. Fischer, P.K. Shukla, M. Wang, On the inexactness level of robust Levenberg-Marquardt methods, textit{Optimization}, textbf{59}(2), (2010), 273--287. [DOI:10.1080/02331930801951256]
16. N.I.M Gould, D. Orban, Ph.L. Toint, CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization, textit{Computational Optimization and Applications}, textbf{60}(3), (2015), 545--557. [DOI:10.1007/s10589-014-9687-3]
17. L. Grippo, F. Lampariello, S. Lucidi, A nonmonotone line search technique for Newton's method, textit{SIAM Journal on Numerical Analysis}, textbf{23}, (1986), 707--716. [DOI:10.1137/0723046]
18. L. Grippo, F. Lampariello, S. Lucidi, A truncated Newton method with nonmonotone linesearch for unconstrained optimization, textit{Journal of Optimization,Theory and Applications}, textbf{60}(3), (1989), 401--419. [DOI:10.1007/BF00940345]
19. L. Grippo, F. Lampariello, S. Lucidi, A class of nonmonotone stabilization method in unconstrained optimization, textit{Numerische Mathematik}, textbf{59}, (1991), 779--805. [DOI:10.1007/BF01385810]
20. L. Kaufman, Reduced storage quasi-Newton trust region approaches to function optimization, textit{SIAM Journal on Optimization}, textbf{10}(1), 56--69, (1999). [DOI:10.1137/S1052623496303779]
21. bibitem{kimiaei0} M. Kimiaei, A new class of nonmonotone adaptive trust-region methods for nonlinear equations with box constraints, textit{Calcolo}, textbf{54}(3), 769--812, (2017) . [DOI:10.1007/s10092-016-0208-x]
22. M. Kimiaei, S. Ghaderi, A new restarting adaptive Trust-Region method for unconstrained optimization, textit{Journal of the Operations Research Society of China}, textbf{5}(4), (2017), 487--507. [DOI:10.1007/s40305-016-0149-8]
23. M. Kimiaei, F. Rahpeymaii, A new nonmonotone line-search trust-region approach for nonlinear systems, textit{TOP}, textbf{27}(2), (2019), 199--232. [DOI:10.1007/s11750-019-00497-2]
24. L. Lukv{s}an, C. Matonoha, J. Vlv{c}ek, Modified CUTE problems for sparse unconstrained optimization. textit{Techical Report}, textbf{1081}, ICS AS CR, November, 2010.
25. L. Lukv{s}an, J. Vlv{c}ek, Sparse test problems for unconstrained optimization, textit{Techical Report}, textbf{1064}, ICS AS CR, November 2003.
26. YU. Nesterov, Modified Gauss-Newton scheme with worst case guarantees for global performance, textit{Optimization Methods and Software}, textbf{22}(3), (2007), 469--483. [DOI:10.1080/08927020600643812]
27. J. Nocedal, S.J. Wright, textit{Numerical Optimization}, Springer, NewYork, (2006).
28. M.J.D. Powell, Convergence properties of a class of minimization algorithms. in Nonlinear Programming, O.L. Mangasarian, R.R. Meyer, and S.M. Robinson, eds., Academic Press, NewYork, 1--27, (1975). [DOI:10.1016/B978-0-12-468650-2.50005-5]
29. D.F. Shanno, K.H. Phua, Matrix conditioning and non-linear optimization, textit{Mathematical Programming}, textbf{14}, (1978), 149--160. [DOI:10.1007/BF01588962]
30. T. Steihaug, The conjugate gradient method and trust regions in large scale optimization, textit{SIAM Journal on Numerical Analysis}, textbf{20}, (1983), 626--637. [DOI:10.1137/0720042]
31. W. Sun, Y. Yuan, textit{Optimization Theory and Methods}: Nonlinear Programming. Springer, Berlin, (2006).
32. S.W. Thomas, textit{Sequential estimation techniques for quasi-Newton algorithms}, Cornell University, 1975.
33. Ph.L. Toint, Numerical solution of large sets of algebraic nonlinear equations, textit{Mathematics of Computation}, textbf{46}(173), (1986), 175--189. [DOI:10.1090/S0025-5718-1986-0815839-9]
34. L. Xu, J.V. Burke, An active set $ell_{infty}-$trust region algorithm for box constrained optimization. Technical Report preprint, Departeman Mathematics, niversity of Washington, Seattle, WA 98195, U.S.A.
35. H.C. Zhang, W.W. Hager, A nonmonotone line search technique for unconstrained optimization, textit{SIAM journal on Optimization}, textbf{14}(4), (2004), 1043--1056. [DOI:10.1137/S1052623403428208]
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
