A Tutorial on Neural Networks and Gradient-free Training
dc.contributor.author | Rozario, Turibius | |
dc.contributor.author | Trivedi, Arjun | |
dc.contributor.author | Goel, Ankit | |
dc.date.accessioned | 2023-01-04T15:49:07Z | |
dc.date.available | 2023-01-04T15:49:07Z | |
dc.date.issued | 2022-11-26 | |
dc.description.abstract | This paper presents a compact, matrix-based representation of neural networks in a self-contained tutorial fashion. Although neural networks are well-understood pictorially in terms of interconnected neurons, neural networks are mathematical nonlinear functions constructed by composing several vector-valued functions. Using basic results from linear algebra, we represent a neural network as an alternating sequence of linear maps and scalar nonlinear functions, also known as activation functions. The training of neural networks requires the minimization of a cost function, which in turn requires the computation of a gradient. Using basic multivariable calculus results, the cost gradient is also shown to be a function composed of a sequence of linear maps and nonlinear functions. In addition to the analytical gradient computation, we consider two gradient-free training methods and compare the three training methods in terms of convergence rate and prediction accuracy | en_US |
dc.description.uri | https://arxiv.org/abs/2211.17217 | en_US |
dc.format.extent | 8 pages | en_US |
dc.genre | journal articles | en_US |
dc.genre | preprints | en_US |
dc.identifier | doi:10.13016/m2cmel-vvqh | |
dc.identifier.uri | https://doi.org/10.48550/arXiv.2211.17217 | |
dc.identifier.uri | http://hdl.handle.net/11603/26531 | |
dc.language.iso | en_US | en_US |
dc.relation.isAvailableAt | The University of Maryland, Baltimore County (UMBC) | |
dc.relation.ispartof | UMBC Mechanical Engineering Department Collection | |
dc.relation.ispartof | UMBC Faculty Collection | |
dc.relation.ispartof | UMBC Student Collection | |
dc.rights | This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author. | en_US |
dc.rights | Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) | * |
dc.rights.uri | https://creativecommons.org/licenses/by-nc-nd/4.0/ | * |
dc.title | A Tutorial on Neural Networks and Gradient-free Training | en_US |
dc.type | Text | en_US |
dcterms.creator | https://orcid.org/0000-0002-4146-6275 | en_US |