A Tutorial on Neural Networks and Gradient-free Training

dc.contributor.authorRozario, Turibius
dc.contributor.authorTrivedi, Arjun
dc.contributor.authorGoel, Ankit
dc.date.accessioned2023-01-04T15:49:07Z
dc.date.available2023-01-04T15:49:07Z
dc.date.issued2022-11-26
dc.description.abstractThis paper presents a compact, matrix-based representation of neural networks in a self-contained tutorial fashion. Although neural networks are well-understood pictorially in terms of interconnected neurons, neural networks are mathematical nonlinear functions constructed by composing several vector-valued functions. Using basic results from linear algebra, we represent a neural network as an alternating sequence of linear maps and scalar nonlinear functions, also known as activation functions. The training of neural networks requires the minimization of a cost function, which in turn requires the computation of a gradient. Using basic multivariable calculus results, the cost gradient is also shown to be a function composed of a sequence of linear maps and nonlinear functions. In addition to the analytical gradient computation, we consider two gradient-free training methods and compare the three training methods in terms of convergence rate and prediction accuracyen_US
dc.description.urihttps://arxiv.org/abs/2211.17217en_US
dc.format.extent8 pagesen_US
dc.genrejournal articlesen_US
dc.genrepreprintsen_US
dc.identifierdoi:10.13016/m2cmel-vvqh
dc.identifier.urihttps://doi.org/10.48550/arXiv.2211.17217
dc.identifier.urihttp://hdl.handle.net/11603/26531
dc.language.isoen_USen_US
dc.relation.isAvailableAtThe University of Maryland, Baltimore County (UMBC)
dc.relation.ispartofUMBC Mechanical Engineering Department Collection
dc.relation.ispartofUMBC Faculty Collection
dc.relation.ispartofUMBC Student Collection
dc.rightsThis item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.en_US
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)*
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.titleA Tutorial on Neural Networks and Gradient-free Trainingen_US
dc.typeTexten_US
dcterms.creatorhttps://orcid.org/0000-0002-4146-6275en_US

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
2211.17217.pdf
Size:
385.32 KB
Format:
Adobe Portable Document Format
Description:

License bundle

Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
2.56 KB
Format:
Item-specific license agreed upon to submission
Description: