Advanced Stochastic Optimization Algorithm for Deep Learning Artificial Neural Networks in Banking and Finance Industries
Abstract
One of the objectives of this paper is to incorporate fat-tail effects into, for instance, Sigmoid in order to introduce Transparency and Stability into the existing stochastic Activation Functions. Secondly, according to the available literature reviewed, the existing set of Activation Functions were introduced into the Deep learning Artificial Neural Network through the “Window” not properly through the “Legitimate Door” since they are “Trial and Error “and “Arbitrary Assumptions”, thus, the Author proposed a “Scientific Facts”, “Definite Rules: Jameel’s Stochastic ANNAF Criterion”, and a “Lemma” to substitute not necessarily replace the existing set of stochastic Activation Functions, for instance, the Sigmoid among others. This research is expected to open the “Black-Box” of Deep Learning Artificial Neural networks. The author proposed a new set of advanced optimized fat-tailed Stochastic Activation Functions EMANATED from the AI-ML-Purified Stocks Data namely; the Log – Logistic (3P) Probability Distribution (1st), Cauchy Probability Distribution (2nd), Pearson 5 (3P) Probability Distribution (3rd), Burr (4P) Probability Distribution (4th), Fatigue Life (3P) Probability Distribution (5th), Inv. Gaussian (3P) Probability Distribution (6th), Dagum (4P) Probability Distribution (7th), and Lognormal (3P) Probability Distribution (8th) for the successful conduct of both Forward and Backward Propagations of Deep Learning Artificial Neural Network. However, this paper did not check the Monotone Differentiability of the proposed distributions. Appendix A, B, and C presented and tested the performances of the stressed Sigmoid and the Optimized Activation Functions using Stocks Data (2014-1991) of Microsoft Corporation (MSFT), Exxon Mobil (XOM), Chevron Corporation (CVX), Honda Motor Corporation (HMC), General Electric (GE), and U.S. Fundamental Macroeconomic Parameters, the results were found fascinating. Thus, guarantee, the first three distributions are excellent Activation Functions to successfully conduct any Stock Deep Learning Artificial Neural Network. Distributions Number 4 to 8 are also good Advanced Optimized Activation Functions. Generally, this research revealed that the Advanced Optimized Activation Functions satisfied Jameel’s ANNAF Stochastic Criterion depends on the Referenced Purified AI Data Set, Time Change and Area of Application which is against the existing “Trial and Error “and “Arbitrary Assumptions” of Sigmoid, Tanh, Softmax, ReLu, and Leaky ReLu.
References
Aman, D., & Payal, P. (2019). Analysis of Non-Linear Activation Functions for Classification Tasks Using Convolutional Neural Networks. Recent Patents on Computer Science, 12(3). https://doi.org/10.2174/2213275911666181025143029
Artist Hans Hoffman wrote, “The ability to simplify means to eliminate the unnecessary so that the necessary may speak.” Retrieved fromhttps://www.brainyquote.com/quotes/hans_hofmann_107805
Barnaby, B. et. at (2016). Complying with IFRS 9 Impairment Calculations for Retail Portfolios, Moody’s Analytics Risk Perspectives, the convergence of Risk. Finance, and Accounting, VII.
Bellotti, T., & Crook, J. (2012). Loss Given Default Models Incorporating Macroeconomic Variables for Credit Cards. International Journal of Forecasting, 28(1), 171-182. https://doi.org/10.1016/j.ijforecast.2010.08.005
Ben, S. (2019). Model Risk Management for Deep Learning and Alpha Strategies, BNP Paribas Asset Management, Quant Summit 2019.
Burton, G. M. (2009). The Clustering of Extreme Movements: Stock prices and the Weather, Princeton University, AtanuSaha, Alixpartners, Alex Grecu, Huron Consulting Group, CEPS working paper No. 186 February, 2009.
Casper, H. (2019). says “Better optimized neural network; choose the right activation function, and your neural network can perform vastly better”. Retrieved from https://mlfromscratch.com/neural-networks-explainerd/#/
Chigozie, E. N. et al. (2018). Activation Functions: Comparison of Trends in Practice and Research for Deep Learning. Retrieved fromhttps://arxiv.org/pdf/1811.03378.pdf
Chigozie, E. N. et al. (2018). Activation Functions: Comparison of Trends in Practice and Research for Deep Learning, Preprint.
Daniel, P. (2006). Estimating Probabilities of Default for German Savings Banks and Credit Cooperatives, University of Applied Sciences, Mainz, Ander Bruchspitze 50, D – 55122 Mainz
David, M. R. (2012). Simulating Default Probabilities in Stress Scenarios, Presented to the PRMIA Global Risk Conference, New York, NY, May 14, 2012.
David, R. (2019). Responding to the AI Challenge Learning from Physical Industries, @2019 The Mathworks, Inc.
Djork-Arne Clevert, Thomas Unterthiner & Sepp Hochreiter (2016). FAST AND ACCURATE DEEP NETWORK LEARNING BY EXPONENTIAL LINEAR UNITS (ELUS), Published as a conference paper at ICLR 2016
Jamilu, A. A. (2015). Banking and Economic Advanced Stressed Probability of Default Models. Asian Journal of Management Sciences, 3(08), 10-18.
Jamilu, A. A. (2015). Estimation of Probability of Default using Advanced Stressed Probability of Default Models, Ongoing Ph.D Thesis, Ahmadu Bello University (ABU), Zaria, Nigeria.
Jamilu, A. A. (2016). Reliable and Sophisticated Advanced Stressed Crises Compound Options Pricing Models. Management and Organizational Studies, 3(1), 39-55. https://doi.org/https://doi.org/10.5430/mos.v3n1p39.
Jamilu, A. A. (2017). An Introduction of Jameel’s Advanced Stressed Economic and Financial Crises Models and to Dramatically Increasing Markets Confidence and Drastically Decreasing Markets Risks. International Journal of Social Science Studies, 4(3), 39-71. https://doi.org/https://doi.org/10.11114/ijsss.v4i3.1326
Jamilu, A. A. (2017). Jameel’s Criterion and Jameel’s Advanced Stressed Models: An Ideas that Lead to Non-Normal Stocks Brownian Motion Models. Noble International Journal of Business and Management Research, 1(10), 136-154. URL: http://napublisher.org/?ic=journals&id=2.
Jamilu, A. A. (2017). Jameel’s Criterion and Jameel’s Advanced Stressed Models: An Ideas that Lead to Non-Normal Stocks Brownian Motion Models. Noble International Journal of Business and Management Research, 1(10), 136-154. URL: URL: http://napublisher.org/?ic=journals&id=2
Jamilu, A. A. (2018). Jameel’s Dimensional Stressed Default Probability Models are Indeed IFRS 9 Complaint Models. Journal of Economics and Management Sciences, 1(2), 104-114. https://doi.org/10.30560/jems.v1n2p102.
Joonho, L. et al. (2019), ProbAct: A Probabilistic Activation Function for Deep Neural Networks, Preprint. Under review.
Klambauer, et, al. (2017). Self-Normalizing Neural Networks, Institute of Bioinformatics, Johannes Kepler University Linz, Austria.
Lichman. (2013). UCI machine learning repository. Retrieved from http://archive. ics. uci. edu/ml 901
M, & Van, N. (1968). Fractional Brownian Motions, Fractional Noises and Applications (M & Van Ness (1968)). SIAM Review, 10, 422-437
Mohit, G. et al. (2019). Learning Activation Functions: A new paradigm for understanding Neural Networks. Proceedings of Machine Learning Research, 101, 1–18.
Nair, et al. (2010). Rectified linear units improve restricted boltzmann machines, ICML'10 Proceedings of the 27th International Conference on International Conference on Machine Learning Pages 807-814, Haifa, Israel — June 21 - 24, 2010.
Nassim, N. T. (2007). Black Swans and the Domains of Statistics. American Statistician, 61(3).
Nassim, N. T. (2009). Errors, Robustness, and Fourth Quadrant, New York University Polytechnic Institute and Universa Investment, United States. International Journal of Forecasting, 25, 744-759.
Nassim, N. T. (2010). Convexity, Robustness, and Model Error inside the “Black Swan Domain”, Draft Version, September, 2010.
Nassim, N. T. (2010). Why Did the Crisis of 2008 Happen, Draft (3rd ed.), August, 2010.
Nassim, N. T. (2011). A Map and Simple Heuristic to Detect Fragility, Antifragility, and Model Error (1st ed.).
Nassim, N. T. (2011). The Future has Thicker Tails Past: Model Error as Branching Counterfactuals, presented in Honor of Benoit Mandelbrot’s at his Scientific Memorial, Yale University, April, 2011.
Nassim, N. T. (2012). The Illusion of Thin – Tails under Aggregation, NYU – Poly, January, 2012
Nassim, N. T. et al. (2009). Risk Externalities and Too bid to Fail, New York University Polytechnic Institute, 11201, New York, United States.
Onali, E., & Ginesti, G. (2014). Pre-adoption Market Reaction to IFRS 9: A Cross-country Event-study. Journal of Accounting and Public Policy, 33(6), 628-637.
Peter Martey Addo et al. (2018). Credit Risk Analysis using Machine and Deep Learning Models. Risks Journal, Risks, 6(38). https://doi.org/10.3390/risks6020038
Ram, Ananth, et al. (2019). Opening the “Black Box”. The Path to Deployment of AI Models in Banking, White Paper, DataRobot and REPLY AVANTAGE.
Reney, D. E., & Michael, R. M. (2016). Forecasting of Stock Prices using Brownian Motion - Monte Carlo Simulation. Proceedings of the 2016 International Conference on Industrial Engineering and Operations Management. Kuala Lumpur, Malaysia, March 8-10, 2016.
Schoenholz, et al. (2017). DEEP NEURAL NETWORKS AS GAUSSIAN PROCESSES, Published as a conference paper at ICLR 2018. Retrieved from file:///C:/Users/pc/Downloads/Deep_Neural_Networks_as_Gaussian_Processes.pdf
Sebastian, R. (2018), STAT 479: Machine Learning Lecture Notes. Retrieved from http://stat.wisc.edu/~sraschka/teaching/stat479fs2018/.
Sebastian, U. (2017). Neural Network Architectures and Activation Functions: A Guassian Process Approach, Technical University Munich, 2017.
Soufiane, H. et al. (2019). On the Impact of the Activation Function on Deep Neural Networks Training. Retrieved from https://arxiv.org/pdf/1902.06853.pdf
Soufiane, H. et al. (2019). On the Impact of the Activation Function on Deep Neural Networks Training, Proceedings of the 36 th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019.
Spreedhar, T. B. et al. (2004). Forecasting Default with the KMV – Merton Model, University of Michigan, Ann Arbor MI 48109.
Steven, R. D. (). Stochastic Processes and Advanced Mathematical Finance, The Definition of Brownian Motion and the Wiener process, Department of Mathematics, 203 Avery Hall, University of Nebraska-Lincoln, Lincoln, NE 68588-0130.
Sven-Patrik, H. (). Machine Learning, Deep Learning, Experimental Particle Physics, University of Glasgow.
Tidaruk, A. (2014). Mathematical Model of Stock Prices via a Fractional Brownian Motion Model with Adaptive Parameters.
Ton, D. (2004). Simulation of Fractional Brownian Motion, Thesis, University of Twente, Department of Mathematical Sciences, P.O. BOX 217, 7500 AE Enschede, Netherlands.
Wenyu, Zh. (2015). Introduction to Ito’s Lemma, Lecture Note, Cornell University, Department of Statistical Sciences, May 6, 2015.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).
