The previous chapter introduced feed-forward neural networks and demonstrated that, theoretically, implementing the training procedure for an arbitrary feed-forward neural network is relatively simple. Unfortunately, neural networks trained this way will suffer from several problems such as stability of the training process – that is, slow convergence due to parameters jumping around a good minimum – and overfitting. In this chapter, we will describe several practical solutions that mitigate these problems. In particular, we discuss minibatching, multiple optimization algorithms, other activation and cost functions, regularization, dropout, temporal averaging, and parameter initialization and normalization.

Mathematical optimization has been used since the early 20th century to improve the profitability of systems and processes. The time-value of money that leads to the concepts of net present value, annual worth, and annual cost of capital investment, is paramount in the optimization of energy systems that typically operate for very long periods. The method of thermoeconomics (which was formulated in the 1960s) and the similar method of exergoeconomics (which emerged in the 1990s) are two cost-analysis methods extensively used for the optimization of energy systems, components, and processes. Calculus optimization and the Lagrange undetermined multipliers are similarly used tools. This chapter begins with an exposition of the basic concepts of economics and optimization theory, and continues with the critical examination of the mathematical tools for the optimization of energy conversion systems using the exergy concept. The uncertainty of the optimum solution, which is an important consideration in all economic analyses, is clarified and an uncertainty analysis for exergy-consuming systems is presented.