Mathematical Optimization: Fundamentals and Applications

By Fouad Sabry

What Is Mathematical Optimization

Mathematical optimization, often known as mathematical programming, is the process of choosing, from among a group of potential solutions, one that is optimal with relation to a set of predetermined criteria. Discrete optimization and continuous optimization are the two subfields that make up the majority of this field. Problems related to optimization appear in each and every one of the quantitative subfields, from computer science and engineering to operations research and economics. For millennia, the field of mathematics has been interested in the creation of methods that may solve these problems.

How You Will Benefit

(I) Insights, and validations about the following topics:

Chapter 1: Mathematical optimization

Chapter 2: Brachistochrone curve

Chapter 3: Curve fitting

Chapter 4: Deterministic global optimization

Chapter 5: Goal programming

Chapter 6: Least squares

Chapter 7: Process optimization

Chapter 8: Simulation-based optimization

Chapter 9: Calculus of variations

Chapter 10: Vehicle routing problem

(II) Answering the public top questions about mathematical optimization.

(III) Real world examples for the usage of mathematical optimization in many fields.

(IV) 17 appendices to explain, briefly, 266 emerging technologies in each industry to have 360-degree full understanding of mathematical optimization' technologies.

Who This Book Is For

Professionals, undergraduate and graduate students, enthusiasts, hobbyists, and those who want to go beyond basic knowledge or information for any kind of mathematical optimization.

• Language: English
• Publisher: One Billion Knowledgeable
• Release date: Jun 30, 2023

Book preview

Chapter 1: Mathematical optimization

The selection of a best element, with reference to some criteria, from some collection of accessible alternatives is the goal of mathematical programming as well as mathematical optimization, which may also be written as the alternate spelling optimisation.

An optimization issue is defined under the more general approach as the process of either maximizing or reducing the value of a real function. This is accomplished by methodically selecting input values from within an acceptable set and calculating the value of the function. A significant portion of the field of applied mathematics is dedicated to the application of optimization theory and methods to many different formulations. In a broader sense, optimization refers to the process of determining the best available values of a particular objective function when that function is applied to a defined domain (or input). This process can be applied to a wide variety of different types of objective functions and a wide variety of different types of domains.

Depending on whether the variables being optimized are continuous or discrete, optimization issues may be broken down into two distinct types:

A discrete optimization problem is an optimization problem that has discrete variables. In a discrete optimization problem, an object such an integer, permutation, or graph has to be located from a set that has a countable size.

The process of extracting an optimum value from a continuous function is referred to as continuous optimization, and it is used to describe a kind of issue that involves continuous variables. Constrained difficulties and multimodal problems are two examples of these types of issues.

The following is one possible representation of a problem requiring optimization::

Given: a function f : A → ℝ from some set A to the real numbers

Sought: an element x0 ∈ A such that f(x0) ≤ f(x) for all x ∈ A (minimization) or such that f(x0) ≥ f(x) for all x ∈ A (maximization).

A issue with such a formulation is referred to as an optimization problem or a mathematical programming problem (a phrase that is not directly connected to computer programming but is still in use, for example in linear programming; see the section on History below for more information). This overarching paradigm lends itself well to the modeling of a wide variety of theoretical and practical challenges.

Considering that the following is true

{\displaystyle f(\mathbf {x} _{0})\geq f(\mathbf {x} )\Leftrightarrow -f(\mathbf {x} _{0})\leq -f(\mathbf {x} ),}

It is sufficient to merely address issues involving minimization. On the other hand, the alternative viewpoint, which is to address solely situations involving maximizing, is also a legitimate one.

In the domains of physics, problems that are posed using this method may refer to the technique as energy minimization. When doing so, they may talk of the value of the function f as reflecting the energy of the system that is being represented. When it comes to machine learning, it is always required to continually assess the quality of a data model by making use of a cost function. In this function, a minimum suggests a set of possible ideal parameters with an optimal (lowest) error.

Typically, A is some subset of the Euclidean space ℝn, often governed by a predetermined list of restrictions, requirements of equality or inequality that must be met by the members of group A.

The domain A of f is also referred to as the choice set or the search space, while the components of A are often referred to as candidate solutions or plausible solutions.

Depending on the context, the function denoted by the letter f may be referred to as an objective function, a loss function or cost function (functions that minimize losses or costs), a utility function or fitness function (functions that maximize benefits), or, in some contexts, an energy function or energy functional. An optimum solution is referred to as a viable solution that optimizes (or maximizes, depending on whether or not that is the aim) the objective function.

Conventional optimization issues in mathematics are often phrased in terms of minimization.

A local minimum x* is defined as an element for which there exists some δ > 0 such that

{\displaystyle \forall \mathbf {x} \in A\;{\text{where}}\;\left\Vert \mathbf {x} -\mathbf {x} ^{\ast }\right\Vert \leq \delta ,\,}

the expression f(x*) ≤ f(x) holds; That is to say, on some portion of the area around x*, every value of the function is either higher than or equal to the value at that element. Local maxima are defined similarly.

A global minimum is at least as excellent as any element that is even remotely possible, as contrast to a local minimum, which is just at least as good as any adjacent components. In a minimization issue, it is possible for there to be more than one local minimum, unless the objective function being minimized is convex. In a problem that is convex, if there is a local minimum that is interior (meaning that it is not on the edge of the set of feasible elements), then it is also the global minimum; however, in a problem that is nonconvex, there may be more than one local minimum, and not all of those local minima need to be global minima.

Many of the proposed algorithms for solving nonconvex problems, as well as the vast majority of the solvers that are available for purchase, are incapable of differentiating between locally optimal solutions and globally optimal solutions, and as a result, they will only consider the former to be true answers to the initial problem. This is true even when the algorithms are designed to solve nonconvex problems. Global optimization is the area of applied mathematics and numerical analysis that focuses on the creation of deterministic algorithms that are able to guarantee convergence to the actual optimal solution of a nonconvex problem within a finite amount of time. This area of study is related to the study of convex optimization.

Notation that is unique is often used when expressing optimization challenges. Here are several instances:

Take into account the note that follows::

{\displaystyle \min _{x\in \mathbb {R} }\;\left(x^{2}+1\right)}

This denotes the minimum value of the objective function x² + 1, when choosing x from the set of real numbers ℝ.

In this scenario, one is the absolute lowest possible value, occuring with x equal to zero.

In the same vein, the notation

\max _{x\in \mathbb {R} }\;2x

requests the value that gives the highest benefit from the objective function 2x, where x may be any real integer. Since the objective function in this scenario is unbounded, there is no such thing as a maximum value; hence, the correct response is infinity or undefined..

Take into account the note that follows::

{\underset {x\in (-\infty ,-1]}{\operatorname {arg\,min} }}\;x^{2}+1,

or equivalently

{\underset {x}{\operatorname {arg\,min} }}\;x^{2}+1,\;{\text{subject to:}}\;x\in (-\infty ,-1].

This represents the value (or values) of the argument x in the interval (−∞,−1] that minimizes (or minimises) the objective function x² + 1 (the actual minimum value of that function is not what the problem asks for).

This being the case, The solution is that x equals -1, as it is impossible for x to equal zero, that is, It is not part of the set of things that are viable.

Similarly, {\displaystyle {\underset {x\in [-5,5],\;y\in \mathbb {R} }{\operatorname {arg\,max} }}\;x\cos y,}

or equivalently

{\displaystyle {\underset {x,\;y}{\operatorname {arg\,max} }}\;x\cos y,\;{\text{subject to:}}\;x\in [-5,5],\;y\in \mathbb {R} ,}

stands for the symbol x, y} pair (or pairs) that maximizes (or maximize) the value of the objective function x cos y, in addition to the restriction that x must fall within the range [5,5] (once more),, It makes no difference what the actual highest value of the expression is).

This being the case, The pairings satisfying the conditions 5 constitute the solutions, 2kπ} and {−5, (2k + 1)π}, where k may take on any value among the integers.

The operators arg min and arg max, which may alternatively be written as argmin and argmax, respectively, stand for the argument of the minimum and the argument of the maximum, respectively.

Formulae based on calculus were discovered by Fermat and Lagrange, and iterative procedures were suggested by Newton and Gauss as a means of progressing toward optimality by these two mathematicians.

Although the most of the theory had been presented by Leonid Kantorovich in 1939, George B. Dantzig is the one who is credited with coming up with the name linear programming to describe particular optimization scenarios. (The term programming does not apply to computer programming in this context; rather, it originates from the usage of the term program by the United States military to refer to planned training and logistical timetables, which were the challenges Dantzig examined at the time.) 1947 was the year when Dantzig released the Simplex algorithm, and John von Neumann was the one who invented the notion of duality in the same year.

The following are some more noteworthy researchers that have contributed to mathematical optimization::

Richard Bellman

Dimitri Bertsekas

Michel Bierlaire

Roger Fletcher

It was