多層パーセプトロン: ニューラル ネットワークをデコードするための基礎と応用

By Fouad Sabry

多層パーセプトロンとは

フィードフォワード人工ニューラル ネットワーク (ANN) の完全接続クラスである多層パーセプトロン (MLP) は、多層パーセプトロンと呼ばれます。 「MLP」という言葉はかなり曖昧な形で使われています。 フィードフォワード ANN を指す場合もあれば、パーセプトロンの複数の層で構成されるネットワークをより具体的に指す場合もあります。 詳細については、「用語」を参照してください。 隠れ層が 1 つだけ含まれている場合、多層パーセプトロンは冗談めかして「バニラ」ニューラル ネットワークと呼ばれることがあります。 これは、この用語がスラングの文脈で使用される場合に特に当てはまります。

どのようなメリットがあるか

(I) 以下のトピックに関する洞察と検証 :

第 1 章: 多層パーセプトロン

第 2 章: 人工ニューラル ネットワーク

第 3 章: パーセプトロン

第 4 章: 人工ニューロン

第 5 章: 活性化関数

第 6 章: バックプロパゲーション

第 7 章: デルタ ルール

第 8 章: フィードフォワード ニューラル ネットワーク

第 9 章: 普遍近似定理

第 10 章: 人工ニューラル ネットワークの数学

(II) 多層パーセプトロンに関する一般のよくある質問に答えます。

(III) 多くの分野で多層パーセプトロンを使用する実際の例。

本書の対象者

専門家、学部生、大学院生、愛好家、趣味人、基礎知識や知識を超えたい人 あらゆる種類の多層パーセプトロンに関する情報。

人工知能とは何かシリーズ

人工知能の 電子書籍 シリーズは、200 を超えるトピックを包括的にカバーしています。 各電子ブックでは、特定の人工知能のトピックを詳しく取り上げており、その分野の専門家によって書かれています。 このシリーズは、読者に人工知能の概念、技術、歴史、応用について徹底的に理解してもらうことを目的としています。 取り上げられるトピックには、機械学習、深層学習、ニューラル ネットワーク、コンピューター ビジョン、自然言語処理、ロボット工学、倫理などが含まれます。 電子ブックは、専門家、学生、およびこの急速に進歩する分野の最新の開発について学ぶことに興味があるすべての人を対象に書かれています。
Artificial Intelligence eBook シリーズは、基本的な概念から最先端の研究まで、詳細でありながらアクセスしやすい探究を提供します。 200 冊を超える書籍により、読者は人工知能のあらゆる側面について徹底的な基礎を身につけることができます。 電子ブックは体系的に知識を構築できるように設計されており、後の巻は以前の巻で築いた基礎の上に構築されます。 この包括的なシリーズは、人工知能の専門知識を開発しようとする人にとって不可欠なリソースです。

• Language: 日本語
• Publisher: 10億人の知識があります [Japanese]
• Release date: Jun 26, 2023

Book preview

Chapter 1: Multilayer perceptron

A multilayer perceptron, sometimes known as an MLP, is a kind of feedforward artificial neural network that is completely linked (ANN).

There is no clear definition of MLP as a phrase, Occasionally used in a broad sense to denote any feedforward ANN, Sometimes used only to refer to networks that are made up of numerous layers of perceptrons (with threshold activation); see § Terminology.

Multilayer perceptrons are a kind of neural network that is frequently referred to informally as vanilla neural networks, in particular when there is just one layer that is concealed.

If a multilayer perceptron has a linear activation function in all of its neurons—that is, a linear function that maps the weighted inputs to the output of each neuron—then linear algebra demonstrates that any number of layers can be reduced to a two-layer input-output model. This is because a linear function maps the weighted inputs to the output of each neuron. Some of the neurons in MLPs make use of a nonlinear activation function, which is a model that was established to simulate the frequency of action potentials, also known as firing, produced by biological neurons.

The two activation functions that have been historically prevalent are both sigmoids, and they are defined by

y(v_i) = \tanh(v_i) ~~ \textrm{and} ~~ y(v_i) = (1+e^{-v_i})^{-1}

.

The first one is a hyperbolic tangent, and its value might be anywhere between -1 and 1, whereas the other pertains to the role of logistics, which is similar to the other in form but varies from 0 to 1.

Here y_{i} is the output of the i th node (neuron) and v_{i} is the weighted sum of the input connections.

There have been suggestions made for different activation functions, consisting of the rectifier as well as the softplus functionalities.

Radial basis functions, which are used in radial basis networks, are an example of a more specific kind of activation function, other models that fall within the category of supervised neural networks).

In the most recent advances in deep learning, the rectified linear unit, also known as ReLU, is being employed more often as one of the viable solutions to solve the numerical issues connected to the sigmoids.

The Multi-Layer Perceptron (MLP) is made up of at least three layers of nonlinearly activating nodes. These layers include an input layer, an output layer, and one or more hidden layers.

Given that MLPs have complete connections, each node in one layer connects with a certain weight w_{ij} to every node in the following layer.

The perceptron is able to learn by adjusting the connection weights after each piece of data is processed. These adjustments are made depending on the degree of mistake in the output in comparison to the result that was anticipated. This is an example of supervised learning, and it is accomplished by backpropagation, which is a generalization of the technique for finding the linear perceptron with the least mean squares.

We can represent the degree of error in an output node j in the n th data point (training example) by e_j(n)=d_j(n)-y_j(n) , where {\displaystyle d_{j}(n)} is the desired target value for n th data point at node j , and {\displaystyle y_{j}(n)} is the value produced by the perceptron at node j when the n th data point is given as an input.

The node weights can then be adjusted based on corrections that minimize the error in the entire output for the n th data point, given by

{\displaystyle {\mathcal {E}}(n)={\frac {1}{2}}\sum _{{\text{output node }}j}e_{j}^{2}(n)}

.

Using gradient descent, the change in each weight w_{ij} is

\Delta w_{ji} (n) = -\eta\frac{\partial\mathcal{E}(n)}{\partial v_j(n)} y_i(n)

where y_{i}(n) is the output of the previous neuron i , and \eta is the learning rate, This is chosen to make certain that the weights arrive at a response in a short amount of time, without oscillations.

As was said in the earlier phrase, {\displaystyle {\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}} denotes the partial derivate of the error {\displaystyle {\mathcal {E}}(n)} according to the weighted sum {\displaystyle v_{j}(n)} of the input connections of neuron i .

The derivative to be calculated depends on the induced local field v_{j} , It fluctuates within itself.

It is not difficult to demonstrate that this derivative, when applied to an output node, may be reduced to

-\frac{\partial\mathcal{E}(n)}{\partial v_j(n)} = e_j(n)\phi^\prime (v_j(n))

where \phi^\prime is the derivative of the activation function described above, which does not fluctuate on its own.

Because of the change in weights to a hidden node, the analysis is made more challenging, However, it is possible to demonstrate that the relevant derivative is.

-\frac{\partial\mathcal{E}(n)}{\partial v_j(n)} = \phi^\prime (v_j(n))\sum_k -\frac{\partial\mathcal{E}(n)}{\partial v_k(n)} w_{kj}(n)

.

This depends on the change in weights of the k th nodes, Those are the layers that reflect the output.

Therefore, in order to adjust the weights of the hidden layer, The derivative of the activation function is used to drive the weighting adjustments made to the output layer, Therefore, the backpropagation of the activation function is represented by this procedure.

It is not correct to refer to a single perceptron as a multilayer perceptron since this word does not apply to such a thing. Instead, it is made up of a great number of perceptrons that are stacked in different levels. Multilayer perceptron network is an alternate formulation. In addition, the perceptrons that are used in MLP are not perceptrons in the most conventional sense. A threshold activation function, such as the Heaviside step function, is used by true perceptrons, which are technically considered to be a subtype of artificial neurons. MLP perceptrons are capable of using a variety of different activation functions. A genuine perceptron is capable of doing binary classification, but an MLP neuron, depending on the activation function that it has, may choose to either conduct classification or regression.

Later on, without regard to the nature of the nodes or layers, the name multilayer perceptron was used. These nodes and layers may be formed of arbitrarily specified artificial neurons, thus the phrase does not refer solely to perceptrons. With this reading, the term perceptron does not have the same connotation as an artificial neuron in general, which prevents the definition from being too broad.

The Perceptron was first released by Frank Rosenblatt in the year 1958. Saito, a student of Amari's, used a five-layer MLP with two changeable layers to do computer experiments. These tests demonstrated that the MLP learnt the internal representations necessary to identify non-linearly separable pattern classes. In reality, Rosenblatt was the one who came up with the phrase back-propagating mistakes and first used it in 1962, MLPs are valuable in research because of their capacity to tackle issues stochastically, which frequently enables researchers to find approximate solutions for exceedingly difficult problems such as fitness approximation.

Since Cybenko's theorem demonstrates that MLPs are universal function approximators, it follows that one may utilize them to develop mathematical models via the process of regression analysis. Since classification is a kind of regression that applies only when the response variable is categorical, multilayer perceptrons (MLPs) are effective