Neural Modeling Fields: Fundamentals and Applications

By Fouad Sabry

What Is Neural Modeling Fields

Neural modeling field (NMF) is a mathematical framework for machine learning that integrates ideas from neural networks, fuzzy logic, and model based recognition. Its acronym stands for "Neural Modeling Field." Modeling fields, modeling fields theory (MFT), and Maximum likelihood artificial neural networks (MLANS) are some of the other names that have been used to refer to this concept.At the AFRL, Leonid Perlovsky is the one responsible for developing this framework. The NMF can be understood as a mathematical description of the machinery that make up the mind. These mechanisms include ideas, feelings, instincts, imagination, reasoning, and comprehension. The NMF is organized in a hetero-hierarchical structure that contains many levels. There are concept-models that encapsulate the knowledge at each level of the NMF. These concept-models generate so-called top-down signals, which interact with input signals that come from lower levels. These interactions are governed by dynamic equations, which are responsible for driving concept-model learning, adaptation, and the development of new concept-models for better correspondence to the input, bottom-up signals.

How You Will Benefit

(I) Insights, and validations about the following topics:

Chapter 1: Neural modeling fields

Chapter 2: Machine learning

Chapter 3: Supervised learning

Chapter 4: Unsupervised learning

Chapter 5: Weak supervision

Chapter 6: Reinforcement learning

Chapter 7: Neural network

Chapter 8: Artificial neural network

Chapter 9: Fuzzy logic

Chapter 10: Adaptive neuro fuzzy inference system

(II) Answering the public top questions about neural modeling fields.

(III) Real world examples for the usage of neural modeling fields in many fields.

(IV) 17 appendices to explain, briefly, 266 emerging technologies in each industry to have 360-degree full understanding of neural modeling fields' technologies.

Who This Book Is For

Professionals, undergraduate and graduate students, enthusiasts, hobbyists, and those who want to go beyond basic knowledge or information for any kind of neural modeling fields.

• Language: English
• Publisher: One Billion Knowledgeable
• Release date: Jul 4, 2023

Book preview

Chapter 1: Neural modeling fields

Combining concepts from neural networks, fuzzy logic, and model-based recognition, neural modeling field (NMF) is a mathematical framework for machine learning. Names like Maximum likelihood artificial neural networks and modeling fields theory (MFT) have also been used to describe this concept (MLANS). Leonid Perlovsky of the Air Force Research Laboratory created this framework. NMF is seen as a mathematical description of mental processes like thinking, feeling, imagining, and gaining insight. NMF is a complex, non-linear hierarchy. Top-down signals, generated by concept-models at each level of NMF, interact with bottom-up signals provided as input. In order to improve their correspondence with the bottom-up signals they receive, these interactions are governed by dynamic equations that drive concept-model learning, adaptation, and the formation of new concept-models.

In general, NMF systems have several tiers of processing power. Bottom-up signals are processed and then used to generate an output signal representing the concepts that were recognized. At this stage, input signals are categorized into categories based on the models' interpretation of them. During the learning process, the concept-models are refined to more accurately represent the input signals, elevating the degree of similarity between the two. Similarity has increased, which can be interpreted as the gratification of an instinct for knowledge and is experienced as esthetic feelings.

There are N neurons per level of the hierarchy, where n is an index from 1 to N. These neurons take in data from lower levels of the processing hierarchy in the form of bottom-up signals denoted by X(n). Bottom-up synaptic activations from neurons are represented by the X(n) field. For simplicity, the activation of a neuron is represented generally as a series of numbers, one for each of its synapses,

{\displaystyle {\vec {X}}(n)=\{X_{d}(n)\},d=1..D.}

D is the number of dimensions needed to fully capture the subtleties of individual neuron activation.

Top-down, or concept-models send signals to prime these neurons, Mm(Sm,n)

{\displaystyle {\vec {M}}_{m}({\vec {S}}_{m},n),m=1..M.} , in which M is the total amount of models.

The parameters of a model are what define it, Sm; They are encoded in the brain's neuronal structure by the relative strength of synaptic connections, mathematically, They have a numerical value assigned to them, {\displaystyle {\vec {S}}_{m}=\{S_{m}^{a}\},a=1..A.}

, where A is the number of spatial dimensions required to characterize a given model.

The following is how models represent signals:.

Let's pretend object m is activating neuron n, and that neuron n sends out a signal, denoted X(n), which is characterized by parameters Sm.

Position is one example of a possible parameter, orientation, or the illuminating of a subject.

Model Mm(Sm,n) predicts a value X(n) of a signal at neuron n.

For example, During the Process of Seeing, a neuron n in the visual cortex receives a signal X(n) from retina and a priming signal Mm(Sm,n) from an object-concept-model m.

If both the bottom-up signal from lower-level input and the top-down priming signal are robust, then Neuron n will be activated.

Bottom-up signals provide evidence for multiple competing models, while adjusting their settings to improve the fit, as will be shown below.

This is a condensed explanation of how we perceive the world.

Even in the most mundane visual tasks, many different levels of processing, from the retina to object perception, are involved.

NMF is based on the idea that fundamental interaction dynamics are governed by the same set of rules regardless of scale.

Detection of minuscule details, or normal, everyday things, or the similar mechanism underlying comprehension of sophisticated abstract concepts.

Concept-models and learning are essential to both perception and cognition.

In perception, objects are analogous to conceptual models; relationship and situation models in cognition.

In NMF theory, learning is propelled by dynamics that increase a similarity measure between the sets of models and signals, L(X,M). This similarity measure is crucial to perception and cognition. The similarity metric is based on the model's parameters and the relationships between the bottom-up and concept-model signals that are fed in as input. There are two guiding principles that must be taken into account when formulating a mathematical description of the similarity measure:

To begin, prior to perception, the contents of the visual field are unknown.

Second, it could have any number of things in it. Any bottom-up signal may contain useful information; As a result, all bottom-up signals, denoted by X, are incorporated into the similarity measure's design (n),

{\displaystyle L(\{{\vec {X}}(n)\},\{{\vec {M}}_{m}({\vec {S}}_{m},n)\})=\prod _{n=1}^{N}{l({\vec {X}}(n))}.}

(1)

This expression is a sum of dissimilar parts, l(X(n)), more than any bottom-up information; as a result, it compels the NMF system to take into account every signal (even if one term in the product is zero), In this case, the sum is 0, Low levels of similarity and an unfulfilled thirst for knowledge; This exemplifies the first guiding principle.

Second, prior to any conscious awareness, The brain is unable to determine the specific stimulus that activated a given retinal neuron.

Therefore, For each input neuron signal, a partial similarity measure is built so that each model is considered separately (as a sum over concept-models).

Its constituent elements are conditional partial similarities between signal X(n) and model Mm, l(X(n)|m).

This metric is conditional on the presence of object m, therefore, when incorporating all of these metrics into a single similarity score, L, They are scaled up by r. (m), which stand for a statistical measure of the likelihood that object m exists.

By integrating these factors with the aforementioned principles,, This is how a similarity index is built::

{\displaystyle L(\{{\vec {X}}(n)\},\{{\vec {M}}_{m}({\vec {S}}_{m},n)\})=\prod _{n=1}^{N}{\sum _{m=1}^{M}{r(m)l({\vec {X}}(n)|m)}}.}

(2)

The above expression is formatted according to well-established rules of probability theory: choice of synthesis over alternates, m, and a wide range of supporting evidence, n, are multiplied.

A probability cannot be inferred from this expression, however, it is built on a probabilistic framework.

Assuming Effective Learning, It generates Bayesian decisions that are close to optimal and provides an approximation of a probabilistic description.

In keeping with the probabilistic language, we refer to l(X(n)|m) (or just l(n|m)) as the conditional partial similarity..

Assuming Effective Learning, The density function of conditional probability is written as l(n|m), a statistical measure of the likeliness that neuron n's signal was generated by m's object.

Then L is a total likelihood of observing signals {X(n)} coming from objects described by concept-model {Mm}.

Pearson's r (m), probabilistic concepts, known as priors,, contain preconceived notions or biases, high values of r(m) are indicative of expected objects m; their true worth is usually hidden and requires education, like other parameters Sm.

Keep in mind, in the realm of probability theory, It is common practice to assume the independence of evidence when using a product of probabilities.

A product over n is present in the expression for L, X however does not assume that the signals are independent of one another (n).

There is a dependence among signals due to concept-models: each model Mm(Sm,n) predicts expected signal values in many neurons n.

In the course of education, Models of ideas are continually evolving.

Usually, models in their operational forms, Mm(Sm,n), are immutable, and the only variable in learning-adaptation models, Sm.

A new idea will occasionally form within a system, while still using the old one; alternatively, Sometimes, the fusion or elimination of older ideas is necessary.

Because of this, the similarity measure L needs to be adjusted; The reason for this is that as the number of models increases, the better they fit the data.

This is a common issue, The issue is solved by minimizing similarities. A skeptic penalty function (Penalty method) L where p(N,M) is a function that increases as M increases in size, and the rate at which this expansion occurs increases as N decreases.

For example, an asymptotically unbiased maximum likelihood estimation leads to multiplicative p(N,M) = exp(-Npar/2), where Npar is a total number of adaptive parameters in all models (this penalty function is known as Akaike information criterion, for more information and citations, please refer to (Perlovsky, 2001).

Learning involves maximizing similarity L between signals and concepts and estimating model parameters S.

It is important to realize that expression (2) for L takes into account any and all permutations of signals and models.

This can be seen by expanding a sum and multiplying all the terms resulting in MN items, vast quantities.

This is the total number of possible permutations of N signals across all models (M).

The origin of Combinatorial Complexity lies here, which NMF is able to solve thanks to the concept of dynamic logic.

Matching the uncertainty of models with the vagueness or fuzziness of similarity measures is a crucial part of dynamic logic.

Initially, Uncertainty over parameter values, and model uncertainty is very high; Similarly, the vagueness of similarity measures is a concern.

During the course of education, Improvements in model accuracy, and a sharper metric of similarity, increases the similarity's value.

Here's how you can maximize similarity L:.

First, the unknown parameters {Sm} are randomly initialized.

Association variables f(m|n) are then calculated,

{\displaystyle f(m|n)={\frac {r(m)l({\vec {X}}(n|m))}{\sum _{m'=1}^{M}{r(m')l({\vec {X}}(n|m'))}}}}

(3).

If l(n|m) in the learning result becomes conditional likelihoods, then f(m|n) becomes Bayesian probabilities for signal n originating from object m, and the corresponding equation looks a lot like Bayes' formula for a posteriori probabilities. The following is a definition of the NMF's dynamic logic::

{\displaystyle {\frac {d{\vec {S}}_{m}}{dt}}=\sum _{n=1}^{N}{f(m|n){\frac {\partial {\ln l(n|m)}}{\partial {{\vec {M}}_{m}}}}{\frac {\partial {{\vec {M}}_{m}}}{\partial {{\vec {S}}_{m}}}}}}

(4).

{\displaystyle {\frac {df(m|n)}{dt}}=f(m|n)\sum _{m'=1}^{M}{[\delta _{mm'}-f(m'|n)]{\frac {\partial {\ln l(n|m')}}{\partial {{\vec {M}}_{m'}}}}}{\frac {\partial {{\vec {M}}_{m'}}}{\partial {{\vec {S}}_{m'}}}}{\frac {d{\vec {S}}_{m'}}{dt}}}

(5)

We have established the following theorem (Perlovsky 2001):

Theorem.

Equations (3), (4), and (5) define a convergent dynamic NMF system with stationary states defined by max{Sm}L.

Maximum similarity states are, therefore, the MF system's stable equilibrium states.

When similarity measures are expressed as density functions of probability (pdf), or likelihoods, the stationary values of parameters {Sm} are asymptotically unbiased and efficient estimates of these parameters.

Dynamic logic has a linear complexity in N from a computational standpoint.

Instead of using an incremental formula, (3) allows f(m|n) to be recalculated at each iteration of the equations' solution process (5).

A demonstration that similarity L grows with each iteration is included in the proof of the aforementioned theorem. One possible psychological reading of this is that each success satisfies the innate desire to learn more, elevating the individual's mood. The NMF-dynamical logic system takes pleasure in intellectual pursuits.

The task of detecting patterns beneath background noise can be extremely challenging.

When the parameters that determine the precise shape of a pattern are unknown, These values can be determined by adjusting the pattern model's fitting parameters.

However, when pattern positions and directions are unknown, Which portion of the data should be used to fit the model is unclear.

Multiple hypothesis testing is a common method for dealing with this type of issue (Singer et al.

1974).

Due to the exhaustive searching of all possible subset and model combinations, Combinatorial complexity is a hurdle for this approach.