Deep Neuro-Fuzzy Systems with Python: With Case Studies and Applications from the Industry

By Himanshu Singh and Yunis Ahmad Lone

Gain insight into fuzzy logic and neural networks, and how the integration between the two models makes intelligent systems in the current world. This book simplifies the implementation of fuzzy logic and neural network concepts using Python.

You’ll start by walking through the basics of fuzzy sets and relations, and how each member of the set has its own membership function values. You’ll also look at different architectures and models that have been developed, and how rules and reasoning have been defined to make the architectures possible. The book then provides a closer look at neural networks and related architectures, focusing on the various issues neural networks may encounter during training, and how different optimization methods can help you resolve them.

In the last section of the book you’ll examine the integrations of fuzzy logics and neural networks, the adaptive neuro fuzzy Inference systems, and various approximations related to the same. You’ll review different types of deep neuro fuzzy classifiers, fuzzy neurons, and the adaptive learning capability of the neural networks. The book concludes by reviewing advanced neuro fuzzy models and applications. 

What You’ll Learn

• Understand fuzzy logic, membership functions, fuzzy relations, and fuzzy inference
• Review neural networks, back propagation, and optimization
• Work with different architectures such as Takagi-Sugeno model, Hybrid model, genetic algorithms, and approximations
• Apply Python implementations of deep neuro fuzzy system 

Who This book Is For 

Data scientists and software engineers with a basic understanding of Machine Learning who want to expand into the hybrid applications of deep learning and fuzzy logic.

• Language: English
• Publisher: Apress
• Release date: Nov 30, 2019
• ISBN: 9781484253618

Book preview

© Himanshu Singh, Yunis Ahmad Lone 2020

H. Singh, Y. A. LoneDeep Neuro-Fuzzy Systems with Pythonhttps://doi.org/10.1007/978-1-4842-5361-8_1

1. Introduction to Fuzzy Set Theory

Himanshu Singh¹  and Yunis Ahmad Lone²

(1)

Allahabad, Uttar Pradesh, India

(2)

Hyderabad, Andhra Pradesh, India

This chapter sets the foundation for the rest of the book. You will be introduced to soft computing and Fuzzy Systems. You will learn about the Classical and Fuzzy Sets and the differences between them. You will then look at the properties of different sets, and you’ll learn how different operations can be performed on them. This chapter also includes a basic introduction to membership functions, which are then explained in detail in the next chapter. Wherever required, Python code is provided for execution purposes.

Soft Computing and Fuzzy Systems

When you have crisply defined data that is precise and easy to understand, applying hard computing to it is perfect. Hard computing is based on binary logic, classical sets, crisp (precise) systems and software, basic numerical analysis, etc. But when you try to apply this same approach to real-world problems that include imprecise data—maybe the dataset is partially true, it has a lot of approximations, and so on—hard computing fails. The best way to tackle this situation is to use the soft computing approach.

A very basic example is 2+2. In this scenario, you can use hard computing to arrive at 4. But when you change the equation to 2+x, where x ranges from 0 to 5, soft computing always gives better results. Before you move on to understanding exactly what soft computing is, study the flowchart in Figure 1-1. It depicts the difference between hard and soft computing when it comes to problem solving.

../images/479940_1_En_1_Chapter/479940_1_En_1_Fig1_HTML.jpg

Figure 1-1

Hard computing versus soft computing

Soft computing tries to imitate the human mind in order to make decisions. These models have cognitive abilities, which include:

Ability to think

Ability to reason

Ability to organize

Ability to memorize

Ability to recognize

Ability to process

When your data is imprecise (it has partial truths and is full of approximations), soft computing is the best approach. The following are features of soft computing-based problem-solving approaches:

Biologically inspired

Fault tolerant

Full of optimizations

Helps make wiser and more intelligent machines

Helps in achieving robustness, tractability, and lower costs

Heavy computation

Goal driven

Table 1-1 lists the basic constituents of the soft computing approach.

Table 1-1

Basic Constituents of Soft Computing

Now that you have read about the basics of soft computing, turn your attention to understanding Fuzzy Systems. Fuzzy Systems are one of the core parts of soft computing, so you need to understand them. Fuzzy Systems, along with soft computing, make a very strong foundation for an inference system, also called an Adaptive Neuro-Fuzzy Inference System, which is discussed later in this book.

Fuzzy Systems are comprised of Fuzzy Sets and not the normal, classical sets. In these systems, you try to follow Fuzzy Logic, because traditional logic cannot be applied to real-world applications. Let’s look at Fuzzy Systems with the help of an example.

When you drive a car, it’s a combination of pressing the accelerator and the brake. Whenever you speed up, you press the accelerator, and whenever you want to slow down, you press the brakes. Suppose you are talking about self-driving cars. In this scenario, both systems should be managed simultaneously, without any manual intervention.

If you consider traditional logic, it follows classical sets (called crisp sets ). In these sets, the value is equal to 0 or 1. Suppose the functioning is defined as shown Table 1-2.

Table 1-2

Applying Brakes Represented Using Crisp Sets

The problem with this encoding is that 1 represents a full pressing and 0 represents a full release. There is no intermediate response. Suppose a car turns in front of your car. In this scenario, the 1 code will get activated and full brakes will be applied. If that car accelerates and is farther away from you, full brakes will be released and the full accelerator will be pressed (codes 0 and 1 will get activated).

Classical Sets

Classical sets, also called crisp sets, are a collection of objects. Objects can be anything belonging in the real world, and sometimes outside the domain as well. For example:

Cars = {Audi, BMW, Mercedes, Porsche}

This set shows a list of premium cars. You denote a set by using the curly braces, {}.

Once you have effectively defined different sets, you can visualize them as well. A Venn diagram is a visual way to represent sets and their relationships with each other. Figure 1-2 shows a normal Venn diagram.

../images/479940_1_En_1_Chapter/479940_1_En_1_Fig2_HTML.jpg

Figure 1-2

Simple Venn diagram

The circles in Figure 1-2 represent two sets, A and B. All the elements that are part of Set A will be present in Circle A, while all the elements that are part of Set B will be present in Circle B. The circles are called Venn diagrams of Set A and Set B.

Universe of Discourse

All the possible elements that can share a domain, or have similar characteristics, are contained in a set. The set of those elements is called the Universe of Discourse. Once you have this set, you can form various subsets as well. Formally speaking, the Universe of Discourse can be defined as follows:

In every discourse, whether of the mind conversing with its own thoughts, or of the individual in his intercourse with others, there is an assumed or expressed limit within which the subjects of its operation are confined. … Now, whatever may be the extent of the field within which all the objects of our discourse are found, that field may properly be termed the Universe of Discourse. (Boole 1854/1958, p. 42)

Look at the Universe of Discourse with the following example:

$$ E=\left\{ Domain\ of\ Machine\ Learning\right\} $$

Let Set E be the Universe of Discourse. It has the complete domain of Machine Learning. Then the possible sets that can be part of E are:

Set of Machine Learning algorithms

Set of basic statistical methods

Set of neural networks, etc.

Properties of Classical Sets

This section explains the various properties of classical sets. These are:

Membership of elements

Cardinality of sets

Family of sets

Null set

Singleton set

Subset

Superset

Powerset

Countable set

Uncountable set

Membership of Elements

If an element is a part of a set, it is called a member of the set. It is denoted by X ε A, which means that Element X is a member of Set A.

$$ A=\left\{1,2,3,4,5\right\} $$

In the set of integers from 1 to 5, each number is called a member of Set A.

Cardinality of Sets

In a set, if you count the total number of elements present, that number is called the cardinality of the set. It can be denoted by n(A) or |A| or #A.

$$ A=\left\{1,2,3,4,5\right\} $$

This set has a cardinality of 5.

Family of Sets

A set can contain anything. But if a set contains a collection of different sets, then that set is referred to as a family of sets. For example:

$$ A=\left\{\left(1,3,5\right),\left(2,4,6\right),\left(5,10,15\right)\right\} $$

Here, (1, 3, 5), (2, 4, 6), and (5, 10, 15) are individual sets contained in Set A. Therefore, Set A is the family of sets, as shown in Figure 1-3.

../images/479940_1_En_1_Chapter/479940_1_En_1_Fig3_HTML.jpg

Figure 1-3

The Family Set A contains different sets

Null Set

If a set has a cardinality of 0, it’s called a null set or empty set . That means that there are no elements inside it.

A = {} is a null set.

Singleton Set

If a set has a cardinality of 1, it is called a singleton set. That means that there is only one element inside it.

A = {1} is a singleton set.

Subset

Say you have two sets, A and X. If all the elements of X are part of A, then X is called a subset of A. It is represented by X⊂A. Figure 1-4 shows a Venn diagram representing Superset A containing Subset X.

../images/479940_1_En_1_Chapter/479940_1_En_1_Fig4_HTML.jpg

Figure 1-4

Venn diagram representing Superset A and Subset X

Superset

Say you have two sets, A and X. If all the elements of X are part of A, then A is called the superset of X. It is represented by A⊃X. In the Figure 1-4, A is the superset of X.

Powerset

Say you have a set A. A set containing all the possible subsets from A, including the null set, is called the powerset. It is represented by P(A). The cardinality of the powerset of A is 2|A|.

For example:

$$ If\ A=\left\{2010,2011\right\} $$$$ P(A)=\left\{(2010),(2011),\left(2010,2011\right),\left(\right)\right\} $$

As you can see, |A| = 2, while |P(A)| = 4.

Countable Set

A countable set is where for every element can be labeled a unique natural number. Also, by the time you finish labeling the elements, more labels are left in the form of natural numbers, or otherwise you have exhausted all the natural numbers. Therefore, infinite sets can also be countable, but not every time.

Uncountable Set

An uncountable set is where, for every element present, you cannot label a unique natural number. That means, by the time you finish assigning labels, the natural numbers list is exhausted. For example, when you take real numbers, the list of natural numbers will be exhausted much before the labeling of a set of real numbers is finished.

Classical Set Operations

This section looks at some of the operations that can be applied to classical sets:

Union

Intersection

Complement

Difference

Union

A union of two sets merges the values in both sets into one single set.

$$ A=\left\{1,3,5\right\} $$$$ B=\left\{2,4\right\} $$$$ A\cup B=\left\{1,2,3,4,5\right\} $$

Figure 1-5 represents a union between two sets, A and B. The shaded region represents the union part.

../images/479940_1_En_1_Chapter/479940_1_En_1_Fig5_HTML.jpg

Figure 1-5

Union of Sets A and B

Intersection

An intersection of two sets finds the common values present in both sets and makes them one single set.

$$ A=\left\{1,3,5\right\} $$$$ B=\left\{3\right\} $$$$ A\cap B=\left\{3\right\} $$

Figure 1-6 represents an intersection between two Sets A and B. The shaded region represents the intersection part.

../images/479940_1_En_1_Chapter/479940_1_En_1_Fig6_HTML.jpg

Figure 1-6

Intersection of Sets A and B

Complement

A complement of a set is all the values present in the Universe of Discourse, except the ones present in the set.

$$ A=\left\{1,3,5\right\} $$$$ B=\left\{3\right\} $$

If A is the Universe of Discourse for B, then Ac will be all the values except the existing one. That is:

$$ B=\left\{3\right\} $$

Figure 1-7 represents the complement of Set A. If U is the Universe of Discourse, then the shaded region represents the complement of A.

../images/479940_1_En_1_Chapter/479940_1_En_1_Fig7_HTML.jpg

Figure 1-7

Complement of Set A

Difference

Say you have two sets,