VMware Certified Professional 6 Exam Guide (Exam #2V0-642): Comprehensive guide right from basics to advanced VMware Network Virtualization concepts (English Edition)

By Rakesh Kumar Verma

Starting with the very basics of Networking virtualization, this book is a comprehensive guide to help you get certified as a VMware Professional.
This book discusses the relationships between physical and virtual network infrastructure, networking devices, their working concepts and moves on to demonstrating the installation, configuration, administration, and operations performance in VMware NSX environment. The easy to follow explanations along with relevant visual aids like snapshots, tables and relevant figures will help you to practically follow the course of the book with ease.
Initial chapters explore the various components of VMware NSX, its architecture and implementation in the network. Going forward its integration with third-party hardware, applications and services have been discussed extensively. Automation, Monitoring, and role assignments have been covered in concluding sections of the guide thus providing an end-to-end visibility on the topic.
With all the information mentioned in this guide, grasped, and fully understood, you can target cracking the prestigious VMware certification VCP6-NV-2V0-642 successfully.

• Language: English
• Publisher: BPB Online LLP
• Release date: Jun 20, 2022
• ISBN: 9789391392758

Book preview

CHAPTER 1

Basics of NSX-vNetwork Virtualization Platform

Introduction

This chapter starts with a brief introduction of the networking concept, followed by an introduction to network virtualization, and a brief overview of various related concepts. We will then introduce VMware’s NSX-V network virtualization solution that in turn allows deploying and managing software-defined networking stack. Post-introduction of VMware’s NSX-V, we will discuss various features and services offered by NSX. VMware announced the end of life for NSX data center (NSX-V) in 2018, and as of mid-2020, it is actively encouraging people to migrate to NSX-T. We will also see the threshold or maximum permissible configuration. This chapter is of prime importance to the readers, as it covers the very basics of Network Virtualization techniques and provides a fair understanding of the terms being used in later chapters to have a good grip over the network virtualization technology facilitated by VMware.

Structure

In this chapter, we will discuss the following topics:

Networking concepts overview

Introducing network virtualization

Concepts of network virtualization

Introducing the NSX-V network virtualization platform

NSX features and services

NSX configuration maximums

Conclusion

Objective

After studying this unit, the reader will be able to understand:

Need of network virtualization

Concepts behind network virtualization

NSX-V as a network virtualization solution

NSX-V features and permissible threshold configuration

1.1 Networking concepts overview

Before we step into the sea of network virtualization, it is better to recall our old networking concepts to grasp the concepts in a better and more efficient way. We will start the discussion with basics and gradually move on to the exact content as mentioned in the objective of this chapter.

Physical network: It is a network of physical machines that are connected so that they can send data to and receive data from each other? VMware ESXi runs on a physical machine.

Virtual network: It is a network of virtual machines running on a physical machine that are connected logically to each other so that they can trans-receive data from each other? Virtual machines can be connected to virtual networks.

Physical Ethernet switch: It manages network traffic between machines on the physical network. A switch has multiple ports, each of which will be connected to a single machine or another switch on the network. Each port can be configured to behave in certain ways depending on the needs of the machine connected to it. The switch will learn which hosts are connected to which of its ports and use that information to forward traffic to the correct physical machines. Switches are the core of a physical network. Multiple switches can be connected together to form larger networks.

vSphere standard switch: It works much like a physical Ethernet switch. It detects which virtual machines are logically connected to each of its virtual ports and uses that information to forward traffic to the correct virtual machines. A vSphere standard switch can be connected to physical switches by using physical Ethernet adapters, also referred to as uplink adapters, to join virtual networks with physical networks. This type of connection is similar to connecting physical switches together to create a larger network. Even though a vSphere standard switch works much like a physical switch, it does not have some of the advanced functionality of a physical switch.

Standard port group: It specifies port configuration options, such as bandwidth limitations and VLAN tagging policies for each member port. Network services connect to standard switches through port groups. Port groups define how a connection is made through the switch to the network. Typically, a single standard switch is associated with one or more port groups.

vSphere distributed switch: It acts as a single switch across all associated hosts in a data center to provide centralized provisioning, administration, and monitoring of virtual networks. You configure a vSphere distributed switch on the vCenter Server system, and the configuration is populated across all hosts that are associated with the switch. This lets virtual machines to maintain consistent network configuration as they migrate across multiple hosts.

Host proxy switch: It is a hidden standard switch that resides on every host that is associated with a vSphere distributed switch. The host proxy switch replicates the networking configuration set on the vSphere distributed switch to the particular host.

Distributed port: It is a port on a vSphere distributed switch that connects to a host’s VMkernel or to a virtual machine’s network adapter.

Distributed port group: It is a port group associated with a vSphere distributed switch and specifies port configuration options for each member port. Distributed port groups define how a connection is made through the vSphere distributed switch to the network.

NIC teaming: NIC teaming occurs when multiple uplink adapters are associated with a single switch to form a team. A team can either share a load of traffic between physical and virtual networks among some or all of its members or provide passive failover in the event of a hardware failure or a network outage.

VLAN: VLAN enables a single physical LAN segment to be further segmented so that groups of ports are isolated from one another as if they were on physically different segments. The standard is 802.1Q.

VMkernel TCP/IP networking layer: The VMkernel networking layer provides connectivity to hosts and handles the standard infrastructure traffic of vSphere vMotion, IP storage, Fault Tolerance, and Virtual SAN.

IP storage: Any form of storage that uses TCP/IP network communication as its foundation. iSCSI can be used as a virtual machine datastore, and NFS can be used as a virtual machine datastore and for direct mounting of .ISO files, which are presented as CD-ROMs to virtual machines.

TCP segmentation offload: TCP segmentation offload (TSO) allows a TCP/IP stack to emit large frames (up to 64 kB) even though the maximum transmission unit (MTU) of the interface is smaller. The network adapter then separates the large frame into MTU-sized frames and prepends an adjusted copy of the initial TCP/IP headers.

1.2 Introduction to network virtualization

The invention and easy reach of the Internet has resulted in the virtualization of almost all aspects of our life. Be it our workspace, be it shopping, be it education, be it entertainment, or be it our day-to-day normal activities, everything is transitioning to a virtual world. The key enabler for all virtualizations is the connectivity via some network, that is, the Internet and various computer networking technologies. Looking at the pace of virtualization, it turns out that computer networking itself has to be virtualized. Virtualization in networking is not a new concept. For example, Virtual channels in X.25-based telecommunication networks and all subsequent networks allow multiple users to share a large physical channel. Virtual local area networks (VLANs) allow multiple departments of an organization to share a common physical LAN with isolation. Similarly, virtual private networks (VPNs) allow organizations and employees to use public networks without compromising on the quality and security of the network. Mainly with the advancement of technologies such as cloud computing, network virtualization has also gained momentum at a new pace. Several new standards have been developed and are being developed:

Figure 1.1: Virtualized equivalent of network services

In short and simple words, with network virtualization, the functional equivalent of a network hypervisor reproduces Layer 2 to Layer 7 networking services (for example, switching, routing, firewalling, and load balancing) in software. These services can then be programmatically assembled in any arbitrary combination, producing unique, isolated virtual networks in a matter of seconds. Refer to figure 1.1, which shows the virtualized equivalent of networking services.

1.3 Need of network virtualization

There are many reasons why we need to virtualize network resources. The five most common reasons are as follows:

Sharing: When a resource is too big for a single user, it is best to divide it into multiple virtual pieces to have optimum utilization of available resources. With the advance technology, we have multi-core processors. Each processor can run multiple virtual machines (VMs), and each machine can be used by a different user. The same philosophy applies to high-speed links and large-capacity disks.

Isolation: Multiple users sharing a common resource requires some kind of isolation from each other. That means users using one virtual component should not be able to monitor the activities or interfere with the activities of other users.

Aggregation: If the resource is too small, it is possible to construct a large virtual resource that behaves like a single large resource. This is the case with storage, where a large number of inexpensive unreliable disks can be used to make up large reliable storage.

Dynamics: Often, resource requirements change fast due to user mobility, and a way to reallocate the resource quickly is required. This is easier with virtual resources than with physical resources.

Ease of management: Last but probably the most important reason for virtualization is the ease of management. Virtual devices are easier to manage because they are software-based and expose a uniform interface through standard abstractions.

1.4 Concepts of network virtualization

A computer network starts with a network interface card (NIC) in the host, which is connected to a Layer 2 (L2) network (Ethernet, WiFi, and so on) segments. Several L2 network segments may be interconnected via switches (also known as bridges) to form an L2 network, which is one subnet in a Layer 3 (L3) network (IPv4 or IPv6). Multiple L3 networks are connected via routers (also known as gateways) to form the Internet. A single data center may have several L2/L3 networks. Several data centers may be interconnected via L2/L3 switches. Each of these network components—NIC, L2 network, L2 switch, L3 networks, L3 routers, data centers, and the Internet—needs to be virtualized. There are multiple, often competing, standards for the virtualization of several of these components. Several new ones are being developed. Refer to figure 1.2 to have clear insight about Network Interface Card abbreviated as NIC:

Figure 1.2: Simple NIC (Network Interface Card)

We all are aware of OSI layers and their functionalities. Just to refresh our previous learnings, refer to table 1.1, OSI seven-layer model:

Table 1.1: OSI seven-layer model

Network virtualization is the virtualization of network resources using software and networking hardware. It enables the faster provisioning and deployment of networking resources. Network virtualization lays the foundation for software-defined networking (SDN), which allows instant deployment of services to be offered to the end-users. Various services such as VPN, DHCP, DNS, load balancers, and so on can be instantly provisioned and deployed because of the software aspect of network virtualization.

The networking hardware allows for physical connectivity, and the software provides the networking logical brain for a feature-rich network service offering. Network virtualization allows for the consumption of simplified logical networking devices and services that are completely abstracted from the complexities of the underlying physical network. Network virtualization is key for a software-defined data center (SDDC).

Each computer system needs at least one L2 NIC (Ethernet card) for communication. Therefore, each physical system has at least one physical NIC. However, if we run multiple VMs on the system, each VM needs its own virtual NIC.

As shown in figure 1.1, one way to solve this problem is for the hypervisor software that provides processor virtualization also implements as many virtual NICs (vNICs) as there are VMs. These vNICs are interconnected via a virtual switch (vSwitch) that is further connected to the physical NIC (pNIC). Multiple pNICs are connected to a physical switch (pSwitch).

In figure 1.1, virtual objects are shown by dotted lines, whereas physical objects are shown by solid lines. There are other different approaches for NIC virtualization. For the sake of simplicity, we will keep our discussion to the one proposed by VM software vendors. This virtual Ethernet bridge (VEB) approach has the virtue of being trans-parent and straightforward:

Figure 1.3: Simplified approach to NIC virtualization.

pNIC vendors (or pNIC chip vendors) have their own solution, which provides virtual NIC ports using single-route I/O virtualization (SR-IOV) on the peripheral-component interconnect (PCI) bus. The switch vendors (or pSwitch chip vendors) have yet another set of solutions that provide virtual channels for inter VM communication using a virtual Ether-net port aggregator (VEPA), which passes the frames simply to an external switch that implements inter VM communication policies and reflects some traffic back to other VMs in the same machine.

As we discussed about pSwitch, pNIC, and vNIC, refer to figure 1.4 to have more insights:

Figure 1.4: pNIC to VNIC mapping.

A typical Ethernet switch has 32–128 ports. The number of physical machines that need to be connected on an L2 network is typically much larger than this. Therefore, several layers of switches need to be used to form an L2 network. IEEE Bridge Port Extension standard 802.1BR, shown in figure 1.3, allows forming a virtual bridge with a large number of ports using port extenders that are simple relays and may be physical or virtual (like a vSwitch):

Figure 1.5: IEEE 802.1BR bridge port extension

Before we dig down a bit more into VMware NSX network virtualization, let us go over some of the key concepts of network virtualization and software-defined networking:

Decoupling: The word decoupling literally means separate (something) from something else. In the context of network virtualization, it means the software works independently of the networking hardware that physically interconnects the infrastructure. Any networking hardware that can inter-op with the software is always going to enhance the functionality, but it is not necessary. Remember that throughput on the wire will be always limited by the network hardware performance.

Control plane: The decoupling of software and networking hardware allows controlling the network better because all the logic resides in the software. This control aspect of the network is called the control plane. The control plane provides the means to configure, monitor, troubleshoot, and allow automation against the network.

Data plane: The networking hardware forms the data plane where all the data is forwarded from source to destination. The management of data resides in the control plane; however, the data plane consists of all the networking hardware whose primary function is to forward traffic over the wire from source to destination.

The data plane holds all the forwarding tables that are constantly updated by the control plane. This also prevents any traffic interruptions if there is a loss of the control plane because the networking hardware, which constitutes the data plane, will continue to function without interruptions.

1.5 Application programming interface (API)

The API is one of the important aspects of a virtualized network and allows for true software-defined networking by instantly changing the network behavior. With the API, one can now instantly deploy rich network services in the existing network. Network services, such as VPN, firewall, load balancers, and so on, can all be deployed on the fly by means of an API.

1.6 Introducing the NSX-V network virtualization platform

VMware NSX is a software-defined network (SDN) solution that mimics the virtual machine deployment model by programmatically reproducing complex networks and security. NSX virtualizes the network and security functions. NSX has no hardware dependency and reproduces the network model for virtual workloads in software.

In simple words, VMware’s NSX-V software abstracts the underlying physical network by introducing a software layer that makes it easy to consume network resources by creating multiple virtual networks. NSX-V also allows for deploying multiple logical network services on top of the abstracted layer.

VMware acquired NSX from Nicira in July 2012. Nicira’s NSX was primarily being used for network virtualization in a Xen-based hypervisor.

VMware provides two versions of NSX are as follows:

NSX for vSphere (NSX-V): Supports hypervisors running ESXi

NSX for Multi-Hypervisor (NSX-MH): Supports multiple hypervisors, including ESXi, XEN Server, Redhat KVM, and Hyper-V.

The two versions have many similarities but also are dissimilar in some aspects. This chapter focuses on the NSX-VMware (NSX-V) version of NSX only. NSX-V will be referred to as NSX for the rest of the book.

1.7 NSX features and services

Before we deep dive into the various remarkable features and services offered by NSX, let us have an overview of those features in brief. We shall be studying these features and services in depth in subsequent chapters.

Logical switching: NSX allows the ability to create Layer 2 (L2) and Layer 3 (L3) logical switching. This, in turn, helps in workload isolation and separation of IP addresses between logical networks. NSX can create logical broadcast domains in the virtual space. This helps in preventing the need to create any logical networks on the physical switches. Technically speaking, we are no longer limited to 4,096 physical broadcast domains (VLANs).

NSX gateway services: The interconnection between logical and physical networks is facilitated by the Edge gateway services, namely, the NSX gateway service. In simple words a virtual machine connected to a logical network can send and receive traffic directly to the physical network through the gateway.

Logical routing: NSX supports the creation of multiple virtual logical networks that may be used by multiple virtual machines. Logical routing helps in routing the traffic across different logical switches or even between a logical switch and public networks. Logical routing can be extended to perform east-west routing that saves unnecessary network hops, and thus, increasing network efficiency. Logical routers can also provide north-south connectivity that allows access to workloads living in the physical networks. Logical routers also help avoid hair-pinning of traffic, and hence, increases network efficiency.

Logical firewall: A firewall is a network security system that monitors and controls incoming and outgoing network traffic based on predetermined security rules. A firewall typically establishes a barrier between a trusted internal network and untrusted external network, such as the Internet. NSX offers an additional service similar to the firewall that allows the option of a distributed logical firewall or an Edge firewall for use within software-defined networking architecture. A virtual firewall runs in the kernel of the ESXi host. A distributed logical firewall allows building rules based on attributes that includes IP addresses, VLANs, virtual machine names, and vCenter objects too.

Extensibility: In order to have seamless connectivity integration and ensure smooth interpretability between VMware NSX and other services, there are third-party VMware partner solutions that allow a vendor choice in multiple service offerings. There are many VMware partners who offer solutions such as traffic monitoring, IDS, and application firewall services that can integrate directly into NSX. This indirectly enhances management and end-user experience.

Load balancer: NSX Edge offers a variety of services, and the logical load balancer is one of them. The logical load balancer distributes incoming requests among multiple servers to allow for load distribution.

To ensure maximum uptime of hosted application, the logical load balancer can also be used in a high availability (HA) mechanism.

Virtual private networks (VPN): A virtual private network extends a private network across a public network and enables users to send and receive data across shared or public networks as if the computing devices are directly connected to the private network. The NSX Edge offers the VPN service that allows provisioning secure encrypted connectivity for end users to the applications and workloads. Edge VPN service offers SSL-VPN plus it allows for user access and IPSEC site-to-site connectivity, which enables two sites to be interconnected securely.

Dynamic Host Configuration Protocol (DHCP): NSX Edge offers DHCP services that allow IP address pooling and static IP assignments. The DHCP service can also relay DHCP requests to the existing DHCP server as well. The NSX Edge DHCP service can relay any DHCP requests generated from the virtual machines to a pre-existing physical or virtual DHCP server without any interruptions.

Domain name system (DNS): DNS translates domain names into IP addresses, hence, allowing to access an Internet location by its domain name. NSX Edge offers a DNS relay service that can relay any DNS requests to an external DNS server.

Service composer: The service composer allows you to allocate network and multiple security services to security groups. Virtual machines that are part of these security groups are automatically allocated the services.

Data security: NSX data security provides visibility into sensitive data, ensures data protection, and reports back on any compliance violations. A data security scan on designated virtual machines allows NSX to analyze and report back on any violations based on the security policy that applies to these virtual machines.

After having learned about the NSX features and services, we can summarize NSX use cases as mentioned in table 1.2:

Table 1.2: NSX use cases

The main themes for NSX deployments are security, IT automation, and application continuity:

Figure 1.6: Themes for NSX deployments; security, automation, and application continuity

The use cases outlined previously are key reasons behind the rising demand for NSX. NSX is uniquely positioned to solve these challenges as it can bring networking and security closest to the workload itself and carry the policies along with the workload.

NSX requires a vSphere environment with vCenter to coordinate changes, including deploying, configuring, and removing NSX components and services. NSX and vCenter have a tight integration. NSX is not an NFV solution but an SDN solution. NSX does virtualize network and security functions, but it does so by using a methodology that goes beyond just replicating the functionality, and all the caveats, the physical network or security appliance provides.

1.8 NSX network and security functions

Following are the summarized network and security functions that NSX can provide. In simple words, we have mapped the functions equivalent with its of NSX components or feature:

Table 1.3: Functional equivalent with NSX component/feature

1.9 NSX configuration maximums

Before we conclude the basics of VMware NSX-V, let us have a look at what the NSX configuration maximums are. Some of these limits are hard limits, whereas most of them are soft limits, beyond which VMware does not support such configurations.

For example, if the number of concurrent connections per Edge gateway is exceeded beyond the mentioned limit, it will affect the gateway’s performance. Note that it will not cause it to halt or reject new connections but will impact the performance part.

The following table shows the limits for various components:

Table 1.4: Showing NSX-vCenter maximums for various components

A transport zone defines the scope of a logical switch and can span one or more vSphere clusters. The following table shows the limits for switching maximums:

Table 1.5: Showing NSX—switching maximums for various components

The following table shows the limits for Distributed Logical Firewall Maximums:

Table 1.6: Showing NSX-DLF maximums for associated policies and components

The following table shows the limits for Distributed Logical Router (DLR) maximums:

Table 1.7: Showing NSX-DLR maximums for various components

The following table shows the limits for NSX Edge Services Gateway (ESG) maximums:

Table 1.8: Showing NSX-ESG maximums for various rules, policies, and components

The following table shows the limits for DHCP and VPN Service maximums:

Table 1.9: Showing NSX-DHCP and VPN Service maximums for various configurations

One of the unique NSX features includes cross-vCenter networking and security, which allows the management of multiple vCenter NSX environments using a primary NSX manager. This not only allows centralized management but also extends one or more services and features across multiple vCenter environments.

The following table shows the limits for Multi-vCenter NSX supported features:

Table 1.10: Showing Multi-vCenter limits for various components

Conclusion

We started this chapter with a brief introduction to networking terminologies, followed by an introduction to network virtualization and software-defined networking. We discussed the concepts of network virtualization and introduced VMware’s NSX network virtualization platform. We then discussed different NSX features and services, including logical switching, logical routing, Edge gateway services, extensibility, service composer, and data security. We also briefly discussed the multi-vCenter NSX feature. We ended the chapter with configuration maximums for NSX. With this reader is supposed to acquire the basic skill related to NSX, its various features, and services along with the maximum values or policies or components supported. Moving forward, we will learn more about NSX core components in upcoming chapters.

Questions

Why do we need virtualization at the network level?

What can all network components be virtualized to realize a fully virtualized network?

Answer the following multiple-choice question:

How many NSX managers are required per NSX -vCenters

1

3

None of the above

What is the maximum number of DRS clusters an NSX-V center support?

2

4

6

12

How many hosts per cluster can be supported by an NSX-vCenter?

4

10

28

32

What are the maximum logical switches and switch ports an NSX—switch can have?

1,200 and 2,400

1,000 and 2,000

10,000 and 5,000

5,000 and 5,000

Select the maximum permissible DLR per host.

500

1,000

1,200

1,500

What is the total number of Edge service gateways per NSX Manager?

100

1,000

200

2,000

Select the max. interfaces and sub interfaces that can be configured per ESG on a trunk.

10 and 200

20 and 400

50 and 100

40 and 120

Answer

3 - i a

3 - ii d

3 - iii d

3 - iv c

3 - v b

3 - vi d

3 - vii a

CHAPTER 2

NSX Core Components

This chapter starts with an introduction of NSX core components, followed by a detailed discussion of its core components. NSX is a network hypervisor that provides a platform to manage virtualized network deployments. We will discuss the three different planes, which are the data, control, and management plane, and how each of these NSX core components fits into the NSX architecture. The NSX architecture has a built-in separation of data, control, and management layers. This separation allows the architecture to grow and scale without impacting workload. We will be discussing each layer in more depth in this chapter. Gradually, we will extend the discussion to VXLAN architecture and transport zones that allow us to create and extend overlay networks across multiple clusters. Lastly, we will be looking at NSX Edge and the distributed firewall in greater detail and take a look at the newest NSX feature: multi-vCenter or cross-vCenter NSX deployment. The concepts discussed in this chapter forms a foundation to understand various network-level implementation. In upcoming chapters, we’ll be using these concepts seamlessly.

Structure

In this chapter, we will discuss the following topics:

Introduction to NSX core components

NSX Architecture

NSX manager

NSX controller clusters

NSX vSwitch

NSX Edge Service Gateway

VXLAN architecture overview

Transport zones

Distributed firewall

Cross-vCenter NSX

Objective

After studying this unit, the reader will be able to explain:

NSX core components and their work

Functional interdependencies among NSX core components

Multi-vCenter or cross-vCenter NSX

2.1 Introduction to NSX core components

Now that we have already gone through the basics of NSX-V network virtualization in Chapter 1: Basics of NSX-vNetwork Virtualization Platform, it is time to go a little bit far and explore various NSX core components.

As we know, NSX is a network hypervisor that provides a platform to manage virtualized network deployments. The foundational core components of NSX are divided across three different planes. The core components of any NSX deployment consist of the following:

NSX manager that resides in Management Plane

Controller clusters and sits in Control Plane

Hypervisor kernel modules and sits in Data Plane

Each of these is crucial for NSX deployment; however, they are isolated to a certain extent allowing for resiliency during the failure of multiple components. In case if the controller clusters fail, virtual machines will still be able to communicate seamlessly with each other without any network disruption. Owing to this reason, it is always preferred that NSX components are always deployed in a clustered environment. That means they are protected by vSphere HA.

The following figure represents NSX architecture and shows how the three planes are interlinked with each other. Management planes instruct the network device what it will be doing, the control plane says how it will be going to perform or do that, and finally, the data plane is used by a network device to perform or execute the operation.

Figure 2.1: NSX architectural schema

Before we move further, let us have a quick introduction to the different planes we discussed in the previous section.

2.1.1 Management plane and consumption platforms

Management planes are those who are responsible for the management traffic of any device. It helps in the configuration and management of any network device. Management plane instructs control plane to enable control plane to provide support for data plane traffic. The management plane primarily consists of the NSX manager. The NSX manager is the management plane for the NSX ecosystem. The management plane directly interacts with the control plane and the data plane. There is a 1:1 relationship between an NSX manager and a vCenter. NSX manager provides configuration and orchestration of:

Logical networking components—logical switching and routing

Networking and Edge services

Security services and distributed firewall

All security services, whether built-in or third-party, are deployed and configured by the NSX management plane. The management plane provides a single window for viewing services availability. NSX manager also provides REST API entry points to automate consumption. This flexible architecture allows for automation of all configuration and monitoring aspects via any cloud management platform, security vendor platform, or automation framework.

2.1.2 Control plane

As the name indicates, the control plane is used to make data traffic flow from source to destination. It is the control plane that makes two MAC address tables at Layer 2 or routing table at Layer 3 with the help of various routing protocols such as Open Shortest Path First (OSPF), Enhanced Interior Gateway Routing Protocol (EIGRP), Border Gateway Protocol (BGP), and so on. The control plane supports the data plane; all the routing tables are present in the routing information base (RIB) table of the network device.

The NSX controller is a key part of the NSX control plane. It is logically separated from all data plane traffic. NSX controller nodes are deployed in a cluster of odd number instances to ensure scalability and high availability (HA). In addition to the controller, the control VM, provides the routing control plane that allows the local forwarding in ESXi and allows dynamic routing between ESXi and north-south routing provided by Edge VM. Note that data plane traffic never traverses the control plane component. NSX controller that manages the state of virtual networks also enables overlay networks (VXLAN) that are multicast-free. The controllers also keep track of all information about the virtual machines, hosts, and VXLAN networks and can perform Address Resolution Protocol (ARP) suppression as well.

2.1.3 Data plane

When RIB is populated, the content of RIB is copied to the FIB table, which is used by the network device to forward traffic; with the help of the data plane, any network device can send the traffic at the exit interface after consultation of FIB. It is better to recall about RIB and FIB. The forwarding information base (FIB) is the actual information that a routing/switching device uses to choose the interface that a given packet will use for egress. Each FIB is programmed by one or more RIB. The RIB is a selection of routing information learned via static definition or a dynamic routing protocol.

The NSX data plane mainly consists of the NSX logical switch. The NSX logical switch is part of the vSphere distributed switch, and it is created when a VXLAN network is created. The logical switch and other NSX services are enabled at the hypervisor kernel level after the installation of the hypervisor kernel modules (VIBs). This logical switch is key in enabling overlay networks that are able to encapsulate and send traffic over existing physical networks. It also allows for gateway devices that allow L2 bridging between virtual and physical workloads.

The data plane receives its updates from the control plane as hypervisors maintain local virtual machine and VXLAN (logical switch) mapping tables as well. The vSwitch in NSX for vSphere is based on the Virtual Distributed Switch (VDS) with additional components added to enable rich services. The NSX VDS abstracts the physical network, providing access-level switching in the hypervisor. This is central to network virtualization as it enables logical networks that are independent of physical constructs (for example, VLANs). The data plane also consists of gateway devices that can provide communication from the logical networking space to the physical network (for example, VXLAN to VLAN). This functionality can happen at either L2 (NSX bridging) or at L3 (NSX routing).

2.2 NSX architecture

NSX architecture (refer to figure 2.1) comprises the three planes that we just studied. Following is the simplified schema, refer to figure 2.2 NSX components, of NSX showing its main components, which are based on three planes:

The components of NSX are as follows:

NSX manager

NSX controller

NSX vSwitch

NSX Edge service gateway

Figure 2.2: NSX components

2.2.1 NSX manager

NSX manager, as the name indicates, lies in the management plane of NSX. NSX manager is a virtual machine and IP-based device. In NSX, vCenter communicates with the NSX manager via HTTPS (SSL and TCP Port 443) protocol.

NSX manager provides the centralized management plane for NSX for vSphere and has a one-to-one mapping to vCenter server workloads.

NSX manager performs the following functions:

It provides a single point of configuration and the REST API entry-points for NSX in a vSphere environment.

It deploys NSX controller clusters, Edge distributed routers, Edge service gateways in the form of OVF appliances, guest introspection services, and so on.

It prepares ESXi hosts for NSX by installing VXLAN, distributed routing and firewall kernel modules, and the User World Agent (UWA).

It communicates with NSX controller clusters over REST and with ESXi hosts over the RabbitMQ message bus. This internal message bus is specific to NSX for vSphere and does not require the setup of additional services.

Generates certificates for the NSX controller instances and ESXi hosts to secure control plane communications with mutual authentication.

The NSX manager vCPU and memory requirement planning are dependent on NSX release, as shown in table 2.1:

Table 2.1: Showing NSX release version mapping with resource requirements

The NSX version at the time of writing is 6.3 and only supports 1:1 vCenter connectivity. NSX manager data (for example, system configuration, events, and audit log tables) can be backed up at any time by performing an on-demand or scheduled backup from the NSX manager GUI. Restoring a backup is only possible on a freshly deployed NSX manager appliance that can access one of the previously backed-up instances.

It is recommended to leverage standard vSphere functionalities (for example, vSphere HA) to ensure that the NSX manager can be dynamically moved in case its ESXi hosts encounter a failure. Note that such a failure scenario would only impact the NSX management plane; the already deployed logical networks would continue to operate seamlessly.

SSL is disabled by default in NSX software release 6.0. In order to ensure confidentiality of the control plane communication, it is recommended to enable SSL. This can be accomplished through an API call. SSL is enabled by default from the 6.1 release onward.

It is always recommended to deploy NSX manager from the OVA template, which creates a unique UUID. In a cross-vCenter environment, each NSX manager needs to have its own unique UUID.

2.2.2 NSX controller

As evident from the name itself, the NSX controller (along with Distributed Router Control VM) resides in NSX Control Plane. The NSX controller contains the Layer 2 control plane, and with the help of the control VM, it is also able to handle the Layer 3 control plane.

An NSX controller performs the following functions:

Provides the control plane to distribute VXLAN and logical routing information to ESXi hosts.

Includes nodes that are clustered for scale-out and high availability.

Slices network information across cluster nodes for redundancy.

Removes the requirement of VXLAN Layer 3 multicast in the physical network.

Provides ARP suppression of broadcast traffic in VXLAN networks.

NSX control plane communication occurs over the management network. Controllers are deployed as virtual appliances and should be deployed in the same vCenter where the NSX manager is connected to.

NSX controller supports an ARP suppression mechanism. This, in turn, reduces the need to flood ARP broadcast requests across an L2 network domain where virtual machines are connected.

Equal workload division is realized by using a mechanism called the slicing mechanism amongst all controller nodes. In simple words, if one controller node goes down due to any reason or if it fails, then the other nodes have reassigned the tasks that were owned by the failed node, and hence, operational status is not compromised. Refer to figure 2.3 for pictorial insight to have a more clear understanding:

Figure 2.3: Showing slicing of controller cluster node roles

Each node in the controller cluster is identified by a unique IP address. When an ESXi host establishes a control-plane connection with one member of the cluster, a full list of IP addresses for the other members is passed down to the host. This is how the ESXi host comes to know which all members of the clusters relate to the controller cluster at any given instant of time. In the case of failure of a controller node, the slices owned by that node are reassigned to the remaining members of the cluster.

In order for this mechanism to be smooth and deterministic, one of the controller nodes is elected as a master for each role. This master is responsible for the following:

Informing the ESXi hosts about the failure of the cluster node. Accordingly, they update their internal node ownership mapping for seamless operations

Allocating slices to individual controller nodes

Determining when a node has failed

Reallocating the slices to the other nodes

NSX version 6.3 only supports up to three controller nodes. Ensure that the controller clusters are deployed on a storage system that has a peak write latency of less than 300 ms and a mean latency of less than 100 ms. Slow disks can cause a controller to become unstable and can cause downtime.

2.2.3 NSX vSwitch

It is present in the NSX data plane and is integrated into the kernel code of the ESXi host. It is used to handle logical switches, Distributed Logical Routers, and Distributed firewalls. Refer to figure 2.4 showing the layers and the functional modules associated with them. Switches receive the Layer 2 and Layer 3 control plane information from the NSX controller and also receive the security information directly from the NSX manager.

These are directly connected to virtual machines, which, in turn, facilitate networking services such as Layer 2 connectivity, routing, firewall services, and so on. In order to route the information packets within the physical network, NSX vSwitches can make use of overlay protocols such as Virtual exextensible Local Area Network (VXLAN):

Figure 2.4: Simplified NSX layers and components schema

VXLAN facilitates networking functions such as encapsulation, routing, and firewall in the hypervisor with line-rate performance. It comprises the combination of vDS and hypervisor kernel modules configured in each host by the NSX manager.

VMware NSX requires the vSphere environment to run the vSphere Distributed Switch (vDS), which is only found with Enterprise Plus licensing. The NSX vSwitch is the combination of the vDS and kernel modules to provide extra functionality such as VXLAN, DLR, and DFW.

2.2.4 NSX Edge service gateway

It is a specialized virtual appliance dedicated to the communication between the physical and virtual networks created within NSX. It is the virtual appliance used to provide services such as IPSEC VPN, NAT, and load balancing. The Edge services gateway is deployed as a virtual machine from the NSX manager, which is accessed using the vSphere Web client. It must be noted that these services are not available to NSX vSwitch. The number of Edge appliances,