The following sections describe the components in the solution and how they are relevant to the network virtualization design.

NSX for vSphere can be consumed by the cloud management platform (CMP), represented by vRealize Automation, by using the NSX REST API and the vSphere Web Client.

NSX for vSphere is consumed by vRealize Automation. NSX offers self-service provisioning of virtual networks and related features from a service portal. Details of the service requests and their orchestration are outside the scope of this document and can be referenced in the Cloud Management Platform Design document.

NSX for vSphere offers a powerful management interface through its REST API.

A client can read an object by making an HTTP GET request to the object’s resource URL.

A client can write (create or modify) an object with an HTTP PUT or POST request that includes a new or changed XML document for the object.

A client can delete an object with an HTTP DELETE request.

The NSX Manager component provides a networking and security plug-in in the vSphere Web Client. This plug-in provides an interface to consuming virtualized networking from the NSX Manager for users that have sufficient privileges.

Consumption Method Design Decisions

Decision ID

Design Decision

Design Justification

Design Implications


For the shared edge and compute cluster NSX instance, end user access is accomplished by using vRealize Automation services. Administrators use both the vSphere Web Client and the NSX REST API.

vRealize Automation services are used for the customer-facing portal. The vSphere Web Client consumes NSX for vSphere resources through the Network and Security plug-in. The NSX REST API offers the potential of scripting repeating actions and operations.

Customers typically interact only indirectly with NSX from the vRealize Automation portal. Administrators interact with NSX from the vSphere Web Client and API.


For the management cluster NSX instance, consumption is only by provider staff via the vSphere Web Client and the API.

Ensures that infrastructure components are not modified by tenants and/or non-provider staff.

Tenants do not have access to the management stack workloads.

NSX Manager provides the centralized management plane for NSX for vSphere and has a one-to-one mapping to vCenter Server workloads.

NSX Manager performs the following functions.

Provides the single point of configuration and the REST API entry-points for NSX in a vSphere environment.

Deploys NSX Controller clusters, Edge distributed routers, and Edge service gateways in the form of OVF appliances, guest introspection services, and so on.

Prepares ESXi hosts for NSX by installing VXLAN, distributed routing and firewall kernel modules, and the User World Agent (UWA).

Communicates with NSX Controller clusters over REST and with hosts over the RabbitMQ message bus. This internal message bus is specific to NSX for vSphere and does not require setup of additional services.

Generates certificates for the NSX Controller instances and ESXi hosts to secure control plane communications with mutual authentication.

An NSX Controller performs the following functions.

Provides the control plane to distribute VXLAN and logical routing information to ESXi hosts.

Includes nodes that are clustered for scale-out and high availability.

Slices network information across cluster nodes for redundancy.

Removes requirement of VXLAN Layer 3 multicast in the physical network.

Provides ARP suppression of broadcast traffic in VXLAN networks.

NSX control plane communication occurs over the management network.

NSX Controller Design Decision

Decision ID

Design Decision

Design Justification

Design Implications


Deploy NSX Controller instances in Universal Cluster mode with three members to provide high availability and scale. Provision these three nodes through the primary NSX Manager instance.

The high availability of NSX Controller reduces the downtime period in case of failure of one physical host.

The secondary NSX Manager will not deploy controllers.

The controllers from the primary NSX manager will manage all secondary resources.

The NSX data plane consists of the NSX virtual switch. This virtual switch is based on the vSphere Distributed Switch (VDS)with additional components to enable rich services. The add-on NSX components include kernel modules (VIBs) which run within the hypervisor kernel and provide services such as distributed logical router (DLR) and distributed firewall (DFW), and VXLAN capabilities.

The NSX virtual switch abstracts the physical network and provides access-level switching in the hypervisor. It is central to network virtualization because it enables logical networks that are independent of physical constructs such as VLAN. Using an NSX virtual switch includes several benefits.

Supports overlay networking and centralized network configuration. Overlay networking enables the following capabilities.

Facilitates massive scale of hypervisors.

Because the NSX virtual switch is based on VDS, it provides a comprehensive toolkit for traffic management, monitoring, and troubleshooting within a virtual network through features such as port mirroring, NetFlow/IPFIX, configuration backup and restore, network health check, QoS, and more.

NSX logical switches create logically abstracted segments to which tenant virtual machines can be connected. A single logical switch is mapped to a unique VXLAN segment and is distributed across the ESXi hypervisors within a transport zone. The logical switch allows line-rate switching in the hypervisor without the constraints of VLAN sprawl or spanning tree issues.

The NSX distributed logical router (DLR) is optimized for forwarding in the virtualized space, that is, forwarding between VMs on VXLAN- or VLAN-backed port groups. DLR has the following characteristics.

High performance, low overhead first hop routing

Scales with number of hosts

Up to 1,000 Logical Interfaces (LIFs) on each DLR

The distributed logical router control virtual machine is the control plane component of the routing process, providing communication between NSX Manager and the NSX Controller cluster through the User World Agent (UWA). NSX Manager sends logical interface information to the control virtual machine and the NSX Controller cluster, and the control virtual machine sends routing updates to the NSX Controller cluster.

The User World Agent (UWA) is a TCP (SSL) client that facilitates communication between the ESXi hosts and the NSX Controller instances as well as the retrieval of information from the NSX Manager via interaction with the message bus agent.

VXLAN Tunnel Endpoints (VTEPs) are instantiated within the vSphere Distributed Switch to which the ESXi hosts that are prepared for NSX for vSphere are connected. VTEPs are responsible for encapsulating VXLAN traffic as frames in UDP packets and for the corresponding decapsulation. VTEPs take the form of one or more VMkernel ports with IP addresses and are used both to exchange packets with other VTEPs and to join IP multicast groups via Internet Group Membership Protocol (IGMP). If you use multiple VTEPs, then you must select a teaming method.

The NSX Edge services gateways (ESGs) primary function is north/south communication, but it also offers support for Layer 2, Layer 3, perimeter firewall, load balancing and other services such as SSL-VPN and DHCP-relay.

NSX includes a distributed kernel-level firewall known as the distributed firewall. Security enforcement is done at the kernel and VM network adapter level. The security enforcement implementation enables firewall rule enforcement in a highly scalable manner without creating bottlenecks on physical appliances. The distributed firewall has minimal CPU overhead and can perform at line rate.

The flow monitoring feature of the distributed firewall displays network activity between virtual machines at the application protocol level. This information can be used to audit network traffic, define and refine firewall policies, and identify botnets.

The NSX logical load balancer provides load balancing services up to Layer 7, allowing distribution of traffic across multiple servers to achieve optimal resource utilization and availability. The logical load balancer is a service provided by the NSX Edge service gateway.