When Network I/O Control is enabled, the distributed switch allocates bandwidth for the following system traffic types.

Fault tolerance traffic

iSCSI traffic

vSphere vMotion traffic

Management traffic

VMware vSphere Replication traffic

NFS traffic

vSAN traffic

vSphere Data Protection backup traffic

Virtual machine traffic

Network I/O Control enforces the share value specified for the different traffic types only when there is network contention. When contention occurs Network I/O Control applies the share values set to each traffic type. As a result, less important traffic, as defined by the share percentage, will be throttled, allowing more important traffic types to gain access to more network resources.

Network I/O Control also allows the reservation of bandwidth for system traffic based on the capacity of the physical adapters on a host, and enables fine-grained resource control at the virtual machine network adapter level. Resource control is similar to the model for vCenter CPU and memory reservations.

The following heuristics can help with design decisions.

Shares vs. Limits

When you use bandwidth allocation, consider using shares instead of limits. Limits impose hard limits on the amount of bandwidth used by a traffic flow even when network bandwidth is available.

Limits on Certain Resource Pools

Consider imposing limits on a given resource pool. For example, if you put a limit on vSphere vMotion traffic, you can benefit in situations where multiple vSphere vMotion data transfers, initiated on different hosts at the same time, result in oversubscription at the physical network level. By limiting the available bandwidth for vSphere vMotion at the ESXi host level, you can prevent performance degradation for other traffic.

Teaming Policy

When you use Network I/O Control, use Route based on physical NIC load teaming as a distributed switch teaming policy to maximize the networking capacity utilization. With load-based teaming, traffic might move among uplinks, and reordering of packets at the receiver can result occasionally.

Traffic Shaping

Use distributed port groups to apply configuration policies to different traffic types. Traffic shaping can help in situations where multiple vSphere vMotion migrations initiated on different hosts converge on the same destination host. The actual limit and reservation also depend on the traffic shaping policy for the distributed port group where the adapter is connected to.

Based on the heuristics, this design has the following decisions.

Network I/O Control Design Decisions

Decision ID

Design Decision

Design Justification

Design Implication

SDDC-VI-NET-005

Enable Network I/O Control on all distributed switches.

Increase resiliency and performance of the network.

If configured incorrectly Network I/O Control could impact network performance for critical traffic types.

SDDC-VI-NET-006

Set the share value for vMotion traffic to Low.

During times of contention vMotion traffic is not as important as virtual machine or storage traffic.

During times of network contention vMotion's will take longer then usual to complete.

SDDC-VI-NET-007

Set the share value for vSphere Replication traffic to Low.

During times of contention vSphere Replication traffic is not as important as virtual machine or storage traffic.

During times of network contention vSphere Replication will take longer and could violate the defined SLA.

SDDC-VI-NET-008

Set the share value for vSAN to High.

During times of contention vSAN traffic needs guaranteed bandwidth so virtual machine performance does not suffer.

None.

SDDC-VI-NET-009

Set the share value for Management to Normal.

By keeping the default setting of Normal management traffic is prioritized higher then vMotion and vSphere Replication but lower then vSAN traffic. Management traffic is important as it ensures the hosts can still be managed during times of network contention.

None.

SDDC-VI-NET-010

Set the share value for NFS Traffic to Low.

Because NFS is used for secondary storage, such as VDP backups and vRealize Log Insight archives it is not as important as vSAN traffic, by prioritizing it lower vSAN is not impacted.

During times of contention VDP backups will be slower than usual.

SDDC-VI-NET-011

Set the share value for vSphere Data Protection Backup traffic to Low.

During times of contention it is more important that primary functions of the SDDC continue to have access to network resources over backup traffic.

During times of contention VDP backups will be slower than usual.

SDDC-VI-NET-012

Set the share value for virtual machines to High.

Virtual machines are the most important asset in the SDDC. Leaving the default setting of High ensures that they will always have access to the network resources they need.

None.

SDDC-VI-NET-013

Set the share value for Fault Tolerance to Low.

Fault Tolerance is not used in this design therefore it can be set to the lowest priority.

None.

SDDC-VI-NET-014

Set the share value for iSCSI traffic to Low.

iSCSI is not used in this design therefore it can be set to the lowest priority.

None.