To accommodate all log data from the products in the SDDC, you must size the compute resources and storage for the Log Insight nodes properly.

By default, the vRealize Log Insight virtual appliance uses the predefined values for small configurations, which have 4 vCPUs, 8 GB of virtual memory, and 510 GB of disk space provisioned. vRealize Log Insight uses 100 GB of the disk space to store raw data, index, metadata, and other information.

Select a size for the vRealize Log Insight nodes to collect and store log data from the SDDC management components and tenant workloads according to the objectives of this design.

Compute Resources for a vRealize Log Insight Medium-Size Node

Attribute

Specification

Appliance size

Medium

Number of CPUs

8

Memory

16 GB

Disk Capacity

510 GB (490 GB for event storage)

IOPS

1,000 IOPS

Amount of processed log data when using log ingestion

75 GB/day of processing per node

Number of processed log messages

5,000 event/second of processing per node

Environment

Up to 250 syslog connections per node

Sizing is based on IT organization requirements, but this design provides calculations based on a single region implementation, and is implemented on a per-region basis. This sizing is calculated according to the following node configuration per region:

Management vCenter Server

Platform Services Controller

vCenter Server

Compute vCenter Server

Platform Services Controller

vCenter Server

Management, shared edge and compute ESXi hosts

NSX for vSphere for the management cluster and for the shared compute and edge cluster

NSX Manager

NSX Controller instances

NSX Edge instances

vRealize Automation

vRealize Automation Appliance

vRealize IaaS Web Server

vRealize IaaS Management Server

vRealize IaaS DEM

vRealize Agent Servers

vRealize Orchestrator

Microsoft SQL Server

vRealize Operations Manager

Analytics cluster nodes

Event forwarding configured between vRealize Log Insight clusters

These components aggregate to approximately 210 syslog and vRealize Log Insight Agent sources. Assuming that you want to retain 7 days of data, use the following calculations:

For 210 syslog sources at a basal rate of 150 MB of logs ingested per-day per-source over 7 days, you need the following storage space:

210 sources * 150 MB of log data ≈ 31.5 GB log data per-day

31.5 GB * 7 days ≈ 220.5 GB log data per vRealize Log Insight node

220.5 GB * 1.7 indexing overhead ≈ 375 GB

Based on this example, the storage space that is allocated per medium-size vRealize Log Insight virtual appliance is enough to monitor the SDDC.

Consider the following approaches when you must increase the Log Insight capacity:

If you must maintain a log data retention for more than 7 days in your SDDC, you might add more storage per node by adding a new virtual hard disk. vRealize Log Insight supports virtual hard disks of up to 2 TB. If you must add more than 2 TB to a virtual appliance, add another virtual hard disk.

When you add storage to increase the retention period, extend the storage for all virtual appliances.

Note

Do not extend existing retention virtual disks. Once provisioned, do not reduce the size or remove virtual disks to avoid data loss.

If you must monitor more components by using log ingestion and exceed the number of syslog connections or ingestion limits defined in this design, you can deploy more vRealize Log Insight virtual appliances to scale out your environment. vRealize Log Insight can scale up to 12 nodes in an HA cluster.

Compute Resources for the vRealize Log Insight Nodes Design Decisions

Decision ID

Design Decision

Design Justification

Design Implication

SDDC-OPS-LOG-003

Deploy vRealize Log Insight nodes of medium size.

Accommodates the number of expected Syslog and vRealize Log Insight Agent connections from the following. This is approximately 210 syslog and vRealize Log Insight Agent sources.

Management & Compute vCenter Server, Platform Services Controller

Management, and shared edge and compute ESXi hosts

Management and compute components for NSX for vSphere

vRealize Automation components

vRealize Operations Manager components

Cross-vRealize Log Insight cluster event forwarding.

Using a medium-size appliances ensures that the storage space for the vRealize Log Insight cluster is sufficient for 7 days of data retention.

You must increase the size of the nodes if you configure Log Insight to monitor additional syslog sources.