vRealize Log Insight provides real-time log management and log analysis with machine learning-based intelligent grouping, high-performance searching, and troubleshooting across physical, virtual, and cloud environments.

vRealize Log Insight collects data from ESXi hosts using the syslog protocol. It connects to other VMware products, like vCenter Server, to collect events, tasks, and alarms data, and integrates with vRealize Operations Manager to send notification events and enable launch in context. vRealize Log Insight also functions as a collection and analysis point for any system capable of sending syslog data. In addition to syslog data an ingestion agent can be installed on Linux or Windows servers or may come pre-installed on certain VMware products to collect logs. This agent approach is especially useful for custom application logs and operating systems that don't natively support the syslog protocol, such as Windows.

You can deploy vRealize Log Insight as a virtual appliance in one of the following configurations:

Standalone node

Highly available cluster of one master and at least two worker nodes using an integrated load balancer (ILB)

The compute and storage resources of the vRealize Log Insight instances can scale-up as growth demands.

For high availability and scalability, you can deploy several vRealize Log Insight instances in a cluster where they can have either of the following roles:

Master Node

Required initial node in the cluster. The master node is responsible for queries and log ingestion. The Web user interface of the master node serves as a single pane of glass, presenting data from multiple sources in the cluster in a unified display. All queries against data are directed to the master, which in turn queries the workers as appropriate.

Worker Node

 Enables scale-out in larger environments. A worker node is responsible for ingestion of logs. A worker node stores logs locally. If a worker node is down, the logs on that worker becomes unavailable. You need at least two worker nodes to form a cluster with the master node.

Integrated Load Balancer (ILB)

Provides high availability (HA). The ILB runs on one of the cluster nodes. If the node that hosts the ILB Virtual IP (VIP) address stops responding, the VIP address is failed over to another node in the cluster. 

The architecture of vRealize Log Insight enables several channels for HA collection of log messages.

Cluster Architecture of vRealize Log Insight
vRealize Log Insight interacts with the interface, with vRO, and with vRLI clients. Inside the vRLI cluster are content packs and the master and worker nodes

vRealize Log Insight clients connect to ILB VIP address, and use the Web user interface and ingestion (by using syslog or the Ingestion API) to send logs to vRealize Log Insight.

By default, the vRealize Log Insight collects data from vCenter Server systems and ESXi hosts. For forwarding logs from NSX for vSphere, and vRealize Automation, use content packs which contain extensions or provide integration with other systems in the SDDC.

You can configure vRealize Log Insight user authentication to utilize one or more of the following authentication models:

Microsoft Active Directory

Local Accounts

VMware Identity Manager

The integration with vRealize Operations Manager provides data from multiple sources to a central place for monitoring the SDDC. vRealize Log Insight sends notification events to vRealize Operations Manager. You can also launch vRealize Log Insight from the vRealize Operations Manager Web user interface.

vRealize Log Insight supports data archiving on NFS shared storage that each vRealize Log Insight node can access. 

You back up each vRealize Log Insight cluster using traditional virtual machine backup solutions that use vSphere Storage APIs for Data Protection (VADP) compatible backup software such as vSphere Data Protection.

The scope of the SDDC design covers multiple regions. Using vRealize Log Insight in a multi-region design can provide a syslog infrastructure in all regions of the SDDC. Using vRealize Log Insight across multiple regions requires deploying a cluster in each region. vRealize Log Insight supports event forwarding to other vRealize Log Insight deployments across regions in the SDDC. Implementing failover by using vSphere Replication or disaster recovery by using Site Recovery Manager is not necessary. The event forwarding feature adds tags to log message that identify the source region and event filtering prevents looping messages between the regions.

Event Forwarding in vRealize Log Insight
vRealize Log Insight instances can forward each other log events. In this way, if one of the Log Insight deployments is not responding, you will have access to all logs from the other instance. Event forwarding replaces traditional disaster recovery.