You can run multiple EM4J-enabled JVMs in a virtual machine. ESXi sets one balloon target for the VM, and that target is distributed among the JVMs running on the VM. There is no coordination of balloons in the separate JVM heaps, so a pragmatic recommendation is to limit the number of EM4J-enabled JVMs running on a VM to approximately four. JVMs that are not using EM4J, for example agents or transient utilities, are not an issue.
When you co-locate tc Runtime instances on VMs, consider how the size and load of the various servers will influence the ESXi balloon targets for the VMs. The algorithm ESXi uses to set balloon targets tends to provide more memory to more active VMs than to less active VMs. It increases balloon targets on less active VMs so it can shift the reclaimed memory to more active VMs. When you assign application servers to VMs, look for opportunities to take advantage of shifting workloads. It is better to co-locate servers that become active at similar times in a VM than to group servers with contrary activity levels in a VM. When all of the servers in a VM become more active simultaneously, ESXi will shift more memory to that VM relative to less active VMs. If instead you distribute servers among VMs so that the activity level in each VM remains relatively constant, the memory allocation will be more static and less able to take advantage of shifting loads.