If you want to use a third-party CA-signed certificate, either with VMCA as a subordinate authority or with a custom certificate authority, you have to send a Certificate Signing Request (CSR) to the CA.

Use a CSR with these characteristics:

2048 bits

PKCS1

No wildcards

Start time of one day before the current time

CN (and SubjectAltName) set to the host name (or IP address) that the ESXi host has in the vCenter Server inventory.