[Fix] vCenter Error “vSphere HA Virtual Machine Failover Failed”

Case: vSphere HA virtual machine failover failed

I’m running VCSA 7.0.3.00100. And I keep seeing the ‘vSphere HA virtual machine failover failed’ alerts when I log in to the web UI. Any ideas what I can do to clear the alerts? ‘Acknowledge’ and ‘Reset To Green’ are both grayed out since the VMs are inaccessible.

--community.spiceworks.com

vCenter Server is the official centralized management utility used to manage virtual machines, multiple ESXi hosts, and all dependent components from a single centralized location. However, in a cluster with an isolation response set to Leave powered on when a host becomes isolated may display the error “vSphere HA virtual machine failover failed” on a virtual machine.

The virtual machine continues to run without a problem. But what exactly causes this error and how to fix it? This article will explain it for you.

The causes of the error “vSphere HA virtual machine failover failed”

VMware vSphere High Availability (HA) is a utility used to restart failed virtual machines on alternative host servers to reduce application downtime. According to VMware, the error “vSphere HA virtual machine failover failed” occurs whenever a High Availability primary agent declares a host dead.

However, this alarm doesn’t mean that HA has failed or stopped working. It only means that one or more virtual machines failed to get powered on by a host in a cluster protected by HA, the virtual machine may continue to run without incident.

The causes can be summarized as follows:

• The host is still running but has disconnected from the network. The cluster's host isolation response is set to Leave powered on: With Leave powered on, the host that became network isolated will lock the files, so the remaining hosts in the cluster will fail to perform the power on the task on the virtual machines, resulting in the alarm triggering.
• The host is still running but has disconnected from the network. The cluster's host isolation response is set to Shut down or Power off: A host will attempt to send shut down or power off commands to its running virtual machines when it recognizes it is isolated. If the virtual machine is not successfully shut down, or the locks are not released, then the alarm will be a trigger.
• The host has failed and the virtual machine storage is in a degraded state: The remaining hosts in the cluster cannot contact the storage device and fail to power up the virtual machines, resulting in the alarm triggering.

How to fix error “vSphere HA virtual machine failover failed”

So, the error “vSphere HA virtual machine failover failed” doesn’t mean that HA has failed and stopped working. In fact, according to VMware, this is expected behavior in VMware vCenter Server 6.x, because the virtual machines continue to run without incident, you can safely ignore this issue.

But this alert may be generated on a large number of the VMs and persists as long as the HA primary proxy claims that the ESXi host crashes. If you want to clear the alarm, you can follow the following steps.

How to clear the alarm from the virtual machine

1. Navigate to Monitor tab, and click Issues and Alarms > Triggered Alarms.

2. Select the alarm and select Acknowledge.

3. Open the vCenter Object and navigate to Monitor > Issues and Alarms > Triggered Alarms. From this object, select the active alarm and click RESET TO GREEN.

Batch backup VMs managed by vCenter Server to protect data

A single vCenter Server instance can support a maximum of 1,000 hosts, 10,000 powered-on virtual machines and 15,000 registered virtual machines. However, vCenter Server doesn't provide the feature of virtual machine backup.

And that makes backup software a commonly accepted choice. Here I introduce you to a VMware backup software AOMEI Cyber Backup, it automates the backup process of multiple VMs managed by vCenter Server, and sends email notifications if there's an error.

✦ Agentless Backup: create complete and independent image-level backup for VMware ESXi and Hyper-V VMs.
✦ Multiple VM Backup: batch backup large numbers of VMs managed by vCenter Server, or multiple VMs on a standalone ESXi host.
✦ Multiple Storage Destinations: backup to a local drive, or network destinations like NAS.
✦ Automated Execution: create backup schedules to automate backups daily, weekly, or monthly.
✦ Restore Entire VM: restore instant available VMs from any selected restore points to an original or new location.

AOMEI Cyber Backup supports VMware ESXi 6.0 and later versions. Next, I will show you how to batch backup VMs managed by vCenter Server with AOMEI Cyber Backup in 4 simple steps. You can click the following button to download the 30-day free trial.

4 simple steps to batch back up VMs managed by vCenter Server

1. Bind Devices: Access to AOMEI Cyber Backup web client, navigate to Source Device > VMware > + Add VMware Device to Add vCenter or Standalone ESXi host. And then click … > Bind Device.

2. Create Backup Task: Navigate to Backup Task > + Create New Task, and select VMware ESXi Backup as the Backup Type.

3. Set Task Name, Device, Target, Schedule, and Cleanup as needed.

• Task Name: you can change the task name or use the default name with an ordinal.
• Device: batch select large numbers of VMs managed by vCenter Server for centralized backup.

• Target: select to back up to a local path, or to a network path like NAS.
• Schedule (optional): perform full, differential, or incremental backup, and automate execution according to the frequency you specified.

• Cleanup (optional): automatically delete the old backup copies that exceed the retention period you specified.

4. Run Backup: Click Start Backup and select Add the schedule and start backup now, or Add the schedule only.

Created backup tasks will be listed and monitored separately, for restoring, progress checking and schedule changing.

When restoring, you can select Restore to new location option to create a new VM in the same or another datastore/host directly from the backup to perform out-of-spacre recovery, saves the trouble of re-configuring the new VM.

Summary

vSphere High Availability (HA) reduces downtime by restarting failed virtual machines on alternative hosts. If the High Availability proxy recognizes an ESXi host crashed, but the Virtual Machines are still running, it may trigger the alarm “vSphere HA virtual machine failover failed”. This article showed the possible causes of this error and how to clear the alarm.

Also, it is important to provide the basic data protection for the virtual machines managed by vCenter Server. You can try AOMEI Cyber Backup, a simple, flexible free VM backup solution that enables you to centrally manage and automate the backup VMware ESXi or Hyper-V VMs.