Virtual Machine Proactive Fault Tolerance using Log-based Anomaly Detection

Abstract

Abstract Virtual Machine (VM) fault tolerance ensures high availability in cloud computing environments. Proactive fault tolerance strategies, which identify potential failures before they occur and move VMs to healthy hosts, help avert service disruptions due to VM failures. In recent years, there has been a wide adoption of Machine Learning (ML) approaches for fault detection. However, existing approaches often rely on ML models trained on labeled data, which can be challenging to obtain. They also require a large amount of training data and may struggle with real-time failure prediction and fast adaptation to dynamic environments. In this work, we propose VMFT-LAD (Virtual Machine Proactive Fault Tolerance using Log-based Anomaly Detection), a semi-supervised log anomaly detection model for proactive VM fault tolerance. VMFT-LAD leverages the efficiency of the Matrix Profile for anomaly detection and the log inference capability of Large Language Models (LLMs) to continuously learn and identify potential failures, including unforeseen fault types, with minimal human intervention. By focusing on detecting anomalies in log data, our approach operates without the need for labeled failure data. Extensive evaluations on several datasets demonstrate VMFT-LAD’s outstanding performance, achieving a Numenta Anomaly Benchmark (NAB) standard score of 90.74 under the criterion of predicting failures before the failure point. Compared to state-of-the-art anomaly detection models, VMFT-LAD demonstrates a high early detection rate of 96.28% while maintaining a low false positive rate of 0.02%, enabling timely VM migration before failures occur. The results highlight the superiority of VMFT-LAD in facilitating reliable and proactive fault tolerance strategies in virtualized environments.