Machine Learning Framework for Performance Prediction and Intelligent Resource Allocation in Complex Data Environments

Abstract

This paper focuses on the problem of database query execution time prediction and optimization. To address the limitations of traditional methods that suffer from error accumulation and insufficient scheduling efficiency in complex query scenarios, it proposes a comprehensive framework that integrates structured modeling with adaptive scheduling. First, a Plan-Graph Guided Latency Modeling (PGLM) mechanism is designed, which explicitly incorporates structural features of query plans to enhance the model's awareness of operator patterns and join topologies, thereby improving prediction accuracy and generalization. Second, an Adaptive Query–Resource Orchestrator (AQRO) is constructed to dynamically match query demands with system resources under a prediction–execution interaction mechanism, ensuring continuous satisfaction of service-level objectives (SLOs) and maintaining system stability. The proposed method demonstrates strong robustness under different hyperparameters, resource quotas, and query template diversity, achieving low prediction errors and reasonable uncertainty calibration in dynamic environments. The results show that the framework performs well in both latency prediction and resource optimization, providing a new technical path for database system performance improvement.