Abstract
This paper addresses the problem of unstable updates and difficulty in constraining behavioral boundaries in large language models during real-world applications due to data noise and distribution perturbations. A robust fine-tuning algorithm framework based on gradient projection and constraint optimization is proposed. The method uniformly models the fine-tuning process as a constrained optimization problem. On the objective side, worst-case robust risk and tail risk are introduced to enhance the resistance to abnormal samples and perturbation inputs. On the constraint side, a feasible region composed of output distribution offset constraints and parameter trust region constraints is constructed to limit distribution drift and parameter drift caused by fine-tuning. To ensure the feasibility of iterative updates, a gradient projection mechanism is designed to decompose and correct the original update direction into an effective descent direction that satisfies the approximation of the local feasible region. Simultaneously, primal-dual updates are used to dynamically allocate constraint pressure, forming a closed-loop synergy between constraint satisfaction and objective optimization. Furthermore, the framework is compatible with efficient parameter fine-tuning. Low-rank incremental parameterization restricts trainable updates to a low-dimensional subspace, reducing training overhead while preserving constraint-driven geometric control capabilities. This research provides a structured, interpretable, and reproducible optimization path for robust fine-tuning and offers a unified technical solution for adapting highly reliable large models to task requirements and constraint boundaries.
