During the imaging process of hyperspectral remote sensing images, the quality of the images often deteriorates due to various types of noise such as detector noise, optical system noise, environmental noise, and statistical noise, which in turn affects the accuracy and credibility of information extraction in subsequent applications. Especially in the infrared spectral band, due to factors such as the thermal vibration of the detector material itself, it is significantly affected by thermal noise. To address this issue, this paper proposes a hyperspectral denoising method based on scale-adaptive spectral dictionary learning. Firstly, an adaptive scale constraint is introduced into the dictionary learning process to obtain the spectral dictionary of the image to be denoised. Secondly, the spatial domain information of the image is utilized as prior knowledge for encoding, and the total variation-variational decomposition and augmented Lagrangian sparse regression methods are applied to solve the sparse coding of the image. Finally, the denoised hyperspectral image is reconstructed using the spectral dictionary and sparse coding. Experimental results demonstrate that, compared to existing hyperspectral denoising algorithms, the proposed method achieves superior performance on both simulated and real datasets.