In order to decrease the time and alleviate the smoothness of the adaptive sparse representation algorithm, a modified adaptive sparse representation algorithm based on difference threshold is introduced in this paper. This modified algorithm computes the difference between the current block and the previous one, then compares the difference with the threshold. When the difference is less than the threshold, the two blocks are considered having the same sparse representation vector and error. It is not needed to compute over current block again. When the difference is greater, they are considered as different. The current block contains the edge area and its position is recorded. It is then protected from averaging in reconstructing the result to alleviate the smoothness. The experimental results performed on millimeter-wave image demonstrated the effectiveness of the proposed method.