Stripe noise is a common phenomenon in remote sensing image due to non-uniformity problems. The stripe noise seriously affects the image quality and subsequent applications. Unlike most destriping methods， structural properties of stripe noise are analyzed and the purpose of destriping is achieved by separating the stripe components. In the proposed optimization model， the L0-norm-based is used to describe global sparse property of stripes. In addition， difference-based constraints are adopted to describe the smoothness and discontinuity in the along-stripe and across-stripe directions， respectively. In order to better protect the detailed information of an image， an edge weighting factor is introduced in the constraints of across-stripe direction. Finally， the proposed model is solved and optimized by the alternating direction method of multipliers （ADMM）. The algorithm is verified by the in-orbit images obtained by Advanced Geosynchronous Radiation Imager （AGRI） in comparison with typical destriping methods. Experimental results show that the proposed algorithm completely eliminates the stripe noise and preserves more details， which shows better qualitative and quantitative result.