The traditional hyperspectral image unmixing algorithm involves the extraction of endmember and the estimation of abundance values for each endmember. Although many models usually provide acceptable unmixing results, the bias may be great in those pixels where an unknown endmember exists. Therefore, a hyperspectral image unmixing algorithm based on support vector data description (SVDD) was proposed. First, hyperspectral image datas were classified into two parts,i.e., innerclass and outerclass. The datas in the innerclass were considered as the pixels mixed by known endmember datas entirely, and the datas in outerclass included unknown endmembers. The boundary between the two classes was considered as points mixed by known and unknown endmember datas. Then, unmixing operation was carried out. Experimental results on synthetic and real hyperspectral data demonstrate that this method reduces effectively the influence of the existing unknown endmembers on unmixing results, and unmixing component with unknown endmember can be given. The results unmixed by the proposed algorithm are hardly affected by unknown endmembers and are superior to that of direct unmixing.