A series of Ge quantum dot samples were grown by ion beam sputtering on Si (100) substrates with a Si buffer layer. The evolution of the topography and dimension of Ge/Si quantum dot were characterized using AFM and Raman spectra. The results show that the density of the quantum dots increased to a maximum and then decreased with the thickness of Si buffer layers, the maximum is up to 1.9×1010 cm-2 due to the influence of the thickness and growth patterns of Si buffer layers. Growth interruption is beneficial to improve the crystallization of Si buffer layer and the density of the quantum dot. The effects of Si buffer layers which manipulate the growth and shape of the Ge quantum dots are discussed in details. In addition, a growth model of the quantum dots is proposed.