A visible and near infrared imaging spectrometer is one of the important parts for spectral detection. According to the actual application requirements of a ground-based visible and near infrared imaging spectrometer, a staring/scanning imaging spectrometer using an area array detector is proposed. The spectrometer also employs a new light splitting device-an Accousto-optical Tunable Filter (AOTF). It has a feature that the selection of spectrum is related to the frequency of the RF signal applied to it. By controlling the RF signal via a program, the spectrum can be controlled. A ground-based lunar observation experiment is carried out with the imaging spectrometer prototype designed. According to the experimental characteristics, another auxiliary coaxial optics is added for rough positioning on the basis of optical design. After the imaging results are acquired, radiometric calibration and SIFT image stitching are implemented. The results show that by combining a two-dimensional pointing mechanism and a staring imaging system with the SIFT image stitching algorithm, wide width high resolution spectral images can be acquired effectively.