State-of-the-art methods for airport detection in panchromatic remote sensing images utilize very limit geometrical features of airport line segments. This paper proposed a new method which uses both bottom-up and top-down saliency. Because the airport runways have features of vicinity and parallelity, and their lengths are among certain range, the concept of near parallelity is introduced after using an improved line segments detector (LSD). It is used as a priori knowledge which can fully exploit geometrical relationship of airport runways to get top-down saliency. Meanwhile, a simplified graph-based visual saliency (GBVS) model is used to extract bottom-up saliency. Candidate regions can be gotten by combining those two clues. After that, scale-invariant features transform (SIFT) and support vector machine (SVM) are used to finally determine whether the regions contain an airport or not. The proposed method is tested on an image dataset composed of different kinds of airports. The experimental results show that the method has advantages in terms of speed, recognition rate and false alarm rate. Also, the method is more robust to complex background.