Millimeter-wave (MMW) technology has been widely utilized in human security screening applications due to its superior penetration capabilities through clothing and safety for human exposure. However, existing methods largely rely on fixed polarization modes, neglecting the potential insights from variations in target echoes with respect to incident polarization. This study provides a theoretical analysis of the cross-polarization echo power as a function of the incident polarization angle under linear polarization conditions. Additionally, based on the transmission characteristics of multi-layer medium, we extended the depth spectrum model employed in direct detection to accommodate scenarios involving multi-layered structures. Building on this foundation, by obtaining multiple depth spectrums through polarization angle scanning, we propose the Polarization Angle-Depth Matrix to characterize target across both the polarization angle and depth dimensions in direct detection. Simulations and experimental validations confirm its accuracy and practical value in detecting concealed weapons in human security screening scenarios.