Abstract: Aimed at the problem of low elevation estimation for VHF radar in the presence of mutual coupling, an auto-calibration direction-of-arrival(DOA) algorithm is proposed. First, the form of VHF radar echo signal in the present of mutual coupling and multipath is analyzed. Then, based on the characteristic of the mutual coupling matrix for uniform linear array, the form of the signal parameters in the received data covariance matrix is transformed. Finally, without any previous mutual coupling calibration information, a subspace-based method is applied to estimate the target angle, and simultaneously the surface reflection coefficients and mutual coupling matrix can be achieved. Compared with many existing array calibration methods, the proposed algorithm retains antenna aperture and avoids iterative processing in the presence of coherent multipath signals and unknown mutual coupling matrix. Furthermore, it requires neither the presence of calibration sources nor previous calibration elements. Computer simulations show the effectiveness of the proposed algorithm.