We calculate the parity-violating nucleon-nucleon potential in heavy-baryon chiral perturbation theory up to the next-to-next-to-leading order. The one-pion exchange comes in the leading order and the next-to-next-to-leading order consists of two-pion-exchange and the two-nucleon contact terms. In order to investigate the effect of the higher order contributions, we calculate the parity-violating asymmetry in $\vec{n} p \to d \gamma$ at the threshold. The one-pion dominates the physical observable and the two-pion contribution is about or less than 10% of the one-pion contribution.