We have developed a new cryogenic distillation system to purify Hydrogen-Deuteride (HD) gas for polarized HD targets in LEPS experiments at SPring-8. A small amount of ortho-H2 (∼0.01%) in the HD gas plays an important role in efficiently polarizing the HD target. Since there are 1-5% impurities of H2 and D2 in commercially available HD gases, it is necessary to purify the HD gas up to ∼99.99%. The distillation system is equipped with a cryogenic distillation unit filled with many small stainless steel cells called "Heli-pack". The distillation unit consists of a condenser part, a rectification part, and a reboiler part. The unit is kept at the temperature of 17-21 K. The Heli-pack has a large surface area that makes a good contact between gases and liquids. An amount of 5.2 mol of commercial HD gas is fed into the distillation unit. Three trials were carried out to purify the HD gas by changing temperatures (17.5 K and 20.5 K) and gas extraction speeds (1.3 ml/min and 5.2 ml/min). The extracted gas was analyzed using a gas analyzer system combining a quadrupole mass spectrometer with a gas chromatograph. One mol of HD gas with a purity better than 99.99% has been successfully obtained for the first time. The effective NTP (Number of Theoretical Plates), which is an indication of the distillation performances, is obtained to be 37.2±0.6. This value is in good agreement with a designed value of 37.9. The HD target is expected to be efficiently polarized under a well-controlled condition by adding an optimal amount of ortho-H2 to the purified HD gas.