Improving upon the present background rejection capabilities of the cryogenic Ge detectors for direct dark matter search involves an in-depth comprehension of the charge collection process in these devices. Experimental data point to the combined effects of lattice and impurity scattering on the anisotropy of electron transport in germanium at mK temperatures. A Monte Carlo simulation code has been implemented to incorporate these features in a consistent model of charge collection. In a novel approach to carrier scattering by charged impurities, the scattering field of the impurities is treated statistically as a random contribution to the collection field, described by the Holtsmark distribution function with a single adjustable parameter, the mean density of the charged centers. Simulation of charge collection along these lines in devices different by their impurity content shows excellent agreement to experiment. Especially noteworthy is the fact that the strength of impurity scattering is reversed from the known concentration of dopant impurities in the crystals, as the crystal with the higher dopant concentration shows lower scattering at low field than the one with the lower concentration. This raises as an issue for further improvement of these devices, the question of the nature of the scattering centers in high-purity Ge crystals at cryogenic temperatures, associated presumably with deep level impurities or crystal defects.