We reply to the comment by Schiller on our paper "The quantum vacuum as the origin of the speed of light" [M. Urban et al., Eur. Phys. J. D 67, 58 (2013)]. We further detail our model and explain why the limits provided by ultra-stable optical cavities on the amplitude of fluctuations of the light speed do not apply to the predictions on group velocity that we gave in [M. Urban et al., Eur. Phys. J. D 67, 58 (2013)]. We then estimate the magnitude of phase velocity fluctuations expected in our model, showing that they are much smaller than those of the group velocity, and hence, well below the best present limits obtained either with ultra-stable resonators or large interferometers.