We discuss several possible caveats which arise with the interpretation of measurements of fluctuations in heavy ion collisions. We especially focus on the ratios of particle yields, which has been advocated as a possible signature of a critical point in the QCD phase diagram. We conclude that current experimental observables are not well-defined and are without a proper quantitative meaning.