We perform an analysis of Higgs portal models of dark matter (DM), where DM is light enough to contribute to invisible Higgs decays. Using effective field theory we show that DM can be a thermal relic only if there are additional light particles present with masses below a few 100 GeV. We give three concrete examples of viable Higgs portal models of light DM: (i) the SM extended by DM scalar along with an electroweak triplet and a singlet, (ii) a Two Higgs Doublet Model of type II with additional scalar DM, (iii) SM with DM and an extra scalar singlet that is lighter than DM. In all three examples the Br(h to invisible) constraint is not too restrictive, because it is governed by different parameters than the relic abundance. Additional light particles can have implications for flavor violation and collider searches.