The wide variety of low metallicity galaxies of the local universe serve as convenient laboratories to study the evolution of gas and dust and conditions for star formation in environments which may resemble those of the early universe. The Herschel Dwarf Galaxy Survey is studying the far infrared (FIR) and submillimeter (submm) properties of the gas and dust in galaxies with metallicity values as low as 1/45 that of solar. With complementary Spitzer, Laboca/APEX, Scuba/JCMT data, the dust spectral energy distributions are well constrained now, providing more accurate dust masses. We find a steep drop in dust-to-gas mass ratio (D/G) when the metallicity is below 12+log(O/H) ∼ 8. A submillimeter excess can be found in some low metallicity galaxies, which, when present, becomes apparent at wavelengths at or longer than 500 μm. While CO is difficult to observe in low metallicity gas, the FIR fine structure lines, on the other hand, are very luminous and highlight a potentially important reservoir of CO-free molecular gas, better traced by the 158 μm [CII] line.