We present the implementation and the first results of cosmic ray ( CR ) feedback in the Feedback In Realistic Environments ( FIRE ) simulations . We investigate CR feedback in non-cosmological simulations of dwarf , sub- L \star starburst , and L \star galaxies with different propagation models , including advection , isotropic and anisotropic diffusion , and streaming along field lines with different transport coefficients . We simulate CR diffusion and streaming simultaneously in galaxies with high resolution , using a two moment method . We forward-model and compare to observations of \gamma -ray emission from nearby and starburst galaxies . We reproduce the \gamma -ray observations of dwarf and L \star galaxies with constant isotropic diffusion coefficient \kappa \sim 3 \times 10 ^ { 29 } { cm ^ { 2 } s ^ { -1 } } . Advection-only and streaming-only models produce order-of-magnitude too large \gamma -ray luminosities in dwarf and L \star galaxies . We show that in models that match the \gamma -ray observations , most CRs escape low-gas-density galaxies ( e.g . dwarfs ) before significant collisional losses , while starburst galaxies are CR proton calorimeters . While adiabatic losses can be significant , they occur only after CRs escape galaxies , so they are only of secondary importance for \gamma -ray emissivities . Models where CRs are ‘ ‘ trapped ’ ’ in the star-forming disk have lower star formation efficiency , but these models are ruled out by \gamma -ray observations . For models with constant \kappa that match the \gamma -ray observations , CRs form extended halos with scale heights of several kpc to several tens of kpc .