We present 21-cm Spectral Line Observations of Neutral Gas with the VLA ( 21-SPONGE ) , a Karl G. Jansky Very Large Array ( VLA ) large project ( \sim 600 hours ) for measuring the physical properties of Galactic neutral hydrogen ( H i ) . 21-SPONGE is distinguished among previous Galactic H i  studies as a result of : ( 1 ) exceptional optical depth sensitivity ( \sigma _ { \tau } < 10 ^ { -3 } per 0.42 km s ^ { -1 } channels over 57 lines of sight ) ; ( 2 ) matching 21 cm emission spectra with highest-possible angular resolution ( \sim 4 ^ { \prime } ) from the Arecibo Observatory ; ( 3 ) detailed comparisons with numerical simulations for assessing observational biases . We autonomously decompose 21 cm spectra and derive the physical properties ( i.e. , spin temperature , T _ { s } , column density ) of the cold neutral medium ( CNM ; T _ { s } < 250 K ) , thermally unstable medium ( UNM ; 250 < T _ { s } < 1000 K ) and warm neutral medium ( WNM ; T _ { s } > 1000 K ) simultaneously . We detect 50 \% of the total H i  mass in absorption , the majority of which is CNM ( 56 \pm 10 \% , corresponding to 28 \% of the total H i  mass ) . Although CNM is detected ubiquitously , the CNM fraction along most lines of sight is \lesssim 50 \% . We find that 20 \% of the total H i  mass is thermally unstable ( 41 \pm 10 \% of H i  detected in absorption ) , with no significant variation with Galactic environment . Finally , although the WNM comprises 52 \% of the total H i  mass , we detect little evidence for WNM absorption with 1000 < T _ { s } < 4000 K . Following spectral modeling , we detect a stacked residual absorption feature corresponding to WNM with T _ { s } \sim 10 ^ { 4 } K . We conclude that excitation in excess of collisions likely produces significantly higher WNM T _ { s } than predicted by steady-state models .