We use the ultra-deep HUDF09 and the deep ERS data from the HST WFC3/IR camera , along with the wide area Spitzer /IRAC data from GOODS-S to derive SEDs of star-forming galaxies from the rest-frame UV to the optical over a wide luminosity range ( M _ { 1500 } \sim - 21 to M _ { 1500 } \sim - 18 ) from z \sim 7 to z \sim 4 . The sample contains \sim 400 z \sim 4 , \sim 120 z \sim 5 , \sim 60 z \sim 6 , and 36 prior z \sim 7 galaxies . Median stacking enables the first comprehensive study of very faint high-z galaxies at multiple redshifts ( e.g. , [ 3.6 ] = 27.4 \pm 0.1 AB mag for the M _ { 1500 } \sim - 18 sources at z \sim 4 ) . At z \sim 4 our faint median-stacked SEDs reach to \sim 0.06 L ^ { * } _ { z = 4 } and are combined with recently published results at high luminosity L > L ^ { * } that extend to M _ { 1500 } \sim - 23 . We use the observed SEDs and template fits to derive rest frame UV-to-optical colors ( U - V ) at all redshifts and luminosities . We find that this color does not vary significantly with redshift at a fixed luminosity . The UV-to-optical color does show a weak trend with luminosity , becoming redder at higher luminosities . This is most likely due to dust . At z \gtrsim 5 we find blue colors [ 3.6 ] - [ 4.5 ] \sim - 0.3 mag that are most likely due to rest-frame optical emission lines contributing to the flux in the IRAC filter bandpasses . The scatter across our derived SEDs remains substantial , but the results are most consistent with a lack of any evolution in the SEDs with redshift at a given luminosity . The similarity of the SEDs suggests a self-similar mode of evolution over a timespan from 0.7 Gyr to 1.5 Gyr that encompasses very substantial growth in the stellar mass density in the universe ( from \sim 4 \times 10 ^ { 6 } to \sim 2 \times 10 ^ { 7 } { M _ { \odot } Mpc ^ { -3 } } )