We explore the relation between the dynamical mass-to-light ratio ( M / L ) and rest-frame color of massive quiescent galaxies out to z \sim 2 . We use a galaxy sample with measured stellar velocity dispersions in combination with Hubble Space Telescope and ground-based multi-band photometry . Our sample spans a large range in \log M _ { dyn } / L _ { g } ( of 1.6 dex ) and \log~ { } M _ { dyn } / L _ { K } ( of 1.3 dex ) . There is a strong , approximately linear correlation between the M / L for different wavebands and rest-frame color . The root-mean-scatter scatter in \log~ { } M _ { dyn } / L residuals implies that it is possible to estimate the M / L with an accuracy of \sim 0.25 dex from a single rest-frame optical color . Stellar population synthesis ( SPS ) models with a Salpeter stellar initial mass function ( IMF ) can not simultaneously match M _ { dyn } / L _ { g } vs . ( g - z ) _ { rest - frame } and M _ { dyn } / L _ { K } vs . ( g - K ) _ { rest - frame } . By changing the slope of the IMF we are still unable to explain the M/L of the bluest and reddest galaxies . We find that an IMF with a slope between \alpha = 2.35 and \alpha = 1.35 provides the best match . We also explore a broken IMF with a Salpeter slope at M < 1 M _ { \odot } and M > 4 M _ { \odot } and a slope \alpha in the intermediate region . The data favor a slope of \alpha = 1.35 over \alpha = 2.35 . Nonetheless , our results show that variations between different SPS models are comparable to the IMF variations . In our analysis we assume that the variation in M / L and color is driven by differences in age , and that other contributions ( e.g. , metallicity evolution , dark matter ) are small . These assumptions may be an important source of uncertainty as galaxies evolve in more complex ways .