A number of recent studies have proposed that the stellar initial mass function ( IMF ) of early type galaxies varies systematically as a function of galaxy mass , with higher mass galaxies having bottom heavy IMFs . These bottom heavy IMFs have more low-mass stars relative to the number of high mass stars , and therefore naturally result in proportionally fewer neutron stars and black holes . In this paper , we specifically predict the variation in the number of black holes and neutron stars based on the power-law IMF variation required to reproduce the observed mass-to-light ratio trends with galaxy mass . We then test whether such variations are observed by studying the field low-mass X-ray binary populations ( LMXBs ) of nearby early-type galaxies . In these binaries , a neutron star or black hole accretes matter from a low-mass donor star . Their number is therefore expected to scale with the number of black holes and neutron stars present in a galaxy . We find that the number of LMXBs per K-band light is similar among the galaxies in our sample . These data therefore demonstrate the uniformity of the slope of the IMF from massive stars down to those now dominating the K-band light , and are consistent with an invariant IMF . Our results are inconsistent with an IMF which varies from a Kroupa/Chabrier like IMF for low mass galaxies to a steep power-law IMF ( with slope x =2.8 ) for high mass galaxies . We discuss how these observations constrain the possible forms of the IMF variations and how future Chandra observations can enable sharper tests of the IMF .