We present an observationally motivated model to connect the AGN and galaxy populations at 0.2 < z < 1.0 and predict the AGN X-ray luminosity function ( XLF ) . We start with measurements of the stellar mass function of galaxies ( from the Prism Multi-object Survey ) and populate galaxies with AGNs using models for the probability of a galaxy hosting an AGN as a function of specific accretion rate . Our model is based on measurements indicating that the specific accretion rate distribution is a universal function across a wide range of host stellar mass with slope \gamma _ { 1 } \approx - 0.65 and an overall normalization that evolves with redshift . We test several simple assumptions to extend this model to high specific accretion rates ( beyond the measurements ) and compare the predictions for the XLF with the observed data . We find good agreement with a model that allows for a break in the specific accretion rate distribution at a point corresponding to the Eddington limit , a steep power-law tail to super-Eddington ratios with slope \gamma _ { 2 } = -2.1 ^ { +0.3 } _ { -0.5 } , and a scatter of 0.38 dex in the scaling between black hole and host stellar mass . Our results show that samples of low luminosity AGNs are dominated by moderately massive galaxies ( \mathcal { M _ { * } } \sim 10 ^ { 10 - 11 } \mathcal { M } _ { \odot } ) growing with a wide range of accretion rates due to the shape of the galaxy stellar mass function rather than a preference for AGN activity at a particular stellar mass . Luminous AGNs may be a severely skewed population with elevated black hole masses relative to their host galaxies and in rare phases of rapid accretion .