We test the history of structure formation from redshift 1 to today by matching galaxies from VIPERS and SDSS with dark matter haloes in the MultiDark SMDPL N-body simulation . We first show that the standard subhalo abundance matching ( SHAM ) recipe implemented with MultiDark characterizes the clustering of galaxies well both at redshift 0 for SDSS and at redshift 1 for VIPERS . This is an important validation of the SHAM model at high redshift . We then remap the simulation timesteps to test alternative growth histories and infer the growth index \gamma = 0.6 \pm 0.3 . This analysis demonstrates the power of using N-body simulations to forward model galaxy surveys for cosmological inference . The data products and code necessary to reproduce the results of this analysis are available at \url https : //github.com/darklight-cosmology/vipers-sham