The presence of extremely compact galaxies at z \sim 2 and their subsequent growth in physical size has been the cause of much puzzlement .
We revisit the question using deep infrared Wide Field Camera 3 data to probe the rest-frame optical structure of 935 galaxies selected with 0.4 < z < 2.5 and stellar masses M _ { * } > 10 ^ { 10.7 } M _ { \odot } in the UKIRT Ultra Deep Survey and GOODS-South fields of the CANDELS survey .
At each redshift , the most compact sources are those with little or no star formation , and the mean size of these systems at fixed stellar mass grows by a factor of 3.5 \pm 0.3 over this redshift interval .
The data are sufficiently deep to identify companions to these hosts whose stellar masses are ten times smaller .
By searching for these around 404 quiescent hosts within a physical annulus 10 ~ { } h ^ { -1 } ~ { } \textrm { kpc } < R < 30 ~ { } h ^ { -1 } kpc , we estimate the minor merger rate over 0.4 < z < 2 .
We find that 13 \% - 18 \% of quiescent hosts have likely physical companions with stellar mass ratios of 0.1 or greater .
Mergers of these companions will typically increase the host mass by 6 \% \pm 2 \% per merger timescale .
We estimate the minimum growth rate necessary to explain the declining abundance of compact galaxies .
Using a simple model motivated by recent numerical simulations , we then assess whether mergers of the faint companions with their hosts are sufficient to explain this minimal rate .
We find that mergers may explain most of the size evolution observed at z \lesssim 1 if a relatively short merger timescale is assumed , but the rapid growth seen at higher redshift likely requires additional physical processes .