We investigate the afterglow properties and large-scale environments of several short-duration gamma-ray bursts ( GRBs ) with sub-arcsecond optical afterglow positions but no bright coincident host galaxies . The purpose of this joint study is to robustly assess the possibility of significant offsets , a hallmark of the compact object binary merger model . Five such events exist in the current sample of 20 short bursts with optical afterglows , and we find that their optical , X-ray , and \gamma -ray emission are systematically fainter . These differences may be due to lower circumburst densities ( by about an order of magnitude ) , to higher redshifts ( by \Delta z \approx 0.5 - 1 ) , or to lower energies ( by about a factor of 3 ) , although in the standard GRB model the smaller \gamma -ray fluences can not be explained by lower densities . To study the large-scale environments we use deep optical observations to place limits on underlying hosts and to determine probabilities of chance coincidence for galaxies near each burst . In 4 of the 5 cases the lowest probabilities of chance coincidence ( P ( < \delta R ) \sim 0.1 ) are associated with bright galaxies at separations of \delta R \sim 10 \arcsec , while somewhat higher probabilities of chance coincidence are associated with faint galaxies at separations of \sim 2 \arcsec . By measuring redshifts for the brighter galaxies in three cases ( z = 0.111 , 0.473 , 0.403 ) we find physical offsets of \approx 30 - 75 kpc , while for the faint hosts the assumption of z \gtrsim 1 leads to offsets of \sim 15 kpc . Alternatively , the limits at the burst positions ( \gtrsim 26 mag ) can be explained by typical short GRB host galaxies ( L \approx 0.1 - 1 L ^ { * } ) at z \gtrsim 2 . Thus , two possibilities exist : ( i ) \sim 1 / 4 of short GRBs explode \sim 50 kpc or \sim 15 kpc from the centers of z \sim 0.3 or z \gtrsim 1 galaxies , respectively , and have fainter afterglows due to the resulting lower densities ; or ( ii ) \sim 1 / 4 of short GRBs occur at z \gtrsim 2 and have fainter afterglows due to their higher redshifts . The high redshift scenario leads to a bimodal redshift distribution , with peaks at z \sim 0.5 and z \sim 2 . The large offset scenario leads to an offset distribution that is well-matched by theoretical predictions of NS-NS/NS-BH binary kicks , or by a hybrid population with globular cluster NS-NS binaries at large offsets and primordial binaries at offsets of \lesssim 10 kpc ( indicative of negligible kicks ) . Deeper constraints on any coincident galaxies to \gtrsim 28 mag ( using the Hubble Space Telescope ) will allow us to better exclude the high-redshift scenario .