Extreme-mass-ratio inspiral ( EMRI ) is an important gravitational-wave ( GW ) source and it normally consists of one stellar-mass black hole ( BH ) whirling closely around a supermassive black hole ( SMBH ) . In this Letter , we demonstrate that the small body , in fact , could be a BH binary ( BHB ) . Previous numerical scatting experiments have shown that SMBHs can tidally capture BHBs to bound orbits . Here we investigate the subsequent long-term evolution . We find that those BHBs with a semi-major axis of a \lesssim 5 \times 10 ^ { -3 } AU can be captured to tightly-bound orbits such that they will successfully inspiral towards the central SMBHs without being scattered away by stellar relaxation processes . We estimate that these binary-EMRIs ( b-EMRIs ) could constitute at most 10 \% of the EMRI population . Moreover , we show that when the eccentricity of a b-EMRI drops to about 0.85 , the two stellar BHs will quickly merge due to the tidal perturbation by the SMBH . The high-frequency ( \sim 10 ^ { 2 } Hz ) GWs generated during the coalescence coincide with the low-frequency ( \sim 10 ^ { -3 } Hz ) waves from the b-EMRI , making this system an ideal target for future multi-band GW observations .