We develop a formalism for studying the dynamics of massive black hole binaries embedded in gravitationally-bound stellar cusps , and study the binary orbital decay by three-body interactions , the impact of stellar slingshots on the density profile of the inner cusp , and the properties of the ejected hypervelocity stars ( HVSs ) . We find that the scattering of bound stars shrinks the binary orbit and increases its eccentricity more effectively than that of unbound ambient stars . Binaries with initial eccentricities e \gtrsim 0.3 and/or unequal-mass companions ( M _ { 2 } / M _ { 1 } \lesssim 0.1 ) can decay by three-body interactions to the gravitational wave emission regime in less than a Hubble time . The stellar cusp is significantly eroded , and cores as shallow as \rho \propto r ^ { -0.7 } may develop from a pre-existing singular isothermal density profile . A population of HVSs is ejected in the host galaxy halo , with a total mass \sim M _ { 2 } . We scale our results to the scattering of stars bound to Sgr A ^ { * } , the massive black hole in the Galactic Center , by an inspiraling companion of intermediate mass . Depending on binary mass ratio , eccentricity , and initial slope of the stellar cusp , a core of radius \sim 0.1 pc typically forms in 1-10 Myr . On this timescale about 500-2500 HVSs are expelled with speeds sufficiently large to escape the gravitational potential of the Milky Way .