Context : Recent research has been constraining the retention fraction of black holes ( BHs ) in globular clusters by comparing the degree of mass segregation with N -body simulations . They are consistent with an upper limit of the retention fraction being 50 % or less . Aims : In this work , we focus on direct simulations of the dynamics of BHs in star clusters . We aim to constrain the effective distribution of natal kicks that BHs receive during supernova ( SN ) explosions and to estimate the BH retention fraction . Methods : We used the collisional N -body code nbody6 to measure the retention fraction of BHs for a given set of parameters , which are : the initial mass of a star cluster , the initial half-mass radius , and \sigma _ { \mathrm { BH } } , which sets the effective Maxwellian BH velocity kick distribution . We compare these direct N -body models with our analytic estimates and newest observational constraints . Results : The numerical simulations show that for the one-dimensional ( 1D ) velocity kick dispersion \sigma _ { \mathrm { BH } } < 50 \mathrm { km s ^ { -1 } } , clusters with radii of 2 pc and that are initially more massive than 5 \times 10 ^ { 3 } M _ { \odot } retain more than 20 % of BHs within their half-mass radii . Our simple analytic model yields a number of retained BHs that is in good agreement with the N -body models . Furthermore , the analytic estimates show that ultra-compact dwarf galaxies ( UCDs ) should have retained more than 80 % of their BHs for \sigma _ { \mathrm { BH } } \leq 190 \mathrm { km s ^ { -1 } } . Although our models do not contain primordial binaries , in the most compact clusters with 10 ^ { 3 } stars , we have found evidence of delayed SN explosions producing a surplus of BHs compared to the IMF due to dynamically formed binary stars . These cases do not occur in the more populous or expanded clusters . Conclusions :