Context : Aims : We present optical long-slit spectra of the Virgo dwarf elliptical galaxy VCC 510 at high spectral ( \sigma \sim 30 km/s ) and spatial resolution . The principal aim is to unravel its kinematical and stellar population properties . Methods : Heliocentric velocities and velocity dispersions as functions of galaxy radius are derived by deconvolving line-of-sight velocity distributions . The luminosity-weighted stellar population parameters age and element abundances are obtained by comparison of Lick absorption-line indices with stellar population models . Results : A maximum rotation v _ { rot } = 8 \pm 2.5 km/s inside half the effective radius ( r _ { e } \approx 20 \arcsec ) and a mean , radially flat velocity dispersion \sigma = 44 \pm 5 km/s are measured . The core extending over the inner 2″ ( \sim 140 pc ) is found to rotate in the opposite sense with v _ { rot } ^ { core } \approx - 1 / 2 v _ { rot } . VCC 510 ( M _ { B } \sim - 15.7 ) is therefore by far the faintest and smallest galaxy with a counter-rotating core known . From the main body rotation and the velocity dispersion profile we deduce that VCC 510 is anisotropic and clearly not entirely supported by rotation . We derive an old luminosity-weighted age ( 10 \pm 3 Gyr ) and sub-solar metallicity ( [ Z / { H } ] = -0.4 \pm 0.1 ) inside the effective radius . There is tentative evidence that the counter-rotating core might be younger and less \alpha / { Fe } enhanced . From the stellar population parameters we obtain a total stellar mass-to-light ratio of \sim 3.6 ( M / L _ { B } ) _ { \odot } which is significantly lower than a rough dynamical estimate obtained from the kinematics through the virial theorem ( \sim 15 ) . This discrepancy hints toward the possible presence of dark matter in the centre of VCC 510 . Conclusions : We discuss the origin of the counter-rotating core and exclude fly-by encounters as a viable possibility . Gas accretion or galaxy merging provide more likely explanations . VCC 510 is therefore the direct observational evidence that such processes do occur in cluster satellite galaxies on dwarf galaxy scales .