Accreting Millisecond X-ray Pulsars like IGR J00291+5934 are important because it is possible to test theories of pulsar formation and evolution . They give also the possibility to constrain gravitational wave emission theories and the equation of state of ultra dense matter . Particularly crucial to our understanding is the measurement of the long term spin evolution of the accreting neutron star . An open question is whether these accreting pulsars are spinning up during an outburst and spinning down in quiescence as predicted by the recycling scenario . Until now it has been very difficult to measure torques , due to the presence of fluctuations in the pulse phases that compromise their measurements with standard coherent timing techniques . By applying a new method , I am now able to measure a spin up during an outburst and a spin down during quiescence . I ascribe the spin up ( \dot { \nu } _ { su } = 5.1 ( 3 ) \times 10 ^ { -13 } Hz s ^ { -1 } ) to accretion torques and the spin down ( \dot { \nu } _ { sd } = -3.0 ( 8 ) \times 10 ^ { -15 } Hz s ^ { -1 } ) to magneto dipole torques , as those observed in radio pulsars . Both values fit in the recycling scenario and I infer the existence of a magnetic field for the pulsar of B \simeq 2 \times 10 ^ { 8 } G. No evidence for an enhanced spin down due to gravitational wave emission is found . The accretion torques are smaller than previously reported and there is strong evidence for an ordered process that is present in all outbursts that might be connected with a motion of the hot spot on the neutron star surface .