This study reports pulse variation analysis results for the forth discovered accretion-powered millisecond pulsar XTE J1807-294 during its 2003 outburst observed by Rossi X-ray Timing Explorer . The pulsation is significantly detected only in the first \sim 90d out of \sim 150d observations . The pulse phase variation is too complex to be described as an orbital motion plus a simple polynomial model . The precise orbital parameters with P _ { orb } = 40.073601 ( 8 ) min and { \it a _ { x } } \sin { \it i } = 4.823 ( 5 ) lt-ms were obtained after applying the trend removal to the daily observed 150s segments pulse phases folded with a constant spin frequency without Keplerian orbit included . The binary barycenter corrected pulse phases show smooth evolution and clear negative phase shifts coincident with the flares seen on the light curve and the enhancements of fractional pulse amplitude . The non-flare pulse phases for the first \sim 60d data are well described as a fourth order polynomial implying that the neutron star was spun-up during the first \sim 60d with a rate \dot { \nu } = ( 1.7 \pm 0.3 ) \times 10 ^ { -13 } Hz/s at the beginning of the outburst . Significant soft phase lags up to \sim 500 \mu s ( \sim 10 % cycle ) between 2 to 20 keV were detected for the nonflare pulse phases . We conclude that the anomalous phase shifts are unlikely due to the accretion torque but could result from the “ hot spot ” moving on the surface of neutron star .