We obtained deep optical imaging of the thermally emitting X-ray bright and radio-quiet isolated neutron star RX J1605.3+3249 with the Subaru telescope in 1999 and 2003 . Together with archival HST images acquired in 2001 these data reveal a proper motion of \mu = 144.5 \pm 13.2 mas/yr . This implies a relatively high spatial velocity and indicates that the star is unlikely to be re-heated by accretion of matter from the interstellar medium . Assuming that RX J1605.3+3249 is a young ( 10 ^ { 5 } -10 ^ { 6 } yr ) cooling neutron star , its apparent trajectory is consistent with a birth in the nearby Sco OB2 OB association at a location close to that derived for RX J1856.5-3754 and perhaps also to that of RX J0720.4-3125 . This suggests that the X-ray bright part of ROSAT-discovered isolated neutron stars is dominated by the production of the Sco OB2 complex which is the closest OB association and a part of the Gould belt . The B and R magnitudes of the faint optical counterpart did not vary from 1999 to 2003 at B = 27.22 \pm 0.10 . Its B-R colour index of +0.32 \pm 0.17 is significantly redder than that of other isolated neutron stars and the optical flux lies a factor 11.5 above the extrapolation of the X-ray blackbody-like spectrum . The red optical colour reveals the presence of an additional emitting component in the optical regime over the main neutron star thermal emission . We also discovered a small elongated H { \alpha } nebula approximately centered on the neutron star and aligned with the direction of motion . The width of the nebula is unresolved and smaller than \sim 0.4″ for a length of about 1″ . The shape of the Balmer emitting nebula around RX J1605.3+3249 is very different from those seen close to other neutron stars and should be confirmed by follow-up observations . We shortly discuss the possible mechanisms which could give rise to such a geometry .