We investigate the rest-frame UV morphologies of a large sample of Lyman- \alpha emitters ( LAEs ) from z \sim 2 to z \sim 6 , selected in a uniform way with 16 different narrow- and medium-bands over the full COSMOS field . We use 3045 LAEs with HST coverage in a stacking analysis and find that they have M _ { \mathrm { UV } } \sim - 20 , below M _ { \mathrm { UV } } ^ { \ast } at these redshifts . We also focus our analysis on a subsample of 780 individual galaxies with i _ { \mathrm { AB } } < 25 for which GALFIT converges for 429 of them . The individual median size ( r _ { e } \sim 1 kpc ) , ellipticities ( slightly elongated with ( b / a ) \sim 0.45 ) , Sérsic index ( disk-like with n \lesssim 2 ) and light concentration ( comparable to that of disk or irregular galaxies , with C \sim 2.7 ) of LAEs show mild evolution from z \sim 2 to z \sim 6 . LAEs with the highest rest-frame equivalent widths ( EW ) are the smallest/most compact ( r _ { e } \sim 0.8 kpc , compared to r _ { e } \sim 1.5 kpc for the lower EW LAEs ) . When stacking our samples in bins of fixed Ly \alpha luminosity and Ly \alpha EW we find evidence for redshift evolution in n and C , but not in galaxy sizes . The evolution seems to be stronger for LAEs with 25 < EW < 100 Å . When compared to other SFGs , LAEs are found to be smaller at all redshifts . The difference between the two populations changes with redshift , from a factor of \sim 1 at z \gtrsim 5 to SFGs being a factor of \sim 2 - 4 larger than LAEs for z \lesssim 2 . This means that at the highest redshifts , where typical sizes approach those of LAEs , the fraction of galaxies showing Ly \alpha in emission ( and with a high Ly \alpha escape fraction ) should be much higher , consistent with observations .