We investigate the stellar populations of Lyman \alpha emitters ( LAEs ) at z = 3.1 and 3.7 in 0.65 deg ^ { 2 } of the Subaru/ XMM-Newton Deep Field , based on rest-frame UV-to-optical photometry obtained from the Subaru/ XMM-Newton Deep Survey , the UKIDSS/Ultra Deep Survey , and the Spitzer legacy survey of the UKIDSS/UDS . Among a total of 302 LAEs ( 224 for z = 3.1 and 78 for z = 3.7 ) , only 11 are detected in the K band , i.e. , brighter than K ( 3 \sigma ) = 24.1 mag . Eight of the 11 K -detected LAEs are spectroscopically confirmed . In our stellar population analysis , we treat K -detected objects individually , while K -undetected objects are stacked at each redshift . We find that the K -undetected objects , which should closely represent the LAE population as a whole , have low stellar masses of \sim 10 ^ { 8 } – 10 ^ { 8.5 } M _ { \odot } , modest SFRs of 1 – 100 M _ { \odot } yr ^ { -1 } , and modest dust extinction of E ( B - V ) _ { \star } < 0.2 . The K -detected objects are massive , M _ { star } \sim 10 ^ { 9 } – 10 ^ { 10.5 } M _ { \odot } , and have significant dust extinction with a median of E ( B - V ) _ { \star } \simeq 0.3 . Four K -detected objects with the reddest spectral energy distributions , two of which are spectroscopically confirmed , are heavily obscured with E ( B - V ) _ { \star } \sim 0.65 , and their continua resemble those of some local ULIRGs . Interestingly , they have large Lyman \alpha equivalent widths \simeq 70 – 250 Å . If these four are excluded , our sample has a weak anti-correlation between Ly \alpha equivalent width and M _ { star } . We compare the stellar masses and the specific star formation rates ( sSFR ) of LAEs with those of Lyman-break galaxies ( LBGs ) , distant red galaxies , submillimetre galaxies , and I - or K -selected galaxies with photometric redshifts of z _ { phot } \sim 3 . We find that the LAE population is the least massive among all the galaxy populations in question , but with relatively high sSFRs , while NIR-detected LAEs have M _ { star } and sSFR similar to LBGs . Our reddest four LAEs have very high sSFRs in spite of large M _ { star } , thus occupying a unique region in the M _ { star } versus sSFR space .