We revisit the cosmological constraints on resonant and non-resonant conversion of photons to axions in the cosmological magnetic fields . We find that the constraints on photon-axion coupling and primordial magnetic fields are much weaker than previously claimed for low mass axion like particles with masses m _ { a } \lesssim 5 \times 10 ^ { -13 } \text { eV } . In particular we find that the axion mass range 10 ^ { -14 } \text { eV } \leq m _ { a } \leq 5 \times 10 ^ { -13 } \text { eV } is not excluded by the CMB data contrary to the previous claims . We also examine the photon-axion conversion in the Galactic magnetic fields . Resonant conversion in the large scale coherent Galactic magnetic field results in 100 \% polarized anisotropic spectral distortions of the CMB for the mass range 10 ^ { -13 } \text { eV } \lesssim m _ { a } \lesssim 10 ^ { -11 } \text { eV } . The polarization pattern traces the transverse to line of sight component of the Galactic magnetic field while both the anisotropy in the Galactic magnetic field and electron distribution imprint a characteristic anisotropy pattern in the spectral distortion . Our results apply to scalar as well as pseudoscalar particles . For conversion to scalar particles , the polarization is rotated by 90 ^ { \circ } allowing us to distinguish them from the pseudoscalars . For m _ { a } \lesssim 10 ^ { -14 } \text { eV } we have non-resonant conversion in the small scale turbulent magnetic field of the Galaxy resulting in anisotropic but unpolarized spectral distortion in the CMB . These unique signatures are potential discriminants against the isotropic and non-polarized signals such as primary CMB , and \mu and y distortions with the anisotropic nature making it accessible to experiments with only relative calibration like Planck , LiteBIRD , and CORE . We forecast for PIXIE as well as for these experiments using Fisher matrix formalism .