Context : Infrared-faint radio sources ( IFRS ) are a class of radio-loud ( RL ) active galactic nuclei ( AGN ) at high redshifts ( z \geq 1.7 ) that are characterised by their relative infrared faintness , resulting in enormous radio-to-infrared flux density ratios of up to several thousand . Aims : Because of their optical and infrared faintness , it is very challenging to study IFRS at these wavelengths . However , IFRS are relatively bright in the radio regime with 1.4 GHz flux densities of a few to a few tens of mJy . Therefore , the radio regime is the most promising wavelength regime in which to constrain their nature . We aim to test the hypothesis that IFRS are young AGN , particularly GHz peaked-spectrum ( GPS ) and compact steep-spectrum ( CSS ) sources that have a low frequency turnover . Methods : We use the rich radio data set available for the Australia Telescope Large Area Survey fields , covering the frequency range between 150 MHz and 34 GHz with up to 19 wavebands from different telescopes , and build radio spectral energy distributions ( SEDs ) for 34 IFRS . We then study the radio properties of this class of object with respect to turnover , spectral index , and behaviour towards higher frequencies . We also present the highest-frequency radio observations of an IFRS , observed with the Plateau de Bure Interferometer at 105 GHz , and model the multi-wavelength and radio-far-infrared SED of this source . Results : We find IFRS usually follow single power laws down to observed frequencies of around 150 MHz . Mostly , the radio SEDs are steep ( \alpha < -0.8 ; 74 ^ { +6 } _ { -9 } % ) , but we also find ultra-steep SEDs ( \alpha < -1.3 ; 6 ^ { +7 } _ { -2 } % ) . In particular , IFRS show statistically significantly steeper radio SEDs than the broader RL AGN population . Our analysis reveals that the fractions of GPS and CSS sources in the population of IFRS are consistent with the fractions in the broader RL AGN population . We find that at least 18 ^ { +8 } _ { -5 } % of IFRS contain young AGN , although the fraction might be significantly higher as suggested by the steep SEDs and the compact morphology of IFRS . The detailed multi-wavelength SED modelling of one IFRS shows that it is different from ordinary AGN , although it is consistent with a composite starburst-AGN model with a star formation rate of 170 M _ { \odot } yr ^ { -1 } . Conclusions :