Context : Infrared-faint radio sources ( IFRS ) are objects that have flux densities of several mJy at 1.4 GHz , but that are invisible at 3.6 \mu m when using sensitive Spitzer observations with \mu Jy sensitivities . Their nature is unclear and difficult to investigate since they are only visible in the radio . Aims : High-resolution radio images and comprehensive spectral coverage can yield constraints on the emission mechanisms of IFRS and can give hints to similarities with known objects . Methods : We imaged a sample of 17 IFRS at 4.8 GHz and 8.6 GHz with the Australia Telescope Compact Array to determine the structures on arcsecond scales . We added radio data from other observing projects and from the literature to obtain broad-band radio spectra . Results : We find that the sources in our sample are either resolved out at the higher frequencies or are compact at resolutions of a few arcsec , which implies that they are smaller than a typical galaxy . The spectra of IFRS are remarkably steep , with a median spectral index of -1.4 and a prominent lack of spectral indices larger than -0.7 . We also find that , given the IR non-detections , the ratio of 1.4 GHz flux density to 3.6 \mu m flux density is very high , and this puts them into the same regime as high-redshift radio galaxies . Conclusions : The evidence that IFRS are predominantly high-redshift sources driven by active galactic nuclei ( AGN ) is strong , even though not all IFRS may be caused by the same phenomenon . Compared to the rare and painstakingly collected high-redshift radio galaxies , IFRS appear to be much more abundant , but less luminous , AGN-driven galaxies at similar cosmological distances .