Since radio sources with Ultra Steep Spectra ( USS ; \alpha \mathrel { \hbox to 0.0 pt { \lower 3.0 pt \hbox { $ \mathchar 536 $ } \hss } \raise 2. % 0 pt \hbox { $ \mathchar 316 $ } } -1.30 ;S \propto \nu ^ { \alpha } ) are efficient tracers of high redshift radio galaxies ( HzRGs ) , we have defined three samples of such USS sources using the recently completed WENSS , TEXAS , MRC , NVSS and PMN radio-surveys . Our combined sample contains 669 sources with S _ { 1400 } > 10 mJy and covers virtually the entire sky outside the Galactic plane ( |b| > 15 ° ) . For our 2 largest samples , covering \delta > -35 \degr , we selected only sources with angular sizes \Theta < 1 \arcmin . For 410 sources , we present radio-maps with 0 \aas@@fstack { \prime \prime } 3 to \sim 5″ resolution from VLA and ATCA observations or from the FIRST survey , which allows the optical identification of these radio sources . Comparison with spectrally unbiased samples at similar flux density levels , shows that our spectral index , flux density , and angular size selections do not affect the angular size distribution of the sample , but do avoid significant contributions by faint foreground spiral galaxies . We find that the spectral index distribution of 143,000 sources from the WENSS and NVSS consists of a steep spectrum galaxy and a flat spectrum quasar population , with the relative contribution of flat spectrum sources doubling from S _ { 1400 } > 0.1 Jy to S _ { 1400 } > 2.5 Jy . The identification fraction of our USS sources on the POSS ( R \mathrel { \hbox to 0.0 pt { \lower 3.0 pt \hbox { $ \mathchar 536 $ } \hss } \raise 2.0 pt% \hbox { $ \mathchar 316 $ } } 20 ) is as low as 15 % , independent of spectral index \alpha < -1.30 . We further show that 85 % of the USS sources that can be identified with an X-ray source are probably contained in galaxy clusters , and that \alpha < -1.6 sources are excellent Galactic pulsar candidates , because the percentage of these sources is four times higher in the Galactic plane . Our sample has been constructed to start an intensive campaign to obtain a large sample of high redshift objects ( z > 3 ) that is selected in a way that does not suffer from dust extinction or any other optical bias .