The fractional Brownian motion with index \alpha is introduced to construct the fractional excursion set model .
A new mass function with single parameter \alpha is derived within the formalism , of which the Press-Schechter mass function ( PS ) is a special case when \alpha = 1 / 2 .
Although the new mass function is computed assuming spherical collapse , comparison with the Sheth-Tormen fitting function ( ST ) shows that the new mass function of \alpha \approx 0.435 agrees with ST remarkably well in high mass regime , while predicts more small mass halos than the ST but less than the PS .
The index \alpha is the Hurst exponent , which exact value in context of structure formation is modulated by properties of the smoothing window function and the shape of power spectrum .
It is conjectured that halo merging rate and merging history in the fractional set theory might be imprinted with the interplay between halos at small scales and their large scale environment .
And the mass function in high mass regime can be a good tool to detect the non-Gaussianity of the initial density fluctuation .