Three-dimensional image formation in an interference confocal scanning microscope under ultra-short pulsed beam illumination is investigated in this study. The novelty of this new image system is that it keeps advantages in femtosecond interferometry but also provides a femtosecond-resolved three-dimensional image without necessarily using an ultrafast detector. For a 5-fs pulsed beam illumination, spatial resolution in the axial and transverse directions in this system is improved by approximately 45% and 15%, respectively, compared with that in the case of continuous wave illumination. However, strong chromatic aberration caused by an ultrashort pulsed beam can result in a degradation of spatial and temporal resolution, whereas weak chromatic aberration may lead to an improvement in transverse resolution.