Intracytoplasmic sperm injection (ICSI) is an important technique in male infertility treatment. Currently, sperm selection for ICSI in human assisted reproductive technology (ART) is subjective, based on a visual assessment by the operator. Therefore, it is desirable to develop methods that can objectively provide an accurate assessment of the shape and size of sperm heads that use low-magnification microscopy available in most standard fertility clinics. Recent studies have shown a correlation between sperm head size and shape and chromosomal abnormalities, and fertilization rate, and various attempts have been made to establish automated computer-based measurement of the sperm head itself. For example, a dictionary-learning technique and a deep-learning-based method have both been developed. Recently, an automatic algorithm was reported that detects sperm head malformations in real time for selection of the best sperm for ICSI. These data suggest that a real-time sperm selection system for use in ICSI is necessary. Moreover, these systems should incorporate inverted microscopes (×400–600 magnification) but not the fluorescence microscopy techniques often used for a dictionary-learning technique and a deep-learning-based method. These advances are expected to improve future success rates of ARTs. In this review, we summarize recent reports on the assessment of sperm head shape, size, and acrosome status in relation to fertility, and propose further improvements that can be made to the ARTs used in infertility treatments.