We study the fluctuating motion of a Brownian-sized probe particle as it is dragged by a constant external force through a colloidal dispersion. In this nonlinear-microrheology problem, collisions between the probe and the background bath particles, in addition to thermal fluctuations of the solvent, drive a long-time diffusive spread of the probe's trajectory. The influence of the former is determined by the spatial configuration of the bath particles and the force with which the probe perturbs it. With no external forcing the probe and bath particles form an equilibrium microstructure that fluctuates thermally with the solvent. Probe motion through the dispersion distorts the microstructure; the character of this deformation, and hence its influence on the probe's motion, depends on the strength with which the probe is forced, Fext, compared to thermal forces, kT/b, defining a Péclet number, Pe = Fext/(kT/b), where kT is the thermal energy and b the bath particle size. It is shown that the long-time mean-square fluctuational motion of the probe is diffusive and the effective diffusivity of the forced probe is determined for the full range of Péclet number. At small Pe Brownian motion dominates and the diffusive behaviour of the probe characteristic of passive microrheology is recovered, but with an incremental flow-induced ‘microdiffusivity’ that scales as Dmicro ~ DaPe2φb, where φb is the volume fraction of bath particles and Da is the self-diffusivity of an isolated probe. At the other extreme of high Péclet number the fluctuational motion is still diffusive, and the diffusivity becomes primarily force induced, scaling as (Fext/η)φb, where η is the viscosity of the solvent. The force-induced microdiffusivity is anisotropic, with diffusion longitudinal to the direction of forcing larger in both limits compared to transverse diffusion, but more strongly so in the high-Pe limit. The diffusivity is computed for all Pe for a probe of size a in a bath of colloidal particles, all of size b, for arbitrary size ratio a/b, neglecting hydrodynamic interactions. The results are compared with the force-induced diffusion measured by Brownian dynamics simulation. The theory is also compared to the analogous shear-induced diffusion of macrorheology, as well as to experimental results for macroscopic falling-ball rheometry. The results of this analysis may also be applied to the diffusive motion of self-propelled particles.