Neural network (NN)-based constitutive models have been used increasingly to capture soil constitutive response. When combined with the self-learning simulation (SelfSim) inverse analysis framework, NN models can be used to extract soil behavior when given field measurements of boundary deformations and loads. However, the data sets used to train and repeatedly retrain the NN models are large, and training times, especially when used in SelfSim, are long. A diverse set of stress–strain data is extracted from a simulated braced excavation problem to train a NN-based constitutive model. Several methods for reducing the data set size are proposed and evaluated. Each of these methods selectively removes training data so that the smallest amount of data is used to train the NN. The Gaussian point method removes data based on its position in each finite element in the model. The lattice method removes data so that all remaining points are evenly spaced in stress space. Finally, the loading path method compares the stress–strain history of each Gaussian point and removes points with similar loading histories. Each of these methods shows that a large amount of the training data (up to 94%) can be removed without adversely affecting the performance of the NN model, with the loading path method showing the best and most consistent performance. Model training times are reduced by a factor of 20. The performance of the loading path method is also demonstrated using stress–strain data extracted from a simulated laboratory triaxial compression test with frictional ends.