A three-dimensional aggregate generation and packing algorithm applicable for modeling asphalt mixture with high content of graded aggregates is presented in this paper. In the algorithm, arbitrary-shaped polyhedra are used to model aggregates, so that the effect of aggregate shape on the mechanical performance of asphalt mixture can be considered. The algorithm consists of two steps: Aggregate generation and packing. Polyhedra are created by extending triangular fundaments and treated as visualized aggregates after passing through convex control and sharpness judgment. After that, graded aggregates are taken out from the aggregate base and randomly packed in a given cylindrical or cubical region one by one equiprobably. Overlapping between nearby aggregates is avoided by the help of Boolean partition operation in ANSYS. Finally, some asphalt mixture samples with a given gradation are modeled as examples, and their effective elastic properties and creep behaviors under uniaxial compression are simulated.