Multi-pass warm rolling with falling temperature was proposed and investigated to obtain AZ31 Mg alloy sheets with a fine-grained microstructure. The results indicated that the grain microstructure of AZ31 alloy sheets was successfully refined from 22.1 to 4.5 μm after multi-pass warm rolling with falling temperature and annealing. Compared to the as-received sheet, the multi-pass warm rolled sheets in annealed condition exhibited weaker (0001) basal texture intensity, which resulted in the significantly increased Schmid factor of 〈a〉 basal slip. After multi-pass warm rolling with falling temperature, the rolled sheets in annealed condition also exhibited much better mechanical properties, e.g., higher tensile strength, larger fracture elongation, and higher Erichsen value, especially the IE of 8-pass warm rolled sheet in annealed condition was significantly increased by ∼33% under the same thickness, which could be attributed to the refined grain microstructure and the weakened basal texture.