Brick-red deposits with palygorskite (Pal) as the main ingredient are widely distributed in nature, but these have not been deployed at a large scale in industry because of their inherent deep colors. In the present study, the brick-red Pal deposit was treated hydrothermally in various reaction media including water, a urea solution, and a thiourea solution. The effects of these processes on the structure, physicochemical features, and color of Pal were studied intensively to understand the structure and composition of the brick-red Pal deposit and to lay a theoretical foundation for the extension of its industrial application. The changes in structural features after hydrothermal treatment were studied by Fourier-transform infrared spectroscopy, X-ray diffraction, scanning electron microscopy, magic-angle spinning nuclear magnetic resonance, and Mössbauer spectroscopy techniques. The results indicated that the color of brick-red Pal did not change after hydrothermal treatment in water or in a urea solution, and the color changed to gray-white after treatment in the thiourea solution. The rod-like crystal morphology of Pal was retained throughout the experiments and no significant change in the main associated minerals, including feldspar, muscovite, and quartz, was observed after hydrothermal treatment. The dissolution of associated hematite (α-Fe2O3 and the reduction of Fe(III) species are the main reason for the change of Pal from brick-red to gray-white.