Sites, here the eighth-century Buddhist shrine Borobudur and other remains of the Hindu-Buddhist past located in colonial (predominantly Islamic) Java, are in this article our analytical tool to provide insight into the local and transnational dimensions of heritage politics and processes of in- and exclusion in Asia and Europe around 1900. Because we recognize these “sites” as centers of multiple historical, political, and moral spaces that transgress state boundaries, we take this concept beyond the nation-state-centered lieu de mémoire. By exploring how site-related objects traveled from temple ruins in Java to places elsewhere in the world (here: Siam, the Netherlands, France, and Great Britain) and back to Java, we show the transformation of heritage engagements around 1900 at multiple locations, and we make clear why, despite professionalizing state-centered heritage politics, state control was limited. We argue that the mechanisms of exchange and reciprocal interdependence, as theorized by Marcel Mauss, are crucial to understand the moral and economic engagements that define the problem of heritage, at local and transnational levels.