Islands of the West Indies are among the most historically impacted by agriculture, yet agricultural influences on forests there have been little studied. This research compared tree species richness and vegetation structure between farmed lands, post-agriculture secondary forests and mature remnant forests in two watersheds in Saint Lucia, and sought to understand the current distribution of these habitats in terms of land use and watershed topography. Farms devoted to annual crops had few trees and much exposed soil. By contrast, agroforests had abundant (mostly planted) trees and vegetation structure comparable to secondary forests. Secondary forests had highest overall species richness, but mature forests had the most developed vegetation structure. Variations in habitat distribution reflected different land use histories, with the more rugged west coast long dominated by tree crop farming and the east coast experiencing a recent boom-bust cycle in bananas. Mature and secondary forests were more likely found at higher altitude, further from roads and at sites more difficult to access, the combined result of government protection of key forest and watershed reserves and farmers’ preferential abandonment of marginal lands. For conservationists, this return of forests is reason for optimism and it presents strategic opportunities for public land acquisition or collaborative management to further forest and watershed protection objectives.