An extensive literature going back three quarters of a century holds that metaphorical mappings between sensory domains conform to a hierarchy of the senses, such that mappings from ‘low’ senses (touch, taste) to ‘high’ senses (sight, sound) are preferred over mappings in the opposite direction. Recent work has established that these directional preferences are partially explained by lexical factors. Theorists have also proposed that perceptual factors play a role in directional preferences, but without testing these factors directly and without controlling for the established effects of lexical factors. This article uses a novel construction, the verbal analogy (e.g., The picture looks like my music sounds), to explore directional preferences while controlling for several crucial lexical factors. A naturalness rating experiment reveals local directional preferences, for mappings between touch and sound and between sight and sound. The experiment finds no evidence for a general preference for mappings in either direction of the purported hierarchy of the senses, suggesting that pervious empirical findings may have been mediated by the effects of lexical factors.