Previous studies showed reduced hemispheric asymmetry in face perception in bilinguals compared with monolinguals, suggesting that hemispheric asymmetry in visual stimulus processing may be modulated by language reading experience. Here we examined whether this phenomenon can also be observed in bilinguals with different language backgrounds. We compared English monolinguals, European–English bilinguals (who know two alphabetic languages), and Chinese–English bilinguals (who have mastered a logographic and an alphabetic language) in an English word sequential matching task. We showed that European–English bilinguals had a stronger right visual field/left hemispheric advantage than the other two groups, suggesting that different language experiences can influence how visual words are processed in the brain. In addition, by using a computational model that implements a theory of hemispheric asymmetry in perception, we showed that this lateralization difference could be accounted for by the difference in participants’ vocabulary size and the difference in word-to-sound mapping between alphabetic and logographic languages.