This study investigated the contribution of second-language (L2) phonetic categorization abilities and vocabulary size to the phonolexical encoding of challenging non-native phonological contrasts into the L2 lexicon. Two groups of German learners of English differing in L2 proficiency (advanced vs. intermediate) participated in an English lexical decision task including words and nonwords with /ɛ/ and /æ/ (/æ/ does not exist in German), an /ɛ/-/æ/ phonetic categorization task and an English vocabulary test. Results showed that the effects of phonetic categorization and vocabulary size on lexical decision performance were modulated by proficiency: categorization predicted /ɛ/-/æ/ nonword rejection accuracy for intermediate learners, whereas vocabulary did so for advanced learners. This suggests that sufficient phonetic identification ability is key for an accurate phonological representation of difficult L2 phones, but, for learners for whom robust phonetic identification is already in place, their ultimate success is tightly linked to their vocabulary size in the L2.