Constraining solution space to improve generalization

Abstract

I suggest that the difficulties inherent in discovering the hidden regularities in realistic (type-2) problems can often be resolved by learning algorithms employing simple constraints (such as symmetry and the importance of local information) that are natural from an evolutionary point of view. Neither “heavy-duty nativism” nor “representational recoding” appear to offer totally appropriate descriptions of such natural learning processes.