It is well known that the integration of facial features into a holistic representation is dramatically disrupted by picture-plane inversion. To investigate the nature of this observation, we tested for the first time the so-called face composite effect at various angles of rotation (0° to 180°, 7 angles). During an individual face matching task, subjects perceived two identical top halves of the same face as being slightly different (increase of error rates and RTs) when they were aligned with different bottom parts. This face composite illusion was equally strong for stimuli presented at 0° until 60° rotation, then fell off dramatically at 90° and remained stable until complete inversion of the stimulus. The non-linear relationship between orientation and holistic processing supports the view that inversion affects face processing qualitatively. Most importantly, it rules out the hypothesis that misoriented faces are perceptually realigned by means of linear rotation mechanisms independent of internal representations derived from experience. Altogether, these observations suggest that a substantial part of the face inversion effect is accounted for by the inability to apply an experience-derived holistic representation to an incoming visual face stimulus that it is rotated horizontally or beyond that orientation.