One of the least understood aspects of mammalian vision is the ability to recognize scenes through significant degradations in image quality. Neural receptive fields have traditionally been described with coherent structures - for example, oriented gratings in V1. However, this does not address how neurons respond to noisy, less coherent visual input, which is arguably more prevalent in the natural world. Previous studies with natural images show that recognition is highly non-linear with respect to noise, and more importantly, that recognition in noise is facilitated by prior experience with the stimuli (Sadr and Sinha, 2004). We extend these studies by using RISE sequences (Random Image Structure Evolution) to present structured images evolving from noise in an fMRI paradigm. Specifically, the direction of RISE evolution - ascending or descending in information content - allows us to control for low-level image features, such as luminance, while trending towards or away from a neuron's experimentally defined preferred stimulus. Any difference in response to ascending and descending stimuli thus reflects prior knowledge facilitating neural recognition in noise. In line with previous behavioral studies, we present evidence for this hysteretic facilitation throughout the visual hierarchy. Furthermore, we show a graded signature of hysteresis from V1 through IT, suggesting that prior knowledge affects lower and higher visual areas in different ways.