We used functional magnetic resonance imaging (fMRI) to measure activity in human visual cortex, including a higher object processing area, the lateral occipital complex (LOC), and primary visual cortex (V1), in response to a perceptually bistable stimulus whose elements were perceived as either grouped into a shape or randomly arranged. We found activity increases in the LOC and simultaneous reductions of activity in V1 when the elements were perceived as a coherent shape. Consistent with a number of inferential models of visual processing, our results suggest that feedback from higher visual areas to lower visual areas serves to reduce activity during perceptual grouping. The implications of these findings with respect to these models are discussed.