Murray et al. (Nat. Neurosci., 2006) demonstrated that a distant sphere that appears to occupy a larger portion of the visual field activates a larger area in V1 than a sphere of equal angular size that is perceived to be closer and smaller. To assess whether this effect is due to an overall greater distribution of activity or due to a positional shift in the neural representation of spatial extent, we presented tori at close and far apparent depths in a rendered three-dimensional scene of a hallway and walls. When fixating its center of mass, the far torus appeared to be larger and occupy a more eccentric portion of the visual field, relative to the close torus. Using functional magnetic resonance imaging, we found that the spatial pattern of V1 activation induced by the far torus was also shifted towards a more eccentric representation of the visual field , while that induced by the close torus was shifted towards the foveal representation, consistent with their perceptual appearances. The peaks of the spatial profiles of V1 activation as a function of eccentricity induced by these two tori were at different positions. This effect was found not only when subjects attended to the torus, but also when they did a demanding fixation task (although weaker). Together with the previous study (Murray et al., 2006), these results strongly suggest that the retinal size of an object and the depth information in a scene are combined early in the human visual system resulting in positional shifts in V1 cortical activation. This is mainly a stimulus-driven process because it is largely independent of attention.