In various versions of the dungeon illusion (P. Bressan, 2001), we show that grouping between targets and contextual disks determines whether remote luminances affect target lightness or not. In the dungeon illusion, target disks surrounded by contextual disks contrast with them rather than with the immediate background. We formally establish the existence of this illusion and show that it reverses when the luminance of the targets is either lower (double decrement) or higher (double increment) than the luminances of both the background and the contextual disks rather than in between them. On the basis of the double-anchoring theory of lightness (P. Bressan, 2006a), we predict and show that grouping gates the effects of remote luminances in such a way that they go in opposite directions in the double-decrement and double-increment inverted-dungeon illusions. Our results support the double-anchoring theory and demonstrate that luminances that are far away from the targets are irrelevant in some conditions but critical in others.