A striking illusion of motion is generated by static repeated asymmetric patterns (RAPs) such as Kitaoka's (2003) “Rotating Snakes” and Fraser and Wilcox's (1979) peripheral drift illusion. How do RAPs generate spurious motion signals, and what critical difference between RAPs and natural static scenes prevents the latter from appearing to move? Small involuntary eye movements during fixation have been suspected to play a critical role in these illusions, but here we give an account that does not depend on fixation jitter. We propose that these illusions result primarily from fast and slow changes over time in the neuronal representation of contrast (“contrast-driven RAPs”) or luminance (“luminance-driven RAPs”). We show that temporal phase advance in the neural response at high contrast can account for the early, fast motion in contrast-driven RAPs (such as “Rotating Snakes”) after each fixation change. An essential part of this explanation is that motion detectors fail to compensate for the dynamics of neuronal encoding. We argue that static natural patterns also generate local gain changes, but that these signals do not often trigger illusory motion because they are not usually aligned to drive global motion detectors. Movies in which real luminance changes over time, to mimic the proposed neuronal adaptations to contrast and luminance, evoke qualitatively similar percepts of motion. Experimental data are consistent with the explanation. Color and overall contrast both enhance the illusion.