Journal of Vision - Temporal dynamics of the human response to symmetry, by Norcia, Candy, Pettet, Vildavski, & Tyler

Anthony M. Norcia	Smith-Kettlewell Eye Research Institute, San Francisco, CA, USA
T. Rowan Candy	School of Optometry, Indiana University, Bloomington, IN, USA
Mark W. Pettet	Smith-Kettlewell Eye Research Institute, San Francisco, CA, USA
Vladimir Y. Vildavski	Smith-Kettlewell Eye Research Institute, San Francisco, CA, USA
Christopher W. Tyler	Smith-Kettlewell Eye Research Institute, San Francisco, CA, USA

Symmetry is a highly salient feature of animals, plants, and the constructed environment. Although the perceptual phenomenology of symmetry processing is well understood, little is known about the underlying neural mechanisms. Here we use visual evoked potentials to measure the time course of neural events associated with the extraction of symmetry in random dot fields. We presented sparse random dot patterns that were symmetric about both the vertical and horizontal axes. Symmetric patterns were alternated with random patterns of the same density every 500 msec, using new exemplars of symmetric and random patterns on each image update. Random/random exchanges were used as a control. The response to updates of random patterns was multiphasic, consisting of P65, N90, P110, N140 and P220 peaks. The response to symmetric/random sequences was indistinguishable from that for random/random sequences up to about 220 msec, after which the response to symmetric patterns became relatively more negative. Symmetry in random dot patterns thus appears to be extracted after an initial response phase that is indifferent to configuration. These results are consistent with the hypothesis (Lee, Mumford, Romero, & Lamme, 1998; Tyler & Baseler, 1998) that the symmetry property is extracted by processing in extrastriate cortex.

Symmetry is a highly salient feature of animals, plants, and the constructed environment. Symmetry extraction, along with contour processing, has been implicated as an important component of efficient shape representation (Kovacs, Feher, & Julesz, 1998; Labonte, Shapira, Cohen, & Faubert, 1995), object recognition (Marr, 1982), direction of visual attention independent of context (Yeshurun, Reisfeld, & Wolfson, 1992), and recovery of the shape of partially occluded objects (Dinnerstein & Wertheimer, 1957). Except near the symmetry axis, symmetry cannot be detected on the basis of local features; rather, comparisons of features must be made across the image. Symmetry detection is, therefore, a member of the class of binding problems that might be expected to share some common features with contour (Field, Hayes, & Hess, 1993; Kovacs & Julesz, 1993) and motion integration (Lorenceau & Shiffrar, 1992; Lorenceau & Zago, 1999).

Although the phenomenology of symmetry processing has been studied for many years (see references in Tyler, 1996), little is known about its neural basis. Lee, et al. (1998) have found that cells in area V1 of the alert behaving macaque are sensitive to the positioning of a figure around their receptive field. They observed enhanced responses after the initial burst of activity when receptive fields were centered on the medial axis (symmetry axis) of simple geometric figures defined by texture. They interpreted this late symmetry response as being generated after feedback from an extrastriate cortical area. Tyler & Baseler (1998) have contrasted blood oxygen level difference (BOLD) signals in human visual cortex generated by symmetric and random patterns. In their study, early retinotopic visual areas (V1, V2, V3, V3a V4v, V5) showed very little differential response to symmetric versus random patterns. Pronounced BOLD signals were, however, found in the middle occipital gyrus. Activation was not observed in the fusiform and lingual gyri, which are known to respond to objects and faces rather than to nonobject textures (Grill-Spector, Kourtzi, & Kanwisher, 2001).

We have developed a visual evoked potential (VEP) paradigm that allows us to measure what we propose to be a symmetry-related response component. Responses are measured to transitions between random dot fields with imposed two-fold symmetry and purely random fields. We found that the evoked response to these patterns is indistinguishable up to 130 to 220 msec, after which responses to symmetric/random sequences deviate from control responses to random/random transitions.

Seven visually normal adult observers, (3 males, 4 females, aged 19 to 48 years) participated. Each observer had 6/6 or better acuity in each eye, normal stereopsis on the Frisby free space stereo–test and was fully refracted for the viewing distance. The research followed the tenets of the World Medical Association Declaration of Helsinki and informed consent was obtained from the subjects after explanation of the nature and possible consequences of the study. The research was approved by the institutional human experimentation committee.

Random dot fields (12 by 12 deg) were comprised of bright dots (3 arcmin, 200 cd/m²) presented on a dark background (5 cd/m²) at 15% dot density. These fields were calculated online and drawn to video memory during vertical refresh. Symmetry was introduced into the patterns by reflecting the upper left quadrant of the pattern about the vertical axis and then reflecting the top pattern about the horizontal axis, thus creating two-fold symmetry about the horizontal and vertical axes (see Figure 1 for an illustration). New patterns, either symmetric or random, were presented every 500 msec, with the pattern remaining continuously visible for 500 msec between image updates. By redrawing the patterns on every transition, we avoided the possibility that the response is determined by a particular feature of any individual pattern. In a typical experiment, the observer was presented with 60 to100 different exemplars of symmetric patterns and a comparable number of random patterns.

Figure 1. Example frames of random dot patterns used to elicit symmetry-related visual evoked potential responses (not to scale). Random and symmetric patterns were exchanged every 500 msec, with new exemplars of each pattern type being supplied at each update. Symmetric dot patterns were created first by randomly drawing bright dots in the upper left quadrant of the display. These dots were then replotted, after mirror reflection, in the upper right quadrant of the display, and, finally, the upper half of the display was replotted, after mirror reflection, in the bottom half of the display. The display thus has two-fold symmetry about the vertical and horizontal axes. Random fields were generated by drawing the same total number of random dots over the whole display.

VEPs were recorded with Grass gold-cup electrodes from O1, Oz, and O2, each referenced to Cz. The skin was prepared with Omni-Prep, and 10-20 conductive cream (D.O. Weaver) was applied. Electrode impedances were between 3 and 10 kilo-ohms. The electroencephalogram was amplified 50,000 times with Grass Model 12 amplifiers and digitized to 16 bits accuracy at a sampling rate of 434 Hz. Analog filter settings were 0.3 to 100 Hz, measured at –6 dB points.

We calculated time averages of the response triggered on every other update of the dot patterns (1 sec epochs, corresponding to the period of one symmetric-, one random-pattern pairing). Spectral analyses were also performed via a Discrete Fourier transform of time averages summed over a 2-sec epoch, resulting in a spectrum resolution of 0.5 Hz. Trials lasting 10 sec were presented in blocks of 3 to 5 trials of the same stimulus condition, randomized over symmetric/random and random/random pairing in the main experiment. A total of 10 trials were collected in each stimulus condition. The observers were instructed to fixate the center of the display and to withhold eye blinks during the trial. The observers initiated trials with a button and could interrupt the trials or abort them as needed to retain blink-free fixation.

Time-averaged responses to a random-random sequence for one observer (Oz versus Cz) are shown in Figure 2. The main response consisted of a biphasic potential with an initial positive peak at 100 msec followed by a negative peak at 145 msec. There are two responses per epoch, and these responses are nearly identical (Red trace). Time-averaged responses to a symmetric-random sequence for the same observer are shown in Figure 2, Blue trace. The initial transition is from random to symmetric (symmetry onset), and the second transition at 500 msec is from symmetric to random patterns (symmetry offset). The initial activation with symmetric/random sequences is the same shape as that for random/random updates (compare Red and Blue traces in Figure 2). However, the response to symmetric/random sequences becomes more negative starting at 200 msec after a random-to-symmetric transition and more positive starting at 165 msec after transitions from symmetric to random patterns.

Figure 2. Time-averaged responses from a single observer recorded at Oz. Red trace indicates the response to random/random image updates. Each image update lasted 500 msec (double lines indicate transition times). Blue trace indicates the response to symmetric/random sequences. The first half of the record shows the response to a transition from random to symmetric patterns and the second half (after 500 msec) indicates the response to transitions from symmetric to random patterns. The initial positive and negative peaks are independent of the type of transition. After 200 msec from the transition from random to symmetric patterns, the response is more negative relative to that measured after the transition between random patterns. Conversely, after about 165 msec after the transition from symmetry, the response is more positive than that recorded after random/random pattern updates.

Figure 3 shows grand average waveform data from all 7 observers plotted as a function of recording channel for random/random sequences (Red) and symmetric/random sequences (Blue). The response comprises an initial positive peak at 65 msec, followed by a negative peak at 90 msec, a positive peak at 110 msec, and then a second negative peak at 140 msec. Activity across this complex is very similar after any of the different image update types (random to random, symmetric to random, random to symmetric).

Figure 3. Grand average spectra (n =7; top left panels) and time averages (top right panels) as a function of recording site. Red indicates the response to random/random image updates, and Blue indicates the response to symmetric/random sequences. Top row: O2; middle row: Oz; bottom row: O1. The response spectrum for the random/random updates is composed solely of even harmonics of the 1-Hz stimulus frequency. In contrast, the response to symmetric/random sequences contains low frequency odd-harmonic components. Top right panel. The first half of the record shows the response to the transition from random to symmetric patterns and the second half (after 500 msec) indicates the response to transitions from symmetric to random patterns. The responses to the same periods during random/random stimulation (Red) are superimposed. The initial positive and negative peaks are independent of the type of transition. Bottom panel. Difference potentials (symmetry/random minus random/random). After 220 msec from the transition from random to symmetric patterns, the response is more negative relative to that measured after the transition between random patterns. Conversely, after about 130 msec after the transition from symmetry, the response is more positive than that recorded after random/random pattern updates.

The response to the random/symmetric transition first diverges from the random/random sequence response at 220 msec. At this point, the response to the symmetric pattern shows a sustained relative negativity. Conversely, the response to the transition from symmetric to random patterns shows a relatively positive, sustained deviation from the random/random sequence about 130 msec after the initial transient response to the second image update in each sequence. Responses to symmetric/symmetric sequences were measured in three observers (data not shown). These responses contained no odd –harmonics and were very similar to those evoked by random/random sequences. The details of the initial activation differed substantially across observers. However, in each observer, the initial activation is essentially the same in all stimulus conditions.

In each observer, the response spectrum for random/random updates was composed of even harmonics of the 1-sec base period, whereas the spectrum obtained from symmetric/random sequences contained odd harmonics as well as even harmonics. The even harmonic structure corresponds to aspects of the waveforms that are identical for symmetry onset and offset, and hence are blind to the presence of symmetry. The odd harmonic components aggregate all aspects of the response that differentiates between the appearance and disappearance of symmetry, and, therefore, represent the identifiable symmetry response. The random/random response spectrum (Figure 3, lower plot in Red) comprises a series of even harmonic components extending to approximately 30 Hz. The spectrum has a band-pass characteristic, peaking in the 8 to –20-Hz region, consistent with the multiphasic, time-averaged response. The spectrum of the response to symmetric/random patterns contains prominent, low-frequency, odd harmonic components (Figure 3, upper plot in Blue), with the even harmonic components being quite similar to those of the random/random response, especially at middle and higher frequencies. All 7 of the observers produced robust odd harmonic responses to symmetry/random sequences that were absent in the response to random/random sequences. The average amplitude of the first harmonic in the symmetry/random condition was 1.73 + 0.17 microvolts, but it was only 0.24 + 0.07 microvolts in the random/random condition (p < 0.0001; paired t test). In contrast, the amplitudes at the second harmonic did not differ (1.68 + 0.65 and 1.39 + 0.64 microvolts; p = 0.76) for the symmetry random versus random/random cases. Amplitudes at the 8th harmonic, which are representative of the higher even harmonics, also did not differ in the two conditions (1.19 + 0.41 and 1.11 + 0.39 microvolts; p = 0.89).

VEP selectivity for the appearance of symmetry in random dot fields first emerges at about 220 msec after an image update. The transition to randomness from symmetry is detected sooner—by about 130 msec after the transition occurs. The asymmetry in response timing for the transition from disordered to order versus the return to the disordered state is reminiscent of similar timing asymmetry that has been observed psychophysically with random dot stereograms and correlograms (Julesz & Tyler, 1976 ; Tyler & Julesz, 1976). In the VEP, both types of selective activity are relatively sustained, lasting almost to the point of the next image update. These results are the first to indicate the absolute timing of symmetry processing in the human visual system. Previous psychophysical studies of the dynamics of symmetry processing have measured the minimum exposure duration needed to extract symmetry. Symmetry in static random dot fields can be extracted with as little as 40 msec of exposure (Tyler, Hardage, & Miller, 1995). Exactly when, in absolute terms, this minimal packet of information is processed cannot be determined psychophysically from exposure duration measurements. On the basis of an analysis of the form of the duration psychometric function, Tyler et al., (1995) concluded that symmetry information was being fully integrated over more than a second. This is consistent with the sustained nature of the configural (predominately odd harmonic) part of the symmetry-evoked response. Our results thus suggest that information about symmetry is extracted relatively late and in a sustained fashion after an early transient stage of form processing, which begins at about 50 msec and extends to 130 to 220 msec, and is insensitive to symmetry.

The present results are broadly similar to previous work on texture-based evoked responses. Victor and coworkers (Victor, 1985; Victor & Zemon, 1985, Victor & Conte, 1991) have measured VEP responses generated by iso-dipole texture pairs. Each member (odd/even) of the texture pair has identical first, second, and third-order element statistics, yet they yield readily distinguishable spatial structures. Iso-dipole textures also share the same first- through third-order statistics with a random checkerboard. The evoked response to an exchange between two differently structured iso-dipole textures is identical to that evoked by random/random exchanges during an initial period up to the initial positive response near 115 msec (Victor & Conte, 1991; Figure 4). Similarly, responses to the transition between even and odd exemplars are the same as those from odd to even, up to the first positive peak, diverging shortly thereafter (Victor & Zemon, 1985; Figure 2). However, the divergence point for iso-dipole textures was about 100 msec earlier than we have observed for symmetric dot patterns, suggesting that higher-order texture is processed at an earlier stage than symmetry. Victor and coworkers also noted that the transition between iso-dipole textures elicited significant odd-harmonic activity and, in fact, most of the quantitative analyses of the iso-dipole texture response have been done in the frequency domain. Purpura, Victor, & Katz (1994) found responses to iso-dipole textures in macaque area V1 single-units and local-field potentials.

More recently, several studies (Bach & Meigen, 1992; Lamme, van Dijk, & Spekreijse, 1992, 1993; Meigen & Bach, 1993; Bach & Meigen, 1997; Caputo & Casco, 1999; Caputo, Romani, Callieco, Gaspari, & Cosi, 1999; Romani, Caputo, Callieco, Schintone, & Cosi, 1999) have examined evoked responses to texture-defined forms. In these studies, a global form is defined on the basis of a gradient in the orientation of small line segments. In each case, there is an initial response that is independent of configuration, followed by a mid-latency response (c.a., 100-200 msec) that is sensitive to the textural configuration. The response to the appearance of the texture-defined form is more negative at mid latencies relative to the response to the disappearance of the form (Caputo & Casco, 1999). We observe a more negative mid-latency response to the appearance of symmetry relative to that measured after the disappearance of symmetry. As in the case of iso-dipole textures, texture-defined form responses appear to begin sooner than the response to symmetry. Within observer comparisons across the different tasks with detailed topographic mapping would be useful in more precisely defining the relative timing and sources of texture-related versus symmetry-related activity.

At this point it is not possible to state conclusively that the responses we observe are specific to symmetry per se rather than the appearance of perceptually salient global structures. One could argue that symmetry is simply another way of defining a global form and that it is thus not surprising that the symmetry-evoked VEP is similar in a number of respects to the VEP elicited by texture-defined forms. At a more abstract level, perhaps all that is necessary is for the images to differ in specific higher-order statistics (c.f., Victor & Conte, 1991). All globally defined stimuli have statistical regularities, as do iso-dipole textures, but the latter do not yield global forms in the sense that the stimuli used in the texture segmentation VEP do.

On the other hand, all stimuli used in the texture segmentation VEP studies reviewed above are highly symmetric, and, perhaps, symmetry-specific processing mechanisms were contributing to the notional texture-segmentation response. In the case of iso-dipole textures in which there are no figure-ground relationships, Victor & Conte (1991) have discounted symmetry as the sole determinant of the strength of the iso-dipole texture response— iso-dipole response magnitudes were not strictly correlated with the degree of symmetry in the local recursion rule used to generate the textures. However, global symmetry played no role, because iso-dipole patterns do not display global symmetries.

It appears that aspects of statistical regularity (Victor, 1985) as well as higher level form properties. such as figure-ground relationships (Caputo et al., 1999) and symmetry, may each play a role in generating the mid-latency activity observed across studies.

The most direct evidence regarding the source of our symmetry-related responses comes from functional magnetic imaging (fMRI) conducted with similar stimuli (Tyler & Baseler, 1998). As noted in the "Introduction," symmetry-related activation was most prominent outside of the early retinotopic visual areas. These results, combined with the relatively late emergence of selectivity for symmetry we have seen in the VEP, suggest that symmetry in random dot fields may first be extracted in nonretinotopic extra-striate visual areas. This interpretation is consistent with the data of Lee et al. (1998) regarding the effects of symmetry-axis activation in V1 cells. Lee et al. (1998) have proposed that medial-axis sensitivity in V1 cells was not generated directly in V1, but was the result of feedback from higher visual areas. They based this proposal on their observation that the medial axis response emerged at 60 to 80 msec, after an initial configuration independent transient response to stimulus onset that began at about 20 msec after stimulus onset.

Orientation-defined forms similar to those used in VEP studies have been reported to not evoke significant differential activation in areas V1 and V2 in human (Kastner, De Weerd, & Ungerlieder, 2000). Rather, BOLD activation is strongest in areas V4, TEO, and less reliably in area V3A. Texture-defined forms thus appear to activate a different subset of extra-striate visual areas, compared to those activated by symmetric random dot fields (middle occipital gyrus but not V4 (Tyler & Baseler, 1998). Iso-dipole textures elicit BOLD activation in human most consistently in the anterior fusiform gyrus, but also in striate, middle occipital, lingual, and posterior temporal regions (Beason-Held et al., 1998a; see also Beason-Held et al., 1998b for similar positron emission tomography data). Iso-dipole textures thus appear to activate some of the areas activated by symmetric random dot fields, but it is notable that they also activate striate cortex. The human imaging results of Kastner et al. (2000) on texture-defined form contrast with the electrophysiological results of Lamme, 1995; Zipser, Lamme, & Schiller, 1996; and Lee et al., (1998) who have found sensitivity to texture-defined forms in area V1 using local field potentials and single-unit activity as response measures in alert, behaving macaques. It is thus possible that current fMRI methods are not as sensitive as invasive electrophysiological measures. The patterns of cortical activation evoked by the different classes of stimuli suggest that some areas may be specific to one stimulus class, while others may be jointly activated. A general finding across higher-order pattern stimuli is that extra-striate cortical mechanisms are most strongly activated.