Recent functional anatomic models of speech suggest that there is a clear contribution to the process from both left and right hemispheres (e.g. Friederici, 2000; Hickok & Poeppel 2000). Building on this observation as well as the fact that the speech signal contains more than one time scale relevant to auditory cognition (e.g. time scales commensurate with processing formant transitions versus scales commensurate with syllabicity and intonation contours), I suggest a timing-based model - asymmetric sampling in time (AST) - that maintains anatomic symmetry while permitting functional asymmetry. AST proposes that the input speech signal has a neural representation that is bilaterally symmetric at an early representational level. Beyond the initial representation, however, the signal is elaborated asymmetrically in the time domain: left (non-primary) auditory areas preferentially extract information from short (~20-50ms) temporal integration windows; the right hemisphere homologues preferentially extract information from long (~150-250ms) integration windows. I discuss psychophysical and brain imaging (MEG, fMRI) experiments that support (i) that there are temporal integration windows of these sizes and (ii) that the different integration windows are deployed differentially in the two hemispheres. If this work is on the right track, it suggests that syllable-sized chunks and (sub)segmental-sized chunks are equally elemental for the brains representation and processing of speech signals.