4

From Temporal Cuts to Spatial Composition

Correspondence, not collision.

11 min read

Two Kinds of Musical Work

I came to montage through two kinds of musical work: deciding how parts should meet, and playing among people who could change what happened next.

Arranging Bach's violin partitas for banjo with Noam Pikelny made the work of placement tangible. We had to map what the hands could do, where they could shift, how a phrase could survive the crossing. In arranging, I could decide which line would meet another, how long a tension would last, which chord would receive it. Those relations could be composed before the performance began. The tablature held decisions the hands would have to take up.

Those relations are composed, but the score cannot decide everything that happens when bodies take them up. Timing, touch, the breath before an entrance remain matters for the players. The arrangement gives them a structure to answer from. What I am calling collision is the composer's placement of elements, not the absence of response among the people sounding them.

First Cycle · The Session Corresponds

In the bluegrass session, montage was something else. No conductor. No score. No hierarchy of chairs. The session is an amoebaself-regulating, leaderless, the roles distributed across the bodies in the circle like organs in a single creature. The bass holds the 1 and the 3. The mandolin chops the upbeats. The guitar braids those two together. The fiddle pads or leads. And the banjothe perpetual motion machinebraids subdivisions of rhythm through harmonic and melodic information, a continuous stream that holds the organism together.

Nobody decides that the banjo's roll will collide with the fiddle's sustain at this particular moment. The meaning is not produced by collision because the elements are not colliding. They are correspondingmoving along together, each shaping the next phrase the next body will play, each adjusting in real time to what the others have done.

The arranger can compose a collision.

The session corresponds.

Everything turns on this distinction. Eisenstein's cinema, developed for the editing room, theorizes only the firstan editor controlling the sequence and timing of collisions. The second has been happening in human bodies for a long time, and cinema is not its culmination. Cinema is the limit case where the ensemble is reduced to one and the audience is reduced to receivers.

XR can stage both. But the medium uniquely enables the second. In cinema, the editor has total control. In XR, the participant moves freely through a field of simultaneous elements, and the relationships she produces with the environment are negotiated in real time between body and space.

Correspondence is not a single mechanism. The session showed me one form of it. There are others.

There is the choir. I have stood in the middle of a chorus and felt the pitch arrive in my chest before I produced itthe section tuning itself through bone conduction, through the lateral pressure of bodies, through the audio-vocal loop closing across an entire ensemble. Heart rates synchronize across singers during unison passages; the autonomic nervous systems of twenty people lock into a shared rhythm by way of the breath the music demands.

This is correspondence at the somatic level. Bodies coupling through breath and bone conduction into a single resonating organism, the boundaries between performers becoming acoustically and physiologically porous. Christopher Small called it musicking: the enactment of relationships, and the relationships are the content.

The choir does not perform a piece. The choir performs itself performing the piece, and the audience is folded into the resonance whether they sing or not.

And there is the jugalbandi. Two musiciansa soloist and a tabla player, or two soloists from different traditionsshare a raga and a rhythmic cycle, and the music they produce is negotiated in real time. The soloist proposes a phrase. The tabla answers. The phrase mutates. Twenty minutes in, a melodic figure introduced at the start returns transfigured by everything that happened to it, and both musicians arrive at the samthe downbeat that closes the cycletogether, as if they had planned it.

They had not planned it. They had listened to each other across twenty minutes of mutual shaping, and the arrival was inevitable from the first phrase.

This is correspondence with memory. Not just bodies coupling in the present, but bodies folding the entire arc of an unfolding event into every subsequent gesture.

The session corresponds in the moment.

The choir corresponds through physiology.

The jugalbandi corresponds through time.

Three deepenings of the same fundamental move. The session is horizontal and immediate. The choir is somatic and entraining. The jugalbandi is temporal and recursive. None of them is what cinema does. All of them are what XR can do.

Second Cycle · Two Lines That Move Along Together

Tim Ingold gives the vocabulary. In The Life of Lines, he distinguishes between interaction and correspondence. Interaction is what happens between two discrete entities that come into contact and then separatebilliard balls, cuts in a film, shots colliding. Correspondence is what happens between two lines that move along together, each shaping the other through co-motion, neither preceding nor following the other but unfolding in parallel.

The river and the bank correspond. The walker and the path correspond. The fiddle and the banjo correspond.

Cinema is interactive. The composed collision is interactive. The session is correspondent. The choir is correspondent. The jugalbandi is correspondent. A written arrangement can contain both kinds of work: composed collisions and the correspondence of the players who perform them.

And the body of the participant in an XR space, moving among elements that respond to her presence, is correspondent.

This is the distinction the medium needs. Eisenstein's collision is real, and XR can stage collisionshard frame transitions, blackouts between scenes, the participant rounding a corner into an unexpected tableau. But the dominant mode in spatial composition is not collision. It is correspondence. The participant's path and the environment's response unfolding together, each shaping the other, neither one the editor and neither one the edited.

In cinema, the cut happens to the viewer. In correspondence, there is no cut. There is only the continuous co-motion of body and world, with meaning emerging from the shape that co-motion describes.

The cut has not migrated from the timeline to the body. The cut has been replaced.

By the line.

The editorial gesture is no longer the splice. It is the trace. What the participant carries away from a spatial composition is the shape of her own line through the space, and the shape that line drew in correspondence with the environment that moved alongside it.

Listening is composition. The listener is a participant in the sonic environment, shaping it by attending to it, becoming part of what the environment is. And an improvising ensemble is distributed cognition, with no boundary between performer and audience left to blur.

The music is the relationship. The relationship is the music.

Spatial composition in XR is closer to the ensemble and the listening field than to Eisenstein's editing room.

The Field, the Threshold, the Trace

Three operations describe how the designer composes for correspondence.

The field is the condition for correspondence. The designer is not making the participant's line. The designer is making the environment through which lines will travel, and the environment has been composed so that any line drawn through it produces meaning. Nobody tells the guest where to go; the environment is shaped so her natural tendencies lead her through a composed sequence.

The participant feels free because she is free. The designer feels in control because the field has been composed.

Both are true.

Fifteen rooms, and no two participants will ever walk them in the same order. Every ordering traces a coherent shape, because the field has been built for traversal rather than for a single correct path.

The threshold is the exception. Abraham Burickson defines a frame as "the lines we draw around a thing so we know what it is and what it is not"and in spatial composition, every frame the participant crosses is a threshold, the one place where correspondence yields briefly to collision. A doorway is a threshold. A change in lighting from one room to the next is a threshold. A boundary between biomes is a threshold the participant crosses with her whole body.

The designer composes the thickness of thresholds the way a composer chooses between a fermata and a caesura. A thick thresholda long corridor, a blackoutcreates a hard cut. A thin thresholda slow crossfade of ambient sound, a permeable scrimbarely interrupts correspondence at all. The participant finds herself in a different state without registering when it changed.

Thresholds are where the designer reasserts something like the editor's cut, inside an experience that is otherwise unbroken co-motion.

The trace is what the participant produces.

Her line through the field is the actual content of the experienceinscribed in the path her feet describe on the floor, written in the rhythm of her dwells and returns.

The designer composes the conditions. The participant composes the trace. The meaning is the relationship between them.

The lines she corresponds with need not be lines of the present. A place is old. The street of 1880 still runs under the street she walks; a voice can lay another era over the corner where she stands. These are lines toolonger than hers, slower, still unfinishedand a walking body can move along with them as surely as it moves along with a fiddle. This is correspondence with memory at the scale of a place. The walker does not travel back. She folds what the ground has been into what she is doing on it now, the way the jugalbandi folds its opening phrase into the sam. A timeline cannot do this. A timeline cuts from era to era, each replacing the lastthe editor's splice stretched across a century. A place that holds its times open, all of them at once, does not cut.

The timeline collides. The layered place corresponds.

Third Cycle · Reading the Trace

How do we evaluate spatial composition? By reading the trace.

Path topology. What shape does the line describe? A straight line is a participant who treated the space as a corridor. A spiral is a participant who treated it as a conversation. More complex paths indicate richer correspondencethe participant entering deeper into co-motion with the environment, less concerned with traversing and more with dwelling.

Correspondence density. How tightly is the line coupled to the environment's responses? How often does movement trigger meaningful change, and how often does change redirect movement? High correspondence density means the participant and the space are effectively musickingeach shaping the other in real time.

Dwell patterns. In temporal montage, the editor controls pacing. In spatial composition, the participant controls pacing through dwelling. Dwell is the body's intuitive assessment of where the correspondence is richestwhere she has found a moment of mutual shaping she does not want to leave.

Dwell maps the live wires.

Return frequency. Returns are the jugalbandi's mechanism. The participant comes back to an earlier element because the correspondence is not yet completebecause something about the first encounter wants to be folded into a later one. The sam is not an arrival. It is a recognition.

The participant has been corresponding with the space all along. The return is the moment when the correspondence becomes audible to her.

Retrospective coherence. When the participant looks back, does her line hold together as a shape? The ultimate measure is whether the correspondence has composed itself into something she carries with her after the field is gone.

The participant is not receiving a montage. She is performing onethe way I perform one every time I sit in a session.

Every time I sit in a circle.

Sam · The Body Remembers What the Mind Does Not Name

I did not know, sitting in those circles, that I was learning the principles of spatial composition.

I did not know that the felt sense of multiple bodies coupled through breath and rhythm and shared attentionthe bass below me, the fiddle to my right, the tabla anticipating my next phrase, the alto section breathing as one creaturewould become the foundation of my understanding of how to design experiences in XR.

But the body remembers what the mind does not name.

And when I first put on a headset and moved through a designed space—hearing sounds from multiple directions, seeing events unfold at different distances, feeling the environment respond to my movement—I recognized a composition still being made.

I recognized the session.

I recognized the choir. I recognized the cycle of the tala, the sam I was about to arrive at without knowing how I would get there.

The cut had not migrated from the timeline to the body. The cut had been replacedby the line. By correspondence. By the slow mutual shaping of body and world that every musical tradition I had inherited already knew.

The designer of an XR experience is not the conductor.

The designer is the elder who chose the tunes, set the key, and sat down to play.

The participant is the line.

The space is the field.

The meaning is the trace they draw between them.

Experience these ideas

  • The Windows Project

    Looking, dwelling, and returning compose a path through material that shares one space.

  • The Room of Four Tellings

    The room makes attention consequential. Its record of your route lets you return to what you followed and what was happening elsewhere.

Explore One World, Many Games