I. Two Ways Into a Room
There are two ways to understand what happens when you enter a new XR environment for the first time.
The first way is Wittgenstein's. You are entering a form of life. You do not know the rules yet, but the rules exist—not as a posted list of commands but as patterns of use that a community has established through practice. A reach means something. A head tilt means something. Proximity means something. You learn what these things mean the way a child learns language: by watching, by imitating, by trying and failing and being corrected—not by explicit instruction but by the responses your actions provoke. You learn to play the game by playing it. This is what I learned in day school before I had the philosophy for it. A page of Talmud is a language game made visible—multiple forms of life arguing on the same surface, each with its own rules, none reducible to the others. The student's job is not to resolve the argument but to learn to play in all the games simultaneously.
The second way is Derrida's. You are entering a text. Every surface is already inscribed. The floor shows wear patterns from prior visitors. The objects carry the traces of prior handling—repositioned, annotated, marked. The walls hold layered records of prior sessions: a technician's arrows pointing to a valve, a designer's sketch hovering over a prototype, a student's hesitant path recorded as a ghost trail. You do not just interact with this environment. You read it. And everything you do—every gesture, every path, every placement—adds to the inscription. You are simultaneously reader and writer of a spatial text that was begun before you arrived and will persist after you leave.
These are not competing accounts. They are two dimensions of the same phenomenon, and understanding their relationship is essential to designing XR interactions that feel alive rather than mechanical.
Wittgenstein tells you how meaning arises in the moment: through use, in context, among participants. Derrida tells you how meaning persists across moments: through traces, inscriptions, and the deferred resolution of signs that were left for someone who has not arrived yet. One is synchronic—meaning now, in this interaction. The other is diachronic—meaning across time, in the accumulation and revision of marks. XR, uniquely among media, operates in both dimensions simultaneously. You are playing a language game and writing a palimpsest at the same time. Wittgenstein and Derrida are two sides of a thaumotrope. Use on one face, inscription on the other, and the lived experience of entering a new XR environment—the thing that actually happens to the body—produced by the spinning between them.
II. Arriving at ITP
I came to NYU's Interactive Telecommunications Program from a music career. I could read a lead sheet, follow a conductor, lock into a groove with a rhythm section, and improvise over changes. What I could not do was think about interaction as a designed thing.
The first weeks were disorienting in a way I did not expect. The disorientation was not intellectual—I could follow the theory. It was physical. I was learning a new form of life in the most literal Wittgensteinian sense: a new set of practices, tools, norms, and embodied habits that made sense only from inside the community that used them. The way people talked about "affordances" was a language game—the word meant something specific in context, and you learned what it meant by watching how people used it, not by reading the definition. The way people prototyped was a language game: the speed at which you were expected to move from idea to physical thing, the social norm that a working prototype trumped a beautiful slide deck, the implicit rule that showing was worth more than telling.
I recognized this. It was exactly how you learn to play in a band.
Learning the banjo was the same process. Nobody teaches you clawhammer by explaining the biomechanics of the hand. You sit with someone who plays, you watch, you try, you fail, you adjust. The language game of old-time music has rules—the rhythmic pulse, the drone of the fifth string, the particular way the picking hand strikes down across the strings rather than plucking up—but these rules are transmitted through practice, not instruction. You learn what a roll sounds like by hearing it in the context of a tune. You learn what a pull-off means by feeling when it works in the groove and when it disrupts. The form of life is the music session: a circle of players, a shared repertoire, the implicit negotiations of tempo, volume, and who takes the next break. You enter the circle not knowing the rules. You leave, years later, unable to articulate most of them—but your hands know.
The bluegrass session circle is Wittgenstein's language game made audible. I started sneaking into sessions at the Baggot Inn when I was fourteen—old-time in the front room, bluegrass in the back, and sometimes a splinter jam of professionals in the middle. My mother drove me the first time. After that I used fake IDs.
What I was learning to enter was a form of life with rules that no one would explain. Who kicks off the tune? The banjoist, usually—but not always, and knowing when to defer is itself a rule you learn by violating it. How do you signal the key? You announce it, or you play a turnaround—a 1-5-1 in the Nashville number system—and the convention of whether you announce or play depends on the room, the tempo, the formality of the gathering. How do you signal you want to take a solo break? A nod, a lean forward, a slight increase in volume during your rhythm part—none of which is written anywhere, all of which is legible to anyone who has been in enough circles.
The kickoff itself has a vocabulary. There are stock phrases with nicknames: "Son-of-a-bitch, I'm tired!" for the canonical banjo lick that announces the downbeat. If you play the wrong kickoff phrase for the style of tune—a bluegrass kickoff on an old-time tune, say—the circle registers the error instantly. Not through correction. Through a quality of silence, a micro-hesitation, a glance. You have made a move in the language game that the other players recognize as wrong, and their recognition is how you learn the rule. Wittgenstein: "To obey a rule, to make a report, to give an order, to play a game of chess, are customs (uses, institutions)." The session circle is a custom. You learn its rules by inhabiting it, the way you learn a language by speaking it badly until the community's responses shape your usage into fluency.
And not knowing a song is almost never a problem—because the songs are all variations of the same nuclear DNA of form and harmonic structure. The vocabulary is finite: I-IV-V progressions, the same structural skeletons, the same melodic phrases migrating between tunes the way Wittgenstein's family resemblances link related concepts without a single defining essence. You do not need to know this specific tune. You need to know the form of life that generates all the tunes. Once you are inside the form of life, the individual tunes are just moves in a game whose grammar you already speak.
Derrida would add: the session circle is also a palimpsest. Every time the circle plays "Salt Creek" or "Whiskey Before Breakfast," it is inscribing a new trace over every prior performance of the same tune—by these players, by other players, by the recordings that taught these players, by the field recordings that taught those recordings. The tune carries its own history as a trace. The fiddle player's particular ornament on the B part is a trace of whoever taught them, who learned it from whoever taught them, all the way back to the anonymous fiddler who first played the tune at a dance in the nineteenth century. The session circle is a living manuscript, each performance a new layer of inscription on a text that has been accumulating for generations. You are simultaneously playing a language game—making moves in the present, among co-participants—and writing on a palimpsest—adding your trace to a tradition that will carry it forward into contexts you cannot foresee.
This is the double inscription the essay describes: Wittgenstein's use and Derrida's trace, happening simultaneously, in a circle of bodies holding instruments, in the back room of a bar.
In a pit orchestra, nobody hands you a manual that says "when the conductor gives the upbeat, breathe together; when the bass player shifts his weight, the tempo is about to change; when the lead trumpet cracks a note, do not look up." You learn these things by being in the room, night after night, until the practices become transparent—until you stop thinking about the rules and just play.
Wittgenstein's insight is that all understanding works this way. There is no moment when you step outside the language game and grasp meaning from a neutral vantage point. Understanding is participation. You are always already inside.
But Derrida's insight is that participation leaves marks. Every night in the pit, we left traces—not just in the audience's memory but in our own bodies. A slightly different way of phrasing a passage that stuck. A mistake that changed how we approached a transition. The score was the same every night, but the performance was a palimpsest: each night's traces layered on every prior night's, the meaning shifting incrementally through accumulated inscription.
ITP worked the same way. The physical space—the floor, the shelves of components, the laser cutter queue, the project boards—was a text written by years of prior students. You learned the program partly by reading that text: the scuff marks that told you where heavy things had been dragged, the labels on bins that encoded a taxonomy of materials, the residue of old projects taped to walls. The space itself was a pedagogical inscription, and your education consisted partly of learning to read it and partly of adding to it.
III. Wittgenstein: Meaning as Use
In Philosophical Investigations, Wittgenstein dismantles the idea that meaning is a fixed property of signs. A word does not mean something because it corresponds to an object or an idea. It means something because of how it is used within a particular practice—a language game—embedded in a particular form of life.
Language games are not just verbal. They include pointing, building, measuring, commanding, questioning, greeting—any patterned activity where signs (words, gestures, actions) are exchanged according to implicit rules that a community maintains through practice. The rules are not written down and consulted. They are shown in how the community applies them, demonstrates them, corrects violations of them. You know the rule by seeing what counts as following it and what counts as breaking it.
Three features of language games matter especially for XR design.
First, family resemblances replace essences. Related practices share overlapping traits without a single defining core. A "grab" in VR might mean "pick up this object" in one context, "claim this territory" in another, and "initiate a handshake" in a third. These uses are related but not identical—they form a family, not a category. Designing for family resemblance means designing gestures that are flexible enough to take on contextual meaning rather than mapping one gesture to one function.
Second, rule-following is public and corrigible. A rule is not something you follow in private; it is something you follow in a way that others can recognize, evaluate, and correct. This undermines the idea of a private language—a system of signs whose meanings are accessible only to one mind. In XR terms: an interaction grammar that only one user understands is not a grammar at all. Legibility requires community. This is why social VR platforms develop richer interaction vocabularies than solo experiences: the presence of other people creates the conditions for rules to be shown, tested, and stabilized.
Third, learning is apprenticeship. You learn a language game not by studying it but by doing it—watching, imitating, receiving feedback, gradually aligning your actions with communal criteria. Fluency is procedural knowledge acquired through practice, not declarative knowledge acquired through instruction. The best XR tutorials are not tutorials at all. They are environments that let you fail safely and learn from the consequences.
IV. Derrida: Meaning as Trace
Jacques Derrida's Of Grammatology makes a complementary move. Where Wittgenstein locates meaning in use—in the living, present moment of practice—Derrida locates meaning in the trace: the mark that persists after the moment of inscription, carrying meaning forward into contexts the inscriber cannot foresee.
Différance—Derrida's neologism combining "to differ" and "to defer"—names the process by which meaning is never fully present in any single sign but is always distributed across a network of traces, each one differing from and deferring to others. A word means what it means partly because of all the words it is not, and its meaning is never final because every new context reshapes the network of differences.
The trace is the substrate: partial, repeatable, never fully present, always conditioned by prior and future inscriptions. Writing, for Derrida, is not the secondary transcription of speech. It is the primary condition of meaning: the iterability of marks—the fact that a sign can be repeated in new contexts—is what makes signification possible at all.
Gregory Ulmer extends this into electracy—the digital cultural apparatus where images, interfaces, and networks become the locus of thinking and writing. If literacy is the cultural apparatus of the alphabet (linear, sequential, analytical), and orality is the cultural apparatus of speech (embodied, rhythmic, communal), then electracy is the cultural apparatus of digital media (networked, multimodal, navigable). Electracy does not replace literacy; it reorganizes what counts as a text, what counts as reading, what counts as writing.
XR intensifies electracy. Bodies become pages. Motion becomes orthography. Persistence and replay become syntax. In this medium, inscription precedes and shapes expression—what you do in space leaves a legible trail that others can read.
V. Before the Alphabet: Proto-Writing as Spatial Grammar
Before alphabetic text, societies wrote with space.
Ovid understood this before Derrida named it. As I discuss in Essay 3, Catharine Edwards shows that Augustan Rome was a spatial text—every portico, every temple, every triumphal route inscribed with political meaning that citizens read with their bodies as they moved through the city. And Ovid's Ars Amatoria was a counter-inscription: the same spaces rewritten as erotic geography, the porticoes of military glory becoming rendezvous points, the monumental architecture of empire turned inside out by a poet who read the city against its intended meaning. This is both Wittgenstein and Derrida at once. Ovid was participating in the language game of Roman public space—he knew the rules, he was fluent in the grammar of civic movement—and simultaneously inscribing a new trace over the existing palimpsest, a trace that changed what subsequent readers could see when they walked the same routes. He was exiled for it. The inscription was that powerful.
A carved footprint at a trail junction encoded direction. A sequence of stones marking water encoded distance and priority. A patterned weaving encoded kinship, season, and ritual obligation through form and spatial arrangement. These are not pre-literate approximations of "real" writing. They are spatial grammars—systems of meaning that work through proximity, orientation, repetition, and the physical relationship of marks to the world they annotate.
Meaning in these systems emerged from how signs are arranged in space. A single stone means nothing. A line of stones means "path." A cairn means "here." A ring of stones means "gather." The grammar is positional: what matters is not the sign in isolation but its relationship to other signs and to the landscape it inhabits.
XR inherits this pre-textual logic directly. A corridor lined with floating icons guides behavior through spatial arrangement, not verbal instruction. A cluster of annotations around an object elevates its salience—density signals importance. The sequence in which a participant encounters markers composes a reading order. Layout, density, proximity, and sequence function as grammar even when no words appear.
This is not an analogy. It is a direct structural continuity. XR interaction design is, whether it knows it or not, a form of spatial writing—the design of environments in which arrangement, density, and sequence carry meaning.
VI. XR as Double Inscription
Here is where Wittgenstein and Derrida converge on a single claim about XR.
Every interaction in XR is simultaneously an act of participation (a move in a language game, meaningful in the present, legible to co-participants) and an act of inscription (a trace left in the environment, meaningful in the future, legible to subsequent visitors or to the same participant returning later).
Consider a collaborative design session in Gravity Sketch. When a designer points a laser at an object and scales it, that gesture is a move in a language game—it stages turn-taking, proposes scope, negotiates focus. The other designers read it immediately, in context, the way you read a word in a sentence. But the scaled object also persists. When the team returns tomorrow, the object is still scaled. It has become a trace—an inscription that carries yesterday's meaning into today's context, where it will be read differently because the context has shifted.
The VR whiteboard makes this layering visible. Strokes appear as volumetric lines. Erasures carve negative space. Layers stack ideas by time and theme. A team returns to the board and scrubs through time, exposing drafts beneath the current surface—reading the negotiation, seeing where contention thickened lines, where a decisive erasure reset direction. The board is a text. Its history is legible. But its meaning is never final, because every new session adds another layer of inscription that recontextualizes everything beneath it. This is différance made operational: meaning deferred across sessions, distributed across layers, never fully present in any single state of the board.
Gesture recording in dance and movement training makes the same principle kinesthetic. Systems like Motion Bank capture skeletal gestures; ghost replays float a translucent dancer in the room, aligning with the learner's body. The record is a kinesthetic script: velocity, arc, and timing inscribed in air. Meaning arises from micro-variations—the pause before a turn, the soft elbow. The learner reads this script by stepping into it, shadowing it, correcting against it. The ghost is a trace. The learner's response is a new inscription.
Spatial annotation tools like Vuforia Chalk treat AR markup as durable writing. A technician draws arrows and circles over a pump; the software anchors these strokes to world geometry so they persist across calls. A second technician revisits the marks, adds highlights, resolves earlier ambiguities. The annotations are language-game moves (pointing, emphasizing, instructing) that have become traces (persistent, revisable, addressable). The meaning of a mark changes as new marks accumulate around it. An arrow that originally meant "look here" acquires new significance when a circle is drawn around what it points to and a note is added that says "replaced 3/15."
VII. The Notation Problem
Abraham Burickson, in his chapter on diagrams, identifies a problem that connects directly to this double inscription: the tools we use to compose experiences determine what experiences we can compose.
Traditional notational systems—scripts, floor plans, recipes—are object-oriented. They describe what things are and where they go. A script tells you what words to say and when. A floor plan tells you where the walls are. A recipe tells you what ingredients to combine in what order. These systems are powerful for their purposes, but they cannot capture the experiential dimension of a designed encounter. They describe the artifact, not the experience.
This is a language-game problem. The notation is itself a language game, and its grammar constrains what can be expressed within it. If your notation is a floor plan, you can express spatial relationships but not temporal dynamics. If your notation is a script, you can express dialogue and stage directions but not the felt quality of a transition or the way ambient sound shifts the participant's attention. The limits of the notation mean the limits of the design—or, as Wittgenstein would say, the limits of my language mean the limits of my world.
Burickson's solution is the diagram: a compositional tool that prioritizes relationships, transitions, and temporal dynamics over static placement. He identifies three categories—symbols (basic units of graphic communication), graphs (grids and charts showing relationships between variables), and maps (descriptions of territory organized around specific priorities). A diagram of an experience might show how attention moves across a space, how emotional intensity rises and falls over time, how different participants' paths intersect and diverge.
What Burickson is proposing, in Derridean terms, is a new writing system—a new way of inscribing experiential intention that is adequate to the medium of immersive experience. The floor plan is a literacy-era tool: linear, static, object-oriented. The experiential diagram is an electracy-era tool: multimodal, temporal, relation-oriented. The shift from one to the other is not just a technical improvement. It is a change in the grammar of design—which means, per Wittgenstein, it is a change in the form of life that design practice constitutes.
And in Derrida's terms, it is a change in what can be traced. If every interaction in XR is an inscription, then the notation system determines what inscriptions are legible. A floor plan makes spatial position legible. A script makes dialogue legible. A diagram that tracks attention, rhythm, and emotional dynamics makes experience itself legible—traceable, revisable, available for re-reading.
VIII. Reading Spatial Traces
What does it feel like to read an XR environment as a text?
Users learn—through practice, through the language game of repeated visits—to read density as emphasis, recency as freshness, and layering as nuance. Thickness denotes commitment: a bold annotation was placed with confidence. Faint lines whisper tentative ideas. Overlaps signal debate; clean alignments imply consensus. The felt experience is browsing a living manuscript: moving around reveals hidden subtexts; stepping through a ghost path enacts a sentence; aligning one's body with a recorded gesture creates a kind of empathy with its author.
Path-tracing in architectural walkthroughs reveals this reading process at scale. Heatmaps of movement condense collective behavior into gradients. Recency glows brighter; dead zones fade. Designers read density and flow, re-routing corridors, resizing nodes. The trace is a social script written by bodies, readable as circulation grammar. But it is also a palimpsest: each cohort's traces layer on the last, and the legibility of the current state depends on the accumulated history beneath it.
This is the phenomenology of spatial reading—the felt sense of entering a space that has been written on by others and learning to parse the inscription. It is not so different from entering a well-used workshop for the first time: the tools are organized according to someone else's logic, the workbench is scarred by prior projects, the drawers are labeled in someone else's shorthand. You learn the space by using it, and you learn its history by reading the traces of prior use.
The difference in XR is that these traces can be made explicitly legible—filterable by author, time, layer, and tool. You can scrub through the history of a space the way you scrub through a recording. You can see who inscribed what, when, and in what order. This makes the palimpsest navigable in a way that physical spaces are not. But it also raises the stakes: if every interaction is an inscription, then every interaction is also a form of surveillance unless it is designed otherwise.
IX. Ethics of Inscription
If every gesture in XR is both a language-game move and a durable trace, then the ethics of XR interaction design must address both dimensions.
On the language-game side: who gets to set the rules? Language games carry power. Normative usage privileges insiders and can gatekeep newcomers. The gestures that "mean something" in a VR community were established by its earliest, most active members—and those members are not a representative sample of humanity. Designers must ask whose bodies are presumed by the interaction grammar, whose cultural norms for proximity and gesture and touch are encoded as defaults, and what happens to participants whose embodied practices do not match the community's established conventions.
On the inscription side: who controls the trace? Persistence must be balanced with erasure rights. If every path I walk through a VR environment is recorded and visible to subsequent visitors, then my movement through space is a form of involuntary authorship. Consent governs not just what I do in the space but what the space remembers about what I did. Users should control visibility and retention of their traces. Authorship needs clear attribution without chilling collaboration. And the temptation to use trace data for surveillance—tracking who went where, for how long, with what hesitation—must be resisted by design, not just by policy.
The deepest ethical question is about legibility itself. Derrida's différance suggests that the meaning of a trace is never fixed—it shifts as new traces accumulate around it. A path I walked six months ago meant one thing in its original context; it may mean something entirely different in the context of everything that has been inscribed since. If my traces are persistent and legible, then I am being read in contexts I cannot foresee. This is the condition of all writing. But in XR, where the "writing" is my body's movement through space, the stakes are more intimate than they are on a page.
X. Measurement
How do we evaluate whether an XR interaction grammar is working?
From the Wittgenstein side: time to basic fluency measures how quickly newcomers can participate in the language game. Interpretive alignment measures whether participants in a shared session understand each other's gestures and actions. Innovation rate tracks whether the community is generating new conventions—new moves in the language game—that are adopted by others. Cross-cultural comprehension tests whether the grammar is legible to participants from different embodied traditions. And graceful degradation measures how quickly repair restores coordination when communication breaks down.
From the Derrida side: trace density measures inscription rate—marks per volume per time. Revision patterns track the life of traces—how often they are modified, branched, or erased. Collaboration metrics track co-inscription—how traces from different authors interact, layer, and respond to each other. And meaning emergence over time tracks whether the accumulation of traces produces increasing legibility or increasing noise.
The most revealing metric might be the ratio between synchronic and diachronic engagement: how much time do participants spend interacting with each other in the present (playing the language game) versus reading and responding to traces left by prior sessions (reading the palimpsest)? A healthy XR environment would show both. A space that is only synchronic—all live interaction, no persistent inscription—loses its history. A space that is only diachronic—all archive, no live practice—loses its life.
XI. The Grammar We Are Writing
Here is the claim, stated plainly:
Every XR environment is a language game and a writing system at the same time. The participants are learning rules through embodied practice (Wittgenstein) and inscribing traces that persist beyond the moment of their making (Derrida). The interaction grammar—the set of gestures, movements, placements, and gazes that carry meaning—is simultaneously practiced (performed in the present, among co-participants) and written (recorded, layered, available for future reading).
The designer's job is to compose both dimensions. The language game must be learnable, flexible, inclusive, and generative—capable of supporting practices the designer did not foresee. The writing system must be legible, revisable, attributable, and erasable—capable of accumulating meaning across sessions without becoming surveillance.
Burickson's diagrams offer a notation adequate to this double task—a way of inscribing design intention that captures both the living practice and the durable trace. Alison's observation that text comes from texere—to weave—is apt. The grammar of XR interaction is a textile: warp threads of present practice interwoven with weft threads of accumulated inscription, producing a fabric of meaning that is stronger than either alone.
The limits of this grammar mean the limits of what can be experienced. And the body that learns this grammar—the body that acquires fluency through practice, that inscribes traces through movement, that reads the palimpsest of prior inscriptions with increasing skill—that body is being shaped by the grammar it inhabits. As with any language, the language game you play determines what you can think, what you can feel, what you can become.
The Passover Seder is this double inscription performed as ritual. The Haggadah is a language game—you learn it by doing it, year after year, in a community of practice that teaches you when to ask the four questions, when to hide the afikomen, when to open the door for Elijah. But it is also a palimpsest—the text that has accumulated commentary for three thousand years, each generation inscribing its traces over the traces of the previous generation, the Marxist Haggadah and the feminist Haggadah and the Reconstructionist Haggadah layered over the medieval text layered over the Mishnaic original. Every Seder is Wittgenstein and Derrida simultaneously: participation in a living form of life and reading of a durable inscription, the present practice and the accumulated trace held together in the same evening, at the same table.
This is the thaumotrope of this essay: use on one side of the disc, trace on the other, and the grammar—the thing that actually structures how bodies move through designed space—produced by the spinning between them. Wittgenstein without Derrida gives you a language game with no memory. Derrida without Wittgenstein gives you a text with no life. XR needs both: the living practice and the durable inscription, the form of life and the palimpsest, the body that learns by doing and the space that remembers what was done.
The body writes. The body reads. The body is written on.