Song DONG

INQ–α: Sugar, Clockwork, and Breath

A Computational Translation of Music Technical guidance: Tim Rodenbröker
Personal Project
Tools & Technologies: Processing, MIDI data parsing, JSON data structures, algorithmic animation, mathematical mapping functions, generative geometry

Introduction
This project originates from the music of my final ballet performance, set to the overture of The Nutcracker. After stepping away from ballet, I returned to this score not to reenact the performance, but to re-encode it computationally. By translating the music into data and algorithmic rules, the project examines how a bodily, time-based experience can be transformed into a generative system—one that preserves rhythm and structure without relying on figurative representation.
( High-resolution images may take a moment to load.)

 Sugar, Clockwork, and Breath — image 1Sugar, Clockwork, and Breath — image 2Sugar, Clockwork, and Breath — image 3
Sugar, Clockwork, and Breath — image 4Sugar, Clockwork, and Breath — image 5Sugar, Clockwork, and Breath — image 6
Sugar, Clockwork, and Breath — image 7Sugar, Clockwork, and Breath — image 8
Sugar, Clockwork, and Breath — image 9Sugar, Clockwork, and Breath — image 10Sugar, Clockwork, and Breath — image 11
Sugar, Clockwork, and Breath — image 12Sugar, Clockwork, and Breath — image 13Sugar, Clockwork, and Breath — image 14
Concept
The project considers music as a system of discrete events unfolding over time, and performance as a synchronization between body and temporal structure. Here, the choreography is absent, but its logic remains embedded in the score.

    Three metaphors organize the system:
  • Sugar — accumulation and crystallization, corresponding to moments of musical density
  • Clockwork — cyclical precision and repetition, reflecting rhythmic structure
  • Breath — expansion, contraction, and pause, recalling the physical act of dancing

Rather than illustrating emotion or narrative, the project encodes these qualities through computational relationships. Musical attributes become parameters; time becomes a controlling variable; form emerges as a consequence of data-driven rules.

Sugar, Clockwork, and Breath — image 15Sugar, Clockwork, and Breath — image 16Sugar, Clockwork, and Breath — image 17Sugar, Clockwork, and Breath — image 18Sugar, Clockwork, and Breath — image 19Sugar, Clockwork, and Breath — image 20Sugar, Clockwork, and Breath — image 21Sugar, Clockwork, and Breath — image 22(The musical score comes from the Russian National Museum of Music; the movements of the suite were handwritten by a copyist)
Data Origin & Translation
The source data comes directly from the music used in my final ballet performance. The translation process unfolds in several stages:

  • Musical Score → MIDI
         The ballet music is converted into a MIDI file. Beyond pitch, MIDI captures timing, velocity, and duration—elements essential to both musical phrasing and       
          bodily movement.
  • MIDI → JSON
         To make the data accessible in Processing, the MIDI file is parsed and converted into JSON format. Each note becomes an object containing its pitch, 
          velocity, duration, and onset time.
  • Pitch Normalization
        Original pitch values ranging from 0 to 47 are compressed into a 0–11 range by grouping identical pitch classes across different octaves. This preserves 
        harmonic relationships while simplifying the dataset for generative operations.

The resulting dataset functions as a structured temporal array, where each value corresponds to a specific musical event.
Sugar, Clockwork, and Breath — image 23Sugar, Clockwork, and Breath — image 24Sugar, Clockwork, and Breath — image 25Sugar, Clockwork, and Breath — image 26Sugar, Clockwork, and Breath — image 27Sugar, Clockwork, and Breath — image 28Sugar, Clockwork, and Breath — image 29(A memento from the last rehearsal backstage)