the Fold: Episode II

"the Fold: episode II" still from VR experience for Oculus Rift & HTC Vive

"the Fold: episode II" is a virtual reality-based art game involving synthetic agents driven by machine learning. Each level of the experience involves agents of differing complexity. These agents are non-playable characters (NPCs) and are comprised of historical elements found from either Eurocentric or Sinocentric history and philosophy. The game itself is essentially an 'escape the room' experience, some rooms involve puzzles while others are more spatially challenging.

Inspired by Yuk Hui's book, The Question Concerning Technology in China: An Essay in Cosmotechnics, this VR-based 'escape the room' project highlights the similarities and differences of technics as it relates to Western (episode I) and Eastern (episode II) philosophy including structuralist & surrealist literature, sentient bodies, metaphysics, mathematics, the virtual object, Buddhism, Zen principles, Qi (氣), and problematizes the affirmation of technics, its outputs and technologies as anthropologically universal within the frame of VR, AI, 3D animation, and video games.

Elements from 'episode I' creep into 'episode II' as an allegory of the colonialist impulses East Asia faced throughout much of the modern era. Spatial thinking differs structurally between these two spheres of knowledge. Because of this, technical thinking (technics) evolved differently throughout much of the period before modernism. 'episode II' begins to fold elements of these two spheres together as a symbiotic virtual world of knowledge. 'the Fold: episode II' provides a glimpse into the performative potential of AI as a machine collaborator to create new forms of artistic expression.

To date, the project involves an array of these rooms:

pinnacle encounters

the Great Buddha is the starting scene for episode II where there are five gates. Inspired by The Great Buddha of Kamakura, each gate corresponds to an element of Zen referred to in the five elements:

Earth 地 (지,ち): collectiveness, stability, physicality, and gravity

Water 水 (수, すい): the fluid, flowing and the formless things in the world

Fire 火 (화,か): the energetic, forceful, moving things in the world

Wind 風 (풍,ふう): things that grow, expand, and enjoy freedom of movement

Void 空 (공,くう): those things beyond and within our everyday comprehension

  the Rock Garden (T-Shape) involves a rock garden populated by artificial intelligent agents competing to get through the door to the next level. Each agent has been trained using machine learning in order to find the door. After 1.5 million training steps, they have learned to climb on top of each other to achieve this goal. If you make a mistake by falling off the platform, you start over in a random new position. At each iteration the wall is high, low or nonexistent. Sometimes the AI gets confused and they make mistakes just like any sentient being. The rock garden provides a context to induce meditation on simulation, sentience, artifice and immateriality.

the Pocket Mazes involves various mazes of varying complexity inspired by pocket puzzles. The VR player has to solve all of them in order to advance to the next level.

the Infinite Rock Garden involves infinitely spawning AI agents named 'Marcgritte', whose pronouns are they/them (a concatenation of Marcel Duchamp and Rene Margritte), have been tasked to catch the VR player as they explore an artificial rock garden. The VR player can navigate outside the bounds of the rock garden, but spawns back on the platform if they fall underneath the garden's lower environment. These quadruped artificial agents are comprised of elements from a previous level. The Marcgrittes can adaptively respond to obstacles and changes in its environment using a non-deterministic algorithm which is based upon a neural model formulated by reinforcement learning.

the Liminal Space (Agent άλφα): Not a Pipe at the Threshold of Consciousness, is a room where a synthetic humanoid, Agent άλφα, has been imbued with some level of sentience by the training of a neural model through machine learning. This digital "Qi (氣)" drives the performance of Agent άλφα whose task is to caress the synthetic representation of a pipe with its digital body. The more times άλφα touches the higher their reward. An exit is provided to the VR player towards the end of a single stanza of the Buddhist mantra in order to get to the next room. The video above switches between training, real-time, and virtual reality based footage.

character study for "Agent άλφα":
a fragmented consciousness in a fragmented body,
a deprecated postmodern condition

the Liminal Space (Agent Qi/氣): Innocence, Agony, Ecstacy, is a room where Agent Qi attempts to caress the synthetic representation of infinity with its digital body. Through machine learning, Agent Qi has been imbued with some level of sentience by the training of a neural model. This digital "Qi/氣" drives the performance of Agent Qi where the more times Qi caresses the higher their reward. The video above switches between training, real-time, and virtual reality based footage. An exit is provided to the VR player towards the end of a single stanza of the Buddhist mantra in order to get to the next room.

character study for "Agent Qi/氣":
entity for the unification of the Heaven and the human
in the context of technics

Temple of Balancers involves AI agents who have learned to balance a golden orb using their hands. Each pair of hands represent a mudra for emotional wellness including: concentration, energy, stability, acceptance (in order from front to back).

the Shrine of Ascension, involves infinitely spawning Agent 氣, Agent άλφα, and Agent Marcgrittes trained to climb a flight of stairs to pass the door to advance to the next level. A planetary sphere containing an ancient star chart aides their journey as they navigate to a higher plane.

Pinnacle Encounter (Neolibus), a manifestation of a threatening entity based upon the all seeing eye with many hands that attempts to throw the player out of an arena similar to that of a dohyo sumo ring.