Combs and garment:
R: Reception
G: Garment Rite de Passage
A: Entrance with mirror
C1: «Zen Garden»: seamless/seamful surfaces
C2: «SciFi»: 1980s’/90s›, quantifiable future
C3: «Office Anachronism»: roots of digitalization
C4: «Echo Chamber»: stage for deprivation
C5: «Mycelium»: eco self-dissolution network
The immersive research setting Journey requires a consistent narration that enables test persons to fully engage with the environment, becoming the only relevant projection surface of experience. The scenes are spatiotemporally staged to trigger associations and emotions and the subjects’ behavior patterns captured by a UWB Positioning System and incorporated sensors. The data is analyzed by rule-based and artificial intelligence algorithms and the findings combined with the qualitative surveys: see “methods”. The scenes also refer to one another and create meaningful links and contradictions. The process of making sense of these situations and narratives is another key investigation focus.
The Garment: The interference of the environment with the test person’s body is to be intensified by a specially designed clothing. Before the journey through the combs, the test person puts on this garment that resembles an armor with features that are supposedly useful for the trip. By doing so, they leave their familiar comfort zone and take on a role in the narrative. This initiation helps to leave behind everyday life and to create expectation and attention. To improve the “rite de passage» the test person is confronted with a mirror in the entrance room.
To intensify the bodily experience and to dissolve the borders between body and environment, the garment includes LED lights on the sleeve, glass fibers on the chest and vibrators along the spine, that communicate with the surroundings in particular situations. It also includes basic tracking features to enable interaction and to record user behavior: ultra-wideband tracking tag, a compass and an accelerometer. To measure psychophysiological biofeedback signals, a GSR fingering is attached to the sleeve.
The Combs: The research setting Journey is composed of five honeycomb-shaped spaces, we refer to as ubiCombs. Each ubiComb possesses an intrinsic quality and a certain network logic that relates to the other combs: by means of recurring elements and cross-ubiComb acoustic stimuli, the interconnectedness between the changeable spaces is emphasized. Depending on the subject’s interaction, the installation evokes different behaviors, idiosyncratic references, associations and interpretations.
ubiComb1: «Zen Garden»
The first sequence starts with a calculated irritation: Far Eastern architectural and garden elements display a stereotypical Zen Buddhism scene. The harmonious design thematizes the seamless surface, the black box, which allows no glimpse of the underlying technology and its potential for surveillance and control. The comb is setup as a courtyard with white gravel, Shoji elements, bamboo-colored fly curtains, Buddhist chimes and wind noises. A retro projection mirrors the test person (captured by a camera) as an abstract aesthetic figure and invites for playful interaction. Opposite the video projection a wooden panel displays the «All-seeing Eye», that hides the video camera (the «All-seeing Eye of Big Brother»). An illuminated bowl with apples displayed on a white pedestal seduces the test person to take an apple, which leads to the symbolic expulsion from paradise (an incorporated scale measures the number of apples). The harmonic scene abruptly changes (light and video projection) and shows the technical control mechanisms behind the seamless scene. After about five minutes, the scene fades out and sounds and light animations lure the test person to ubiComb2.
The connection between spirituality and emphatic technological culture is particularly striking with regard to the origins of Silicon Valley: at the center of California’s Silicon Valley culture has always been a certain penchant for spirituality and a highly idiosyncratic interpretation of Buddhist teachings. Thus, the history of the Palo Alto Research Center (PARC), where Mark Weiser used to develop his concept of ubiquitous computing (ubiComp), is also deeply imbued with once countercultural attitudes that always knew how to promote a change of perspective in the face of the status quo, if they did not even make such a change possible in this specific form in the first place.
ubiComb2: «SciFi»
After the spiritual introduction, a cliché-like vision of the future awaits the test person with its cybernetic thinking of the 90s (human as a measured machine). The subject enters an imaginary access control point of a spaceship and is invited to step on different color-coded LED floor panels to get scanned. The yellow panel invites for DNA Analysis, the red one for Brain Analysis (mental fitness), the green one for Virus Scan (Pandemia) and the blue one for SpyCam Analysis (video recorded in ubiComb1, terror allegation). A retro projection displays the different scanning processes. 4 doors display hand scanning devices and suggest access to other imaginary combs. Thus, the test person is invited to solve the riddle (spatial imagination, mental mapping, sense-making) and to deliver the right scanning results to proceed to the next comb. Only one of the doors actually provides a way to the next comb, the other 3 are fake and display “Access denied!” The room also triggers activities of the garment worn by the test persons: LEDs on the sleeve get activated while hand scanning and vibrators positioned on the spine are activated during the body scan. The scene ends when access is allowed: the video and hand scan indicate access and the sliding door opens automatically, giving access to C3. The teaser for C3 is the old phone ringing and typewriter sounds.
Omnipresent data displays exemplify a constant flow of computational processes – the intention is to refer to the networking quality of technocratic regimes, as this is continuously thematized in contemporary science fiction novels, but also how such borrowings are frequently discussed in the context of popular reporting: China’s social credit system, London’s CCTV rhizome, or even Google’s attempts to establish ubiquitous algorithmic interpretive sovereignty are stereotypically reflected here, mapping an explicit break with spiritual entry. What is projected here is a present draft of the future, which must remain as a karrikaturesque distorted image vis-à-vis a possible future present. At the same time, however, it is also a sharpened image of the development of the Silicon Valley mindset, which is continuously «emancipating» itself from its Zen-like roots.
ubiComb3: «Office Anachronism»
In this comb, the test person is invited to time-travel by visiting a private office from a time before the PC was distributed to every household. The room is decorated with a cliché 70s wallpaper and furnished with antiquated furniture and corresponding technological artifacts. The subject becomes the silent observer of an imaginary inhabitant’s space, who is presumably a writer and utopic thinker. The artifacts displayed, are still in everyday use but in digital form or used as metaphors such as desktop, file and wastebasket. The scene is full of information to discover: telephone and radio (spoken information), typewriter (piece of paper) flashcard system (categories system), wastebasket (rejected ideas), pictures (utopias). After a while, the sounds of neighbors of imaginary adjacent rooms are displayed (private scenes and historic events represent interconnectedness) mentally overlaying the imaginary combs visited beforehand. After a defined time, the activities and lights dim out, an “On Air” light sign and sounds guide the test person to open the door to access ubiComb4.
For although digital representation is not explicitly used here, “proto-digital markers” do exist: for example a box of notes containing neatly sorted address cards, a telephone that, when you pick up the receiver, a voice sounds that tells a Kafkaesque story about complete disconnection in a continuous loop. Also, a wafting projection of a «classic» 70s wallpaper that both refers to the countercultural use of psychedelic drugs and makes clear that this room is also simulated in its entirety – animated in the emphatic sense of the word.
ubiComb 4: «Echo Chamber»
This room is reminiscent of a stage situation that unmistakably invites the test person to perform: a round stage element, red curtains, a stage microphone and animated spotlights. The imagined audience is audible from behind the curtains: murmuring and laughing. The speakers already play a background beat composed of sound samples of the formerly visited combs: gravel and Buddhist bells (Zen Garden), scanning sounds (SciFi), 70s telephone and typewriter sounds (Office Anachronism). When stepping on stage, the curtains are automatically pulled back and the emerging hall of mirrors multiplies the test person on stage endlessly. The performer is thus not presented to an audience, but is thrown back on him/herself. Audience and performer are present in the face of the mirrors in personal union. If the person dares to sing or speak into the microphone, the sounds get woven into the soundscape. After 5 minutes the lights and sounds fade out and the emerging sounds coming from ubicomb5 lure the test person to proceed.
This situation is supposed to express, ex negativo, the social media mindset under the sign of ubiquitous social performance – a kind of social media backstage. Here, the visitor should be able to experience that he/she is always performing him/herself, but does not have to expect a direct (physical) reaction from a foreign audience, but finds him/herself in a real mirror situation (Echo Chamber) by means of fleeting likes and comments. The (self-)realizations of this space are a stepping stone to the final space, the fusion.
ubiComb 5: «Mycelium»
When entering Mycelium the test person goes on an excursion in an alien natural environment with organically networked life forms. After the show in Echo Chamber, the senses and attention have to adapt to a calm, dark and mystical environment. The borders of the space disappear into the dark and animated creatures of different types (robotic mechanisms) are dispersed over the walls and floor. The environment resembles an underwater world but also the inwards of an organism. Bioluminescent colors (blacklight) are pulsating to breathe life into the scene and to visualize data flow. The creatures seem to mind their own business but also respond to the test person. The space is also animated by ventilators (tangential fans) moving hanging structures and textiles. The sound reminds of organic processes (flow, bubbles, dripping, …). The applied materials and lights are similar to the ones built into the garment, worn by the test person. Like with a camouflage dress, the test person merges with the environment and becomes part of it. By touching the pendant tentacles, intense motion or sitting down on the central seat, the illumination, sounds and motion patterns of the creatures change their characteristic. When staying completely inactive the system also changes to a harmonic loop. After 5 minutes, the activity is turned down and the exit illuminated. In contrast to the cybertechnical appearance of comb 2, comb 5 stages smart and organic technologies that allow intuitive interfacing and spiritual connectedness. But they also implicate self-dissolution and loss of personal identity – the human is becoming an insignificant piece of the whole and ultimately also controllable.
The finale is marked by a consistently dormant engagement of the individual in the ubiComp context. If one still felt somewhat alien in ubiComb2 – because one was reduced to the role of a monitored/measured object – here an organic solidarity is being forced. The reaction of the space (network-like, luminous structures) generates the feeling of being in the center, but at the same time being an insignificant piece of the whole and ultimately also controllable. The aesthetics possess biological-organic borrowings (wood wide web), which runs counter to the karrikaturesk-smooth, technocratic vision of the future, in a sense allowing for a merging dissolution, much more in tune with the first space and its spiritual connectedness. The performance element from ubiComb4 can be found again, as well as references to the (supposedly) sterile «world» from ubiComb3. In ubiComb5 everything is connected – organically speculative.
The entire installation allows interaction with the technical system on different levels: on the one hand, each ubiCombhas its own story and mechanisms; on the other hand, there is also an overarching narrative and technological implementations like a tracking system and a time-based system that control space and time for all the combs. Both aspects merge in a fluid experience that is driven by explorative curiosity through the explicit breaks.
Since test persons will react in individual ways, we identify and contextually categorize behavioral patterns with the help of a simple AI system. In order to evoke an ambivalent relationship between technological system and human on a responsive level, we create a techno-social hybrid with shifting proportions of agency. The garment also functions in this respect, coming as close as possible to the body and yet remaining an extrinsic part of it. It marks the individual as a third party who will always experience processes of border crossing. User or used?
With regard to the entanglement of human actors and technological sensor-actuator systems, the concept of affordance becomes the central pivot: the environment produces perceptual offers and invites the subjects to re-act. The interpretative handling of the underlying cultural and functional anchors is an important design element.
The entire artistic setting is a «method space» for scientific investigations, which are not concerned with the development of concrete applications, but with a critical reflection on the manifold connections between technical, human and design epistemology. Strategies and forms of expression of media art and Speculative & Critical Design – such as staging, intensification and abstraction – make it possible to make key situations tangible and thus negotiable. In the form of the interactive spaces described above, narrative structures and cultural-historical references create disconcerting, confrontational or ambiguous situations that know how to mediate between humans and technology. The situations trigger appropriation processes by evoking emotional reactions, stimulating cognitive interpretation processes, and inviting action.
The exemplary ubiComp paradigms of invisibility, immediacy, networking and ubiquity serve as content-creative foundations and evaluation criteria. The research installation is evaluated with test subjects in terms of appropriation criteria of experience (perception of events), sensation (emotional perception), behavior (action and reaction), association (cognitive attribution), and interpretation (reflection). The artistic implementation and evaluation will also serve to develop alternative design options, elaborate new design principles, situate possible implications, and sketch future scenarios. In all of this, the following dimensions will map a consecutive guideline:
Ultimately, it is a matter of opening up the realm of agency, which is subject to massive change and is visibly shifting into a hybrid-relational space.
This distinction between user or used, is largely irrelevant at a time when Big Data is becoming virulent – for each of us is ultimately an individual data point in the framework of Big Data, thus equally a beneficiary of the predictive calculations, as well as their homogenized data material. Accordingly, hypothetically speaking, agency as a relationally constituted state of affairs is rooted in the hybrid dimension of the techno-social, cannot be clearly located on one side of the continuum, but is at home in the field of tension of the in-between.
Affordances are structured along the respective (inter-) subjectively composed sensory apparatus: What is «expected» of human users in ubiComp contexts are optical stimuli in the perceptible color spectrum with the corresponding frame rate and also acoustic events between 20 and 20,000 Hz. The fact that signals are also emitted outside these physical limits primarily concerns the communication of the technological components with each other or with themselves. It is even explicitly desirable that the separation of the communication paths is carried out in such a strict manner, because otherwise interference could occur which would undermine the practicable functionality of a ubiComp system: If all communication participants used the same communication channels, a kind of positive feedback, i.e., cumulative information overload, would quickly occur.