Convusion Alien Mama
VR Developer & Design Advisor
A project dubbed TLOF_7 which aims to provoke and study fear caused by Artificial Intelligence (AI) generated images in Virtual Reality (VR).
2023
Collaboration
Tetem is a dutch organization specializing in digital arts and culture. During their Development Weeks they invite artists and designers to conduct research and talk with experts. They invited me as a VR development expert to meet Farah Shretah of Alien Mama, which led to an interesting collaboration. Together, we developed a VR experience aimed at studying fear induced by AI-generated images. Users create images with prompts via a phone application and the VR player's reaction is analyzed using an EEG scanner to assess the provoked fear.
Prototyping
My primary role in our collaboration was translating the artist's idea into a feasible prototype. Initially, I conducted research on how AI image generators could integrate with game engines like Unity, and how the generated images could be projected within the game engine. I concluded that the best approach was to utilize OpenAI's AI called DALL-E with Unity and projecting the generated images on curved planes to enhance immersion. The immersion could be further amplified by sound and particle effects.
Above, you can see one of the first functional prototypes.
Technical Challenge
During the prototype's development, I established a system allowing a phone to connect to a PC and send messages to the Unity engine, using the extOSC plugin in Unity. This complex integration challenge pushed my C# programming skills into new territory, prompting me to collaborate with a skilled developer, Tycho Tuitert, who eventually joined our team to tackle the more advanced programming requirements.
Below, you can see part of the Shader Graph used to create the glitch effect.
Teamwork
In teamwork with Tycho, I focused on creating particle effects using Unity's particle system and a glitch effect for the images using Unity's Shader Graph. I also maintained communication with the artist, translating her ideas into designs and communicating these with Tycho. Tycho concentrated on integrating the AI and establishing PC-phone connection. Farah created together with a sound designer the music.
The final prototype, visible at the top of this page, was showcased at FilmForward Vrijplaats 2023: Creating The Experience, an event where both the client and I were invited as experts.