[youtube]https://www.youtube.com/watch?v=9RS92_vnYpw
It’s not often that ancient Chinese texts about divining the future and social media meet. It’s even less often that they meet in the form of a giant light show that takes place on the facade of a former Olympic venue. But this is what takes place in artist Jennifer Wen Ma and lighting designer Zheng Jianwei’s Nature and Man in Rhapsody of Light at the Water Cube.
The installation was unveiled last month at the Beijing National Aquatics Center (known as the Water Cube) and uses a computer program to translate the I Ching and the collective mood of the Chinese people, through the emoticons they use on microblogging site Weibo, into a real-time light display on the building’s blistered exterior.
Ink painting for the fire theme. Courtesy of Jennifer Wen Ma
The finished lighting design for the fire theme. Photo courtesy of Wang Xin
Using custom software, millions of emoji are sifted through, and in combination with the I Ching, they inform the color, tone, and movement of the light. In the video above Ma talks about how the piece juxtaposes the old and the new and is also a representation of nature and man, and a symbol of society.
While Jianwei notes how, through the unpredictable interplay of light on the building’s membranous form, the whole structure becomes an embodiment of the I Ching’s philosophy of chaos and inter-connectivity.
Ink painting for earth theme. Courtesy of Jennifer Wen Ma
Earth theme on the Water Cube. Photo courtesy of Wang Xin
To turn the emoji and hexagrams of the I Ching into patterns of light was quite an undertaking and it was down to Guillermo Acevedo as creative technology director to manage turning the expressions of the Chinese people into expressions of light. He and his team did this using the visual program language vvvv, which allowed them to code the data from Weibo into patterns for the different themes: fire, water, wind, earth, and thunder.
We fired off a few questions to Acevedo to find out a bit more about how he did.
The Creators Project: Describe your role in the project?
Guillermo Acevedo: The title I have in it is creative technology director. What this means is that I’m responsible for any part of technology in which creativity is present. Basically to interpret Jennifer’s ideas and make them happen. To illustrate this here are some of the tasks I did:
Talking with Jennifer we came up with the idea of measuring micro-blogging as a way of interpreting the day’s emotion. I also outlined the different parts of the system from scraping Weibo to generating the visuals to doing the daily updates. I worked with a superb Chinese programmer (Liang Mao Yin) to do the Weibo scraping (he did all the Weibo coding I just gave him requirements). I created the programming that makes the effects and renders the animations, and then I trained two programmers (Zhao Fei and Zhao Liang) on the programming and how to adjust it. Jennifer would tell me what effects were needed, then we would discuss what that would look like and I would then do the coding to make them happen. The Beijing programmers would sit with Jennifer tweaking the effects until they looked how she wanted them. I would troubleshoot and help if needed in this part, but a ton of detail work was done by them. I also helped choose the hardware. In a way I was her engineer and also her paintbrush.
The Creators Project: How does the relationship between the data (the emotional state of Weibo and the I Ching) and the visualizations work?
Acevedo: The piece is about the day that just happened, and the the day is a combination (a harmonious relationship) between the nature of the day and the people’s experience. We know the nature of the day thanks to the I Ching, and we gathered the experience of the people through Weibo. The nature of the day is the foundation. For example, the I Ching might tell us that the nature of a particular day is thunder (the I Ching specialist Ren Zhong gave us the information for each day). Jennifer’s color theorist (Steven Bleicher) would tell us the colors, in this case certain greens, and the Weibo aggregate data modifies these elements.
The thunder theme
In the case of thunder the emotional data determines the amount of thunder strikes and the frequency lightning bolts. For a different day the effects are different, for example another day could be “Heaven” and the emotional content determines the speed of the growth of the circle (heaven is represented by a circle that grows and shrinks. A sad day gives us very slow movement, a happy day gives us faster movement). In another animation the emotional content might determine the overall color.
The Creators Project: What was the most exciting thing about programming for such a large surface?
Acevedo: Programming for such a large and public surface was great! Of course it is awesome to make something that is so visible and by so many people, but I would say that my favorite part is standing right next to it where the size is overwhelming and the animations and color changes are so large that it feels like you are standing next to a giant animal.
It is impossible to comprehend what is going on to get a view of the whole thing, but you can see the tiny little variations made by all the hundreds of lamps behind the skin. The ripples of color come at you and swoop by, leaving you behind and making you wonder what it was that just happened. Since the imagery that drives the animation is so organic it really feels alive at that distance, as if you are standing next to a giant alien animal and you can detect the breathing and blood running beneath the skin. It’s what I imagine swimming next to a blue whale must be like.
The Creators Project: What were some of the programming challenges you faced?
Acevedo: I live in New York, and the testing had to be made in Beijing with results video taped and then emailed back. This made for a very slow cycle. It was also impossible to get the colors correct until we were in Beijing ourselves as any color variation was always attributed to whatever camera was used to film that night’s test.
Another challenge was the translation from normal computer screens with millions of pixels to the water cube that has only about 72 pixels across, and they are not lined up in a straight line. It took a while until we were able to start seeing how the water cube would react. For example any small element would be invisible, and straight line or border comes out like a zig-zag instead. Towards the end however we were able to use this to our advantage and were able to use simple geometric shapes that render quickly (for example water is just a bunch of rectangles, fire is a stream of bubbles). It was an interesting mental shift that we had to do.
