The Interactive Programming class is a capstone course in the New Media & Web Development majors and is used to teach programming and visualization concepts to students. All of the pieces for this show were programmed using the Processing language, which is loosely based on Java syntax.  It’s a great language for teaching programming concepts, but also it provides an extremely powerful framework for interactive (and static) visualizations.  This is the first year I’ve dedicated class time to teaching Processing, and the students have really enjoyed working with the tool.

The project requirements were that students generate interactive art from input sources other than keyboard and mouse.  Many groups are using video camera input, physical knobs, ardunios with sensors to detect distance, temperature, pressure.  One group is even using a MIDI keyboard.

On Thursday we had an in-class critique of the (mostly) completed projects, below are a few low quality photos of the work, and a brief description of the projects.  I will write a follow-up post with videos and photos from the opening on April 20.

PROJECT NOSTALGIA : ryan gorrie, david oslin, brian carraghan, andrew messina

The piece using inputs from 4 Griffin Powermates (USB rotation knobs) to create an interactive “etch-a-sketch” but the visualization builds dramatically on the concept.  Certain combination of inputs change the thickness and quality of the line, an input controls a gradient of color changes, and over time each of the lines appear to drip and overlap creating a graffiti/painted interactive scene.

MUSIQUBE : francesco galati, freddy diaz, tim unger, kyle kreoll

This team is the only group without a digital visualization (all the other teams are projecting their visualization), instead their interactivity is expressed by way of audio output and a series of LEDs.  The team is reading input from every key on a MIDI keyboard, and through an arduino, they are lighting a series of LEDS that have been built into 4 cubes.

IMPLIED COERCION : robert stevens III,  donna chastain, lindsey corbin, kelly burham-crisman

The Implied Coercion piece reads a variety of inputs using (THREE!) arduinos connected to sensors: distance, pressure from a floor mat, and temperature, as well audio from a microphone.  The piece reacts visually to each input in unexpected ways.  The plan is to “wire” the gallery with inputs so users in the space will build the artwork dynamically.

E-BOOTH : brandon huertas, justin gabrie, oscar duarte

E-booth is built on the concept of visualizing the “elements” specifically fire, water, smoke, and earth.  Using a variety of libraries, the team has transformed video input into an scene where the user is immersed in the selected element.  The different elements are triggered (switched) from audio and button-press. This project is built using the openCV library.

PNG PONG : salman guen, james darwin, robert lowrie, gary gipson

The piece is built around camera input, specifically, the display starts as a video of you captured from a web cam. Over time spheres fall from the top of the screen and begin to stack.  As your body interacts with the spheres they disappear.  As more users interact with the space they create different “piles” of fallen elements.  This project is built on the openCV library.

DOS HELL : josh boese, adam macasevich

The Dos Hell piece is also based on video input, but this piece focuses on the outline of body shapes and fills the body shape with an animated binary pattern (think “The Matrix”).  The pattern is manipulated by audio input captured from a microphone that’s part of the scene.

In other news about me, I will be presenting at New Vox Gallery in Bethlehem this Wednesday at 7pm as part of their “Artist Salon” series.  I will be discussing my design process, teaching, and showing some work-in-progress.

As promised, here are a few shots from the Soft Machine show:

For my pieces I spent most of my efforts revising Frequency Ink, to create a more fluid, interactive experience.  I optimized the code to draw more efficiently and modified the software to run off-line from a XML file pulled from kuler’s top 100 color selections from December 17, 2011 (the date of opening of the show).  The piece cycles colors every 15 seconds from the kuler color schemes, and draws visualizations based on the volume, frequency, and duration of audio captured in real-time in the gallery space.  The visualization is projected on a large wall-mounted three-dimensional surface.

As a new addition to the piece, I built a small Processing application that dynamically creates a time-lapse video from captures from the piece, which is shown on a video monitor displayed in the gallery.  The output is similar to to the video that I manually created (with After Effects) for a previous show, but it now works in real-time.  The advantage of the new application is that images can be dynamically added to the full-screen video, so as new shots of Frequency Ink are saved (happens every 30 seconds), they are automatically added to the time-lapse.  The images are looped at 30 frames per second, which creates a very fluid video which will run and expand during the course of the show.  Each image will represent a moment in time that captures not only the audio signature of the gallery at that moment, but also a combination of the color combinations submitted and voted upon by people all over the planet.

In a follow-up blog post I’ll post some photos from the show, and eventually the complete time-lapse capture, at the completion of the show.  The exhibition is up from December 17 – January 18th @ Soft Machine Gallery- please check it out if you’re in the area.

tell application “QuickTime Player”

  quit

end tell

delay 1

do shell script “open -F ~/Documents/kolam-drawings/final.mpg”

tell application “QuickTime Player”

  activate

  delay 1

  tell application “System Events”

     keystroke “l” using {command down, option down}

     keystroke “f” using {command down, control down}

  end tell

  play document 1

end tell

So, here’s what’s going on.  We already have a new MPG generated from the shell script, so we first kill quicktime and explicitly open the file from the command line.  The do shell script ”open -F ~/Documents/kolam-drawings/final.mpg” seemed to be the best way to open Quicktime without retaining any old preferences, specifically the -F flag opens it fresh.  The reason this is an issue is that in Lion, Quicktime (all Apple apps really) try to remember your previous settings, so if you previously had a video open, it will open it again, if you had the video set to loop, it will set it again.  So if you run a script that opens a file and turns on looping, the looping will be toggled on and off every time.  So opening it fresh forces a new state every time.

One the video is open, I literally send key commands to loop and fullscreen the video (the keystroke commands).  From what I can tell, Quicktime app is fully scriptable, but I was unable to reliably access the loop and full screen settings from the built-in scripts.  And finally the video is played & then is looped until the shell script calls to the AppleScript, which kills the video and starts the process all over again.  If anyone knows a better way to do this, please email me, I’d be happy to post variations on this page.

As you may know from browsing the site, I just finished up a gallery show with my colleague, Bruce Wall.  The show is entitled The Art of Indian Kolam: Traditional Designs & New Media and it featured a true integration between Bruce’s fine art work and my digital interactive pieces.  Bruce and I collaborated on a variety of installation pieces for the show, all of which are documented extensively on the site.  In the name of “open source” I plan to put some code from the show up on this blog in the coming months so it can be scrutinized and picked apart by the nerd community.  All of the interactive pieces were written in Processing, and utilized a variety of libraries to work on multiple screens, interface with a kinect, and replace/enhance the built-in quicktime libraries (which as it turns out only work in very specific scenarios).  The final piece we created for the show was the Animated Virtual Kolam which captured all of the images draw on the Kolam Cube and concatenated the images together into 2 movies, one that runs at 10 kolams per second and one that runs at 1 kolam per second.  Then the videos were combined together and looped on a wall-projection.  The script to run this is updated every 10 minutes so the newest kolams are displayed “live” with almost no interruption.  The Animated & Virtual Kolam videos give a good overview of the pieces:

Animated Virtual Kolam

Kolam Cube

Okay, so now for the exciting part… code!  The goal of the animated piece was to be live, looping, and engaging.  There’s no way that we could manually download the images, create a looping animation and copy it to the mac mini with the projection- it had to be automated.  So I turned to the power of the unix command line in OS X to make it happen.  The first step was installing mencoder & ffmpeg on OS X (with some terminal tinkering, you can easily install ffmpeg via the command line without having to mess with fink). The I wrote a shell script that looped every 10 minutes which grabbed the newest kolams and created the video.  Here’s the code:

#/bin/sh

sleep 30
while : ; do

 # get only the new drawings
 rsync -avzv -e ssh SERVER.com:~/DIRECTORY/*.png ~/Desktop/kolam-drawings/

 # make 2 videos one at each speed
 mencoder "mf://*.png" -mf fps=10 -o fast.avi -ovc lavc -lavcopts vcodec=mjpeg
 mencoder "mf://*.png" -mf fps=1 -o slow.avi -ovc lavc -lavcopts vcodec=mjpeg 

 # convert videos to mpg, convert to 30fps, but retain speed of original
 ffmpeg -i slow.avi -sameq -r 30 -y slow-30fps.mpg 2> /dev/null
 ffmpeg -i fast.avi -sameq -r 30 -y fast-30fps.mpg 2> /dev/null

 # concatinate the two video files together
 cat fast-30fps.mpg slow-30fps.mpg > final.mpg

 echo ":::: starting applescript :::: " `date`

 #run applescript to launch quicktime with the file, loop & full screen
 open ~/Desktop/playmovie-exe.app

 # wait 10 minutes
 sleep 600
done

The first thing you should know is that images from the Kolam Cube piece are uploaded to kolamncc.com (via scp) each time an image is drawn- it’s what’s used to draw the dynamic kolam gallery on the site.  Which means the images were already out there… so they key was to pull down only the new images that weren’t currently on the animated kolam mac.  I considered a variety of other options like zipping the directory on the server and wget’ing it, decompressing, and starting from scratch but as the library grew that would be unmanageable.  The answer: rsync.  As you will see in the code, with a few switches you can set rsync to download only new files and download them to a set location.

After we get the files, I use mencoder to create an 2 avi’s at different FPS- one at 10FPS and one at 1FPS.  I found that mencoder was able to convert from PNGs to AVIs without loss of quality and limited errors (though in the production box, I added 2> /dev/null to these lines to hide any errors).  So, now I had AVIs, which were playable in the Quicktime player, but they were completely different frame rates so they couldn’t be combined on there own.  FFMPEG seemed to be the easiest answer to this problem.  I’ll stop here for a moment, and for the 3 people who continued reading this far, here’s a tip… FFMPEG is the greatest unix application of all time.  I’ve used it to convert every imaginable video format and frequently use it in PHP projects to convert videos on the fly (it converts anything to FLV’s about 50 times faster than Adobe Media Encoder)… incredibly useful.  Anyway, FFMPEG takes the AVIs, converts them to 30FPS and retains the original speed- because if it lost the speed, there would be no point to the whole process.  I decided to convert them to MPGs because without and “wrapper” garbage, you can simply concatenate two MPGs together to combine videos… which is exactly what happens next.  Then I pipe that final movie into an applescript that kills the current video loop, opens the new program and full screens it.  I’ll include applescript in another post. I’ve already rambled too long about super nerdy unix commands.  And finally, the last line of code waits 10 minutes and repeats the infinite while loop again.

Took a lot of tinkering to get this to work, so hopefully this will help others.  Too much for the first blog post??

Kolam is an ancient folk art form that is still practiced daily on the floors of Hindu Temples and on the doorsteps of homes by the women of South India.  Through this exhibition, Professor Bruce Wall and I modernized and reinterpreted the ancient art of Kolam creation. My role in the project was the creation of a variety of digital interactive art installation pieces.  Details of the work can be found on the Kolam Gallery page.

At the completion of our most recent exhibition, I generated a render of the 1060 digital kolams that were drawn and added to the Live Gallery from the Virtual Kolam installation: