This past weekend I brought about 10 students to the 2013 Lehigh Valley Tech Hackathon. The event brings together designers, developers, and engineers with the goal of creating a “product” at the end of the weekend.  On Friday everyone was able to pitch an idea and we formed teams.  I ended up on a team with 4 of my Kutztown students: Kyle Desimone, Jerry Cavill, Kory Walton and Trevor Dilg, and 2 friends Ryan Hickey & Donna Chastain.  Our concept was absurd, we wanted to create an interactive application and physical objects around the theme of the brilliant 90′s cartoon Captain Planet. The gist of the cartoon is that 5 Planeteers have power rings and when they come together they summon Captain Planet, who according to wikipedia has the following powers:

• Weather and climate manipulation
• Ability to generate and control earth, fire, wind and water
• Near invincibility
• Mullet
• Invisibility
• Telepathy
• Flight
• Super strength

Team Captain Planet:

To make our Captain Planet concept come to life, we decided to build a series of ridiculous elements which included custom 3D printed rings fitted with RGB LEDs controlled by an arduino, a capacitive touch ring powered by a makey makey, a servo controlled captain planet head that moved up and down, a website that tracked the captain, a text message calling tree that alerts users when the captain is activated (powered by nexmo, which is awesome by the way), and a Processing sketch that ties everything together.

So first the rings. They were modeled in Lightwave 3D, and were hollowed out to allow space for the LED which sticks out through the opening in the top.  The file was converted to an STL and printed on a 3D printer at the Northampton Fab Lab. Download the STL and print your own Captain Planet ring!

The next step in the process was to wire the RGB LEDs to a an arduino that would power and control the lights when triggered. The sketch waits for a serial data (sent from Processing), parses it and lights the appropriate LEDs with our selected colors (and blink sequences). Download the aduino sketch.

Using some hobby servo motors we also were able to make a cardboard captain planet and control his head motion, but during the 40 or 50 tests of the process we burnt-out the servos, leaving a lifeless captain planet mascot full of broken electronics:

The final step was to bring it all together in Processing. The makey makey read physical input and sent it to a computer as keyboard input, so we mapped different inputs to keys and had Processing listen for those keys.  We wrapped a wire around the front of the rings so it could detect the presence of another person when touched (because humans conduct electricity):

The next piece was the Processing sketch (download it) which waited for “keyboard input” from the makey makey, when it received input it triggered a video to play (corresponding to the appropriate Planeteer), send a message to the arduino to light the appropriate LEDs, and then waited for the next input.  When all 5 Planeteers had been activated, the full one minute long captain planet theme song & video was displayed, and all the rings pulsed random light colors.  After the video ended all the users in the system received a text message alert and data was added to the Captain Planet Locator, which is programmed with PHP a simple mySQL database.

The whole event was a ton of fun, and we were all out of our comfort zones for the whole project.  We knew from the beginning that the project was ridiculous, but we had a great time making it.  We didn’t win (which is total BS), but the winning team totally deserved it, so we weren’t bitter (great job Joe Fritz and team!).

I’ve posted the presentation PDF and sample files.

In related news, I am teaching a “Creative Coding” class this summer in Processing at Kutztown. The class is open to students and the community.

A brief overview of the process that I used to create this video in After Effects:

1) steal audio of the clip (using listentoyoutube.com), transcribe words & record timing

2) bring the HD video into After Effects, used Warp Stabilizer to make the video look like it was shot with a steadycam… takes a very long time to process, but the results were impressive

3) use the motion tracking features in After Effects to track a couple of elements on the beach (had to split the clip into 3 shots due to panning), and generate a virtual camera motion

4) parent the text layers to the motion tracked points so the text moves with the slowly panning camera

5) then align the text in the frame & animate it for each individual phrase

6) tweak lighting (used several lights), turned all the layers to “3D Layers”, added fake shadows on the ground, blurred and adjusted levels of the background videos (to make a sunny day look ominous)

7) render like 50 times because RAM preview doesn’t work well with 1440×1080 video clips with tons of effects on it

Here’s what it does:

You enter two twitter usernames and it will generate a visualization of the user’s followers.  Here I am comparing my followers to my friend (and talented designer- follow him!), Jarrett Fuller.  My data is on the left, Jarrett’s is on the right.  From this you can quickly see that I follow a lot more people and most don’t tweet very often- with a few exceptions.  Jarrett has a far more curated list, he follows about 1/3 the number of people that I do, and most tweet often but not excessively.  So, the Follower Data tab shows the way you consume information, whereas the Tweet Data tab shows how you use twitter communicate.

The following diagram is my data vs a friend who is a prolific retweeter.  He communicates by tweeting other people’s data about the same amount of time he @ replies to someone.  I communicate directly with people more often than retweeting. Once you start dropping your friend’s usernames into this tab it’s addicting (and satisfyingly stalkerish) to see the “way” people communicate, and who they communicate with most often.

The final tab, Historical Data, visualizes when we most often communicate. As I’ll describe below, the app pulls your last 400-600 tweets, so there’s a fairly large dataset of day/times.  The graph quickly shows trends in when you tweet, and how often.

Some “how it work” details are covered in a recent blog post.

Here’s what it does:
You enter two twitter usernames and it will generate a visualization of the user’s followers.  Here I am comparing my followers to my friend (and talented designer- follow him!), Jarrett Fuller.  My data is on the left, Jarrett’s is on the right.  From this you can quickly see that I follow a lot more people and most don’t tweet very often- with a few exceptions.  Jarrett has a far more curated list, he follows about 1/3 the number of people that I do, and most tweet often but not excessively.  So, the Follower Data tab shows the way you consume information, whereas the Tweet Data tab shows how you use twitter communicate.

The following diagram is my data vs a friend who is a prolific retweeter.  He communicates by tweeting other people’s data about the same amount of time he @ replies to someone.  I communicate directly with people more often than retweeting. Once you start dropping your friend’s usernames into this tab it’s addicting (and satisfyingly stalkerish) to see the “way” people communicate, and who they communicate with most often.

The final tab, Historical Data, visualizes when we most often communicate. As I’ll describe below, the app pulls your last 400-600 tweets, so there’s a fairly large dataset of day/times.  The graph quickly shows trends in when you tweet, and how often.

So, How Does this work?
The web app is built on the twitter API on the backend using PHP, based on the code stolen from my friend Matthew Reidsma’s TweepGrabber examples.  Matt’s code was a starting point, and then I began pouring through the twitter API documentation, which has been a source of great frustration.  The twitter API, as it turns out designed to easy capture of stream of data from “this moment forward”, or query information about followers, but it’s not as easy to get historical data.  You actually have to jump through a lot of hoops to get data, there are caps on how much data you can get per call, how many calls you can make by IP, etc.  You can increase these limits slightly if you force users to authenticate, but that’s impractical (and off-putting) for my application.  More on this later.

The front-end of the application is based on D3.js, a sexy javascript visualization library.  I’m using a heavily modified version their sunburst example (which, for some reason, is called Partition when you download their source from github).  There’s limited documentation, but as compared to the twitter API nightmare, it was a pleasure.  The line graphs in the historical tab are based on an extremely simple visualization library called Bluff, once I calculated the historical data on the backend, it was trivial to graph it.  That was the easy part.

Wait, but didn’t you said this wasn’t done?
Yeah, it’s not.  It’s possible that when you visit the size you will be greeted by this horrible image:

This means I’ve hit my twitter API limit for the “hour” but the limit isn’t well-defined and neither is the hour, usually it’s less.  Because I don’t require users to authenticate, everyone who visits the site contribute to the same artificial connection limit.  After some communication with themattharris, I learned that if I use his awesome tmhOAuth library, and authenticate as myself, I can increase my connections to 150 calls per hour (which is about 30-40 visualization… still not very much).

I haven’t rewritten my code using thematt’s library, and it was on my to-do list for the summer, but with school starting (yeah! new job!) it’s not going to get done any time soon.  But with all the talk of the twitter API getting even worse, I thought I should publish what I have so people can use it.  So, all 4 readers of my blog, promise to not tell too many people, because when it goes viral everyone will see my loading gif and will think it’s my fault, but really it’s just twitter being an asshole.

The Interactive Programming class is a capstone course in the New Media & Web Development majors and is used to teach programming and visualization concepts to students. All of the pieces for this show were programmed using the Processing language, which is loosely based on Java syntax.  It’s a great language for teaching programming concepts, but also it provides an extremely powerful framework for interactive (and static) visualizations.  This is the first year I’ve dedicated class time to teaching Processing, and the students have really enjoyed working with the tool.

The project requirements were that students generate interactive art from input sources other than keyboard and mouse.  Many groups are using video camera input, physical knobs, ardunios with sensors to detect distance, temperature, pressure.  One group is even using a MIDI keyboard.

On Thursday we had an in-class critique of the (mostly) completed projects, below are a few low quality photos of the work, and a brief description of the projects.  I will write a follow-up post with videos and photos from the opening on April 20.

PROJECT NOSTALGIA : ryan gorrie, david oslin, brian carraghan, andrew messina

The piece using inputs from 4 Griffin Powermates (USB rotation knobs) to create an interactive “etch-a-sketch” but the visualization builds dramatically on the concept.  Certain combination of inputs change the thickness and quality of the line, an input controls a gradient of color changes, and over time each of the lines appear to drip and overlap creating a graffiti/painted interactive scene.

MUSIQUBE : francesco galati, freddy diaz, tim unger, kyle kreoll

This team is the only group without a digital visualization (all the other teams are projecting their visualization), instead their interactivity is expressed by way of audio output and a series of LEDs.  The team is reading input from every key on a MIDI keyboard, and through an arduino, they are lighting a series of LEDS that have been built into 4 cubes.

IMPLIED COERCION : robert stevens III,  donna chastain, lindsey corbin, kelly burham-crisman

The Implied Coercion piece reads a variety of inputs using (THREE!) arduinos connected to sensors: distance, pressure from a floor mat, and temperature, as well audio from a microphone.  The piece reacts visually to each input in unexpected ways.  The plan is to “wire” the gallery with inputs so users in the space will build the artwork dynamically.

E-BOOTH : brandon huertas, justin gabrie, oscar duarte

E-booth is built on the concept of visualizing the “elements” specifically fire, water, smoke, and earth.  Using a variety of libraries, the team has transformed video input into an scene where the user is immersed in the selected element.  The different elements are triggered (switched) from audio and button-press. This project is built using the openCV library.

PNG PONG : salman guen, james darwin, robert lowrie, gary gipson

The piece is built around camera input, specifically, the display starts as a video of you captured from a web cam. Over time spheres fall from the top of the screen and begin to stack.  As your body interacts with the spheres they disappear.  As more users interact with the space they create different “piles” of fallen elements.  This project is built on the openCV library.

DOS HELL : josh boese, adam macasevich

The Dos Hell piece is also based on video input, but this piece focuses on the outline of body shapes and fills the body shape with an animated binary pattern (think “The Matrix”).  The pattern is manipulated by audio input captured from a microphone that’s part of the scene.