Motion Chain
A Webcam Game for Crowdsourcing Gesture Collection
Extended Abstracts of ACM SIGCHI, 2012
Markerless Motion Capture in the Crowd
Ian Spiro, Thomas Houston, Christoph Bregler (2012)
Collective Intelligence, 2012
3D Skeletal Reconstruction from Low-resolution Multi-view Images
Mayank Rana, Graham Taylor, Ian Spiro, and Christoph Bregler (2012)
IEEE CVPR Workshop on Human Activity Understanding from 3D Data (HAU3D)
Learning Invariance through Imitation
Graham Taylor, Ian Spiro, Christoph Bregler, and Rob Fergus (2011)
Proceedings of the 24th IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Pose-Sensitive Embedding by Nonlinear NCA Regression
Graham W. Taylor, Rob Fergus, George Williams, Ian Spiro and Christoph Bregler (2010)
Proceedings of Advances in Neural Information Processing Systems 23
Hands by Hand: Crowd-sourced Motion Tracking for Gesture Annotation
Ian Spiro, Graham Taylor, George Williams and Chris Bregler (2010)
IEEE CVPR Workshop on Advancing Computer Vision with Humans in the Loop
Eye-Mouse Coordination Patterns on Web Search Results Pages
Kerry Rodden, Xin Fu, Anne Aula, and Ian Spiro (2008)
Extended Abstracts of ACM SIGCHI, 2012
I created Cryptagram to enhance online photo privacy by adding password protection to JPEG images. Image data is first encrypted then encoded as shades of gray that are written to a new image. The protocol is robust to a limited amount of JPEG compression so that uploading a Cryptagram to a social network will not corrupt the embedded data.
Cryptagram includes a browser extension so that images can be decrypted in-place. This means users can maintain the same basic experience of any social media or website. I wrote the Chrome Extension, making extensive use of object-oriented JavaScript through the Closure framework.
The Cryptagram Encoder is a drag-and-drop web application that resizes, encrypts and encodes images 100% client-side. Written in JavaScript / Closure.
Motion Chain is a game where players collaboratively build a video corpus of human gesture. I created the concept and implemented it with PHP, CodeIgniter, and Flash. Motion Chain won the CHI 2012 Student Game Competition for best serious game. video
In Charades mode, users create short clips and friends try to guess the answer.
As a member of the Movement Lab, I have assisted in a variety of challenging motion capture shoots. We collaborated with the New York Times on several movement-oriented digital features. We captured the conductor of the NY Philharmonic , an Olympic swimmer and two Olympic divers.
I developed a keyframe-based rotoscoping tool for use on Mechanical Turk, written in Flash with a backend in PHP for queueing and managing tasks. The pipeline produces tracking of human subjects quickly and cheaply. video
I supervised a rewrite of the system in JavaScript and scaled it to handle larger marker sets. We have used the system to obtain 13-point 2D motion capture of iconic walk cycles and baseball pitches. video
I made an iOS app that does real-time filtering on photos and video. Image processing is achieved with C++ code that leverages OpenCV. video
Squidball is a motion-capture-based system for crowd gaming. A Vicon system produces motion capture data which is streamed through a server to the game engine. For five days, we ran live events at Ars Electronica in 2010.
I created several programs on top of the Squidball platform with Processing including a breakout game where the crowd attempts to break the virtual blocks.
In Guessball, the crowd tosses around the balls to reveal and guess a secret image.
I created the Tachistoscope with Processing and PHP. Users type in search queries which are used to retrieve 30 relevant images from Google Images. Images are played back at high frame rate to create visual ambiance. Framerate and other variables are controlled with a MIDI keyboard. video
I used Processing to create Hottest or Not. The software displays live thermal camera data and highlights, objectively, the hottest person in the room. video
I made Dot Party with C++ and OpenCV. Videos uploaded through the Dot Show app appear at random, creating a visual experience that is consistent but always changing. video
Dot Show is an iOS app that converts video into green dots– pure motion signal that greatly obfuscates human identity. The early iPhone API did not allow image processing so the app uploads to a remote server where video is processed in C++ / OpenCV. I developed the modot filter as an extension of Lucas-Kanade optical flow. video
Wlog is a site I built for keeping track of my progress walking around New York City, written in JavaScript with the Google Maps API.
In the newer version of Wlog, I record my progress with GPS. I use RunKeeper on an iPhone then export the data as KML. As an experiment, I am tracking my coordinates for all of 2013.
Sometimes I sketch abstract forms using myself as a pencil and upload the results to a Tumblr blog.
I wrote Plenovision in Objective C++ / Cocoa for a final class project. The application interactively visualizes the concept of the light field in 2D, as explained in the writeup.
During an internship at Google, I had the pleasure of working on the first Gmail April Fool's prank. I created the concept, co-wrote the copy, and worked with product managers to get the right visual feel.
I worked with a designer to create a homepage for my floor at NYU. Using CSS, I converted Photoshop mocks into a PmWiki skin so the site could be updated without writing HTML.
I designed and built homepages for two Stanford professors.
I implemented a design for this fashion company's homepage in Flash.
The header logo was dynamically generated by culling images from blogs that started with the appropriate letter.
I implemented a basic storefront with PHP and the PayPal API.
We Feel Fine is a web-based art project by Jon Harris and Sep Kamvar that tries to visualize how the blogosphere is feeling. I helped build and maintain the data collection system to harvest fifteen to twenty-thousand sentences per day.
The interface allows users to search across millions of records for feelings matching any combination of six variables. I implemented sharding on the MySQL database to allow these queries to happen in a reasonable amount of time.
Houndbite is a podcasting site based on audience-submitted content. I consulted on the concept and design and did the entire build in PHP plus a Flash audio player. This also included user registration, voting, integration of a Java recording applet, a store, and administrative tools for maintaining the site.
Dlog is a visualization of the writing process I created with JavaScript. As a user types, all text and edits are captured down to the millisecond. The document can then be played back as a movie.
Dlog documents can be saved to the server for future playback and shared with others. Documents can be saved as hidden, read-only, or password-protected. Data is stored as zipped JSON in MySQL. Some examples: a blog post, a poem, a screenplay treatment
I built Fast Food Maps with PHP, JavaScript and the Google Maps API. I scraped and geocoded fifty-thousand locations from public websites with shell scripts. This was the first Maps mashup that could handle so many data points. The site appeared in Wired Magazine, The Wall Street Journal, and the MoMA exhibit Design and the Elastic Mind.
The Maps API could not handle more than a few hundred markers at a time, a limitation I overcame by aggregating points based on zoom level and by only drawing markers that appear in the current view. As the user drags, new location data is seamlessly downloaded and rendered.
I created static visualizations with Matlab and Illustrator to better reveal the overall distribution of each of the ten chains.
I built several Flash games for teen entertainment site Urbaniacs. I turned sprites and audio files into fully functional games. Which Way Is Up? is a coin-toss gambling game.
Flingo Flango is a clone of Joust that includes a basic AI system and simulated squirrel physics.
Goatfist of Fury is a whack-a-mole clone.
I cofounded The Wooden Robot, a comedy group in San Francisco. The Wooden Robot developed a variety of web-content, produced short films, and put on several live shows. My focus was creating comics with Illustrator and interactive Flash pieces.
I used Flash to create an experimental genre that combined humor and games. In Potato Chips, the user controls an articulated arm to guide chips into a robot's mouth. In Cretaceous Quest, the user helps a dinosaur eat grass. Bacon Quest is very easy. The interface for the games is a puzzle, intentionally non-intuitive for comedic value.
I created the Hipster Tee Shirt Generator, one of our most popular pieces. The program uses a grammar and a library of images to generate random shirts that parody a style popular in 2006.
Trainovision is a trick for creating 3D images with a single moving camera. I mounted a web-cam on the window of the Caltrain and recorded movies of the passing landscape. I wrote Java software to combine successive frames from the movie to form stereo pairs.
I curated stills from my 3D movies to show interesting scenes along The Peninsula. Images can be viewed in regular 2D, anaglyph, convergent stereogram or divergent stereogram.
Images are viewable in 3D by wearing red-cyan 3D glasses.
Lifegraph is a design concept I implemented in Illustrator.
I revisited the concept in 2009 for my alma mater class book.
Wedding Towne was the first data-driven website I built. I wrote a script that looked for sites of the form name-and-name.com for all combinations of the thousand most popular names, then checked for the presence of the word wedding. In 2005, there were at least 15,000 such websites.
The site provides access to a wealth of hidden stories, a snapshot of life in the 21st century.
The interface encouraged users to vote for sites and post commentary.
College Prowler is a startup that produced full-length guides for popular universities. I wrote a 150 page guide for Stanford University which was published in October 2005.
I was the editor of the 105th volume of the Stanford Chaparral, the nation's second oldest continuously published humor magazine, which distributes to over 6,000 students. My responsibilities included leading weekly meetings, producing and editing content, designing layouts, and developing business plans.
I regularly collaborated with artists to produce covers. Pictured above are covers I cocreated for the Origins issue and a National Geographic parody.
As art director and eventually editor, I laid out dozens of issues over four years. I liked to combine humor with design to create highly visual pieces.
Each year, the Stanford Chaparral creates a parody issue, that historically included spoofs of publications such as National Geographic, Playbill, and the Sky Mall catalog. My volume of the magazine created a parody of a literal pile of paper. This fictional pile belonged to an eccentric character and included letters, pamphlets, to-do lists, and advertisements.
I created the pile concept, wrote about a third of the content, did most of the layout, and coordinated the complex production of the issue. We produced 3,500 copies of the pile and distributed them haphazardly throughout campus. The project was written up in the The New Yorker.