ArtLens Exhibition is an experimental gallery that puts visitors into an immersive world of art and interactivity.

The permanent exhibit combines 16 completely custom games with thousands of artworks in the museums collection, allowing visitors to explore, interact with and learn about art.

The Cleveland Museum of Art has a wonderful description of the overall experience on their website, but below is a brief overview of the three main areas of the exhibit that I was involved with, focusing more on the technical aspects:

Interactive Lenses:
(6) 14-foot tall projections surround the outer walls of the space, each with it's own Microsoft Kinect camera, focused audio and multi-user interaction space.

Each interactive allows visitors to explore 14 gesture-based games across the museums entire collection using:

• Hand Tracking + Drag and Drop
• Skeleton Tracking + Matching
• Facial Tracking
• Facial Recognition
• Facial Replacement
• Computer Vision

Gaze Tracker:
A more intimate experience, this single-user seated interactive combines a capactive touchscreen with an SMI Red-N eyetracker.

Visitors are asked to stare at various artworks for a set amount of time and see what they discover. They are then presented with a replay of their eye positions, along with those of previous visitors to the museum.

Personal Taste:
Visitors are presented with a rapid display of artworks, and their facial expressions detected, classified and captured.

After time has expired an interactive menu allows visitors to see both their reactions and the reactions of others. They can then create animated gif sequences of their reactions and send them to their mobile device for sharing to social media.

Forest Friends is a multi-location interactive installation on the Ohio State Wexner Medical Center's Pediatric Radiation Oncology floor.

Patients and their families are given NFC-enabled wrist bands that the are able to scan at multiple locations. This allows them to travel with an interactive animal friend who serves as a positive distraction and a model of behavior during each stage of treatment.

The installation consists of the following parts:

Waiting Room
The waiting room is where patients start their journey and select an animal. This room consist of a 6 screen multitouch wall where patients can explore a forever-changing landscape, plant trees, discover hidden interactions and play with their animals. An ambient soundscape provides soothing audio for parents, while focused sound domes provide interactive feedback for patients.

Exam Rooms
Three separate exam rooms each contain a multitouch monitor in portrait mode that allows patients to play a 360 degree game of hide-and-seek with their animal. In addition, each unit provides ambient LED lighting that matches the current area of the interactive, allowing the entire room to take on soothing glow.

Induction
Young patients need to be sedated nearly every day for the length of their treatment, which adds to the anxiety levels, treatment time and overall family stress. The induction room interactive consists of a projection in a dark room that shows a patient's animal friend slowly and calmly walking in the room, yawning and indicating that it is time to rest. This provides a soothing experience more similar to nap-time than a hospital sedation, and helps both patients and staff.

The Bell
At the end of treatment, patients traditionally ring a loud bell so that everyone in the hospital can congratulate them on their achievement and be inspired for their own treatment. We wanted to incorporate this tradition into our expreience, but we did not want the technology to overtake the physical symbol that the bell had become.

In order to do this we embedded a small Piezo sensor attached to an ARM-Based microcontroller and had a custom mounting plate fabricated that held all electronics without any visible technology. We then were able to send UDP messages to all of the interactives in the space when the bell was rung, which set off a multitude of animations, animal dances and celebratory sounds.

The Art Institute of Chicago had three separate applications, and none of them were working how they envisioned. They needed a new solution that would offer indoor navigation, media playback and membership card functionality all in one app.

We were able to collaborate with them to plan, develop, test and deploy the next generation of the Art Institute of Chicago's mobile experience.

This project was developed 100% using Swift, and the museum has open-sourced it since it's release. You can view the source code (stripped of commits for security reasons) here:
Art Institute of Chicago Mobile App on Github

Highlights:

• 'Always On' mapping feature
• Indoor location, with floors and 5 meter precision
• New podcast-style audio tours integrated with the user map and location
• Persistent audio player with cover image, artwork and artist information and text translations for hearing impaired
• Legacy audio guide tours integrated through a custom keypad
• Integration with museum membership, 'Scan Your Phone'

Pennant is an interactive exploration of baseball data that allows users to quickly and accurately recreate and investigate the history of the annual race to be the best team in Major League Baseball.

Highlights:

Original Thesis: The past century has seen a rapid progression in the way we see and experience live sports. As we now find ourselves with access to every live game detail imaginable across a multitude of devices, we must now ask ourselves the question “Where does all of this information go?” Pennant is both an attempt to explore the vast amount of baseball data that has been collected in the last sixty years as well as a study in using interactivity as a means for investigating the extremely large data sets that are becoming increasingly available.

Update 2017:
Pennant was more successful than I could have ever imagined. What originally began as my Parsons MFADT Thesis project turned into one of the top-downloaded apps in the first few months of the Apple iPad app store. This led to:

• An Apple Design Award at the Apply WWDC 2011
• A contract with HP to port the app over to WebOS (RIP)
• Multiple mentions and awards in both sports and technology publications
• A sale of the application to Topps, Inc. (the baseball card people)
• An iPhone version

As of 2017, Topps is sadly no longer supporting the application, which is dissapointing, but overall it was a great 6 year run and an amazing learning experience.

Smithsonian Networks is a joint venture between CBS Corporation's Showtime Networks, Inc. and the Smithsonian Institution.

As a new network the Smithsonian Channel needed a mobile experience that emphasized discovery, making it easy for users to experience the shows that interested them and keep coming back for more.

What began as an iPad experience in 2011 slowly evolved into a multi-platform, multi-device ecosystem. I was a core member of the first several versions, and as the scope of the project evolved I worked with the network, designers and multiple development teams to expand the product while leveraging a shared front-end and back end.

Each experience is uniquely tailored to a specific form factor, offering not only full access to the network's catalog of shows but also a unique way on each platform to keep discovering all the great content that Smithsonian Networks has to offer.

Platforms:

• iPad
• iPhone
• Universal App
• Apple Watch
• Android

Reduce, Reuse, Recycle and Rethink is a 20+ person multi-touch interactive in West Palm Beach, FL.

The Solid Waste Authority of West Palm Beach recently completed their $1 Billion waste to energy facility and needed away to explain it to a diverse range of tour groups.

Goals:

• Entertain and educate school groups
• Explain the technology behind the new facility
• Provide an overview of all of SWA's operations
• Allow tour leaders and educators to switch modes, trigger videos and start/stop games at any time.

In order to accomplish this we created at 10 Screen/36 touch experience with 5 distinct modes. These modes and the interactives within them are controllable by the group leader using a cross-platform mobile application. This means that each tour can be uniquely timed and the group leader has the power to define their own custom experience.

Supported Modes:

• Welcome mode: When the group first arrives a custom message welcoming to the facility can be entered using the mobile app.
• Video mode: The tour leader can play different videos on demand in order to explain concepts or introduce tour members to the facility/
• Interactive Map: A full-table map of both Palm Beach County and the facility, including live streaming videos from various stages of waste to energy, semi-live tracking of trash trucks and in-depth information about the various processes taking place.
• Sorting game: The group plays as a team, attempting to hand-sort as many incorrectly recycled products as possible before they reach the trash.
• Quiz Game/Shopping: The group leader can pose different questions to group members in a gameshow format

The Northwestern University Connections Wall is a 13 foot wide 9 user Multi-Touch experience at the Segal visitors center. The goal of this wall is to connect prospective students with the history of the University in a way that shows off the present day high-tech focus.