How Sesame Street Is Re-imagining A Show’s Storyline For Kinect-Era Television [PSFK LONDON]

How Sesame Street Is Re-imagining A Show’s Storyline For Kinect-Era Television [PSFK LONDON]

The Creative Director of the Content Innovation Lab at Sesame Workshop talks to PSFK about the future of educational programming centered on interactivity.

Timothy Ryan, PSFK Labs
  • 8 september 2012

We are excited to have J Milligan, Creative Director of the Content Innovation Lab for Sesame Workshop, come speak at PSFK CONFERENCE LONDON. Sesame Workshop has recently partnered with Microsoft to leverage the Kinect for enhanced educational programming centered on interactivity. On September 13th, Jason will discuss reimagining a show’s storyline for Kinect-era television.

Sesame Workshop has been experimenting with digital learning materials for some time. What are the challenges around designing interactive programming for kids?

If you’re interested in teaching literacy, numeracy, rhymes or anything else, you will always be fighting the platform. The first set of instructions is always about moving the cursor around, picking up the mouse, or just trying to deal with basic interface issues before we can even teach anything.

The first thing we always have to teach is how to use the platform, but that’s been falling away more and more lately. First, going to touch, which is such an intuitive thing for kids. Many children can already navigate an iPhone at a very young age. Touch is intuitive, and Kinect is even better because it’s just about moving your body and using your voice to make things happen on screen.

I’ve been designing digital curricula for a while and it has always been a challenge to design for an age group appropriately. Whenever you hear people talk about design, they talk about designing for the extreme user; if you design for the extreme instead of the middle, the logic goes, everything will work itself out. Designing for preschoolers is definitely one end of the extreme.

Recently you’ve partnered with Microsoft to develop programming leveraging the Kinect platform. What is the opportunity?

When Kinect came along we had a viable platform available at scale that millions of people already had in their homes. Bringing video to the platform and making it react to the player and what they’re actually doing finally brought together some of our thinking around creating engaging educational material in a significant and incredibly interesting way.

Sesame Workshop had always been interested in using television as a tool to teach. If a platform can react to a viewer or player in a meaningful way, it makes an experience that much more personal. We’d been looking at ways to do this for a long time: back in the 1980s we did a project with Viewmaster using VHS to build an interactive TV experience, which is kind of incredible. We did a big workshop with AFI in 2002 to use a DVD platform to create an interactive experience, and we also did some work with CableVision using the remote. At that time, a lot of people who were working the space were doing what I call ‘science experiments;’ you could make interactive TV, but you had to build your own hardware and software to make anything happen.

What specifically with respect to the Kinect is so engaging and intuitive for Sesame Workshop’s audience?

Sesame is all about making the whole thing fun and entertaining, and ‘engagement’ is exactly the right way to describe it. The more you’re engaged in the content, the more you’ll get out of the learning materials we’re presenting. For example, in the case of the Kinect-enabled game Letter Tree, what we’re really teaching you is that ‘A’ is the first letter sound. All we’re asking you to do is jump, and that’s something kids are very willing to do.

We’re focused on embedding interactivity into the narratives we’re creating, where the things we’re asking the kid to do just feel natural and organic. Now we’re able to bring that experience in to your living room. When Elmo throws you a ball through the glass of the TV, you throw it back to him and he catches it. When Cookie Monster says, “I need your help, there’s something stuck in this tree that I want to eat, jump to shake this tree,” the viewer will start jumping. It’s just incredibly natural for a kid to make a throwing motion or start jumping. We’ve seen this in testing, and I’ve seen this in my own home – it’s great.

When it comes to Kinect, I would definitely say we’re still figuring it out, but currently we have eight shows coming out; eight episodes coming out around the time of the conference and then another eight on Christmas day.

Have the goals of Sesame Workshop changed at all over the years?

Our mission is to reach kids on every available platform, and give them developmentally-appropriate, educational content to get them ready for school and life. Sesame Workshop, as you may know, is going into our 43rd season of production this fall. It was a show that Joan Ganz Cooney and Lloyd Morisett invented in the late ’60s (with help from Jim Henson and the Muppets) to harness the power of television to get kids excited and prepared for school.

What is your role in creating those platforms?

I’m the creative director of the Content Innovation Lab, a small group of six people who look three to five years out to determine what platforms are going to be relevant and what Sesame should be working on. We build capacity for the company as a whole on things like gesture and voice and my role has always been on the digital side. I’ve been designing interactive experiences for (mostly) preschool kids for almost 17 years.  I’ve designed on every major platform to come along since the mid-1990s: things like PC CD‑ROM to web, consoles like PS1, handheld, Gameboy and then lately moving to mobile apps, touch‑screen iPad, iPod, iPhone and now Kinect. Back in the day, Sesame Workshop was doing software for things like the Apple II, and so the digital side of things goes back almost as far as the television show. Now we’re bringing it all together. The project that I’m bringing to the PSFK London Conference, “Connect Sesame Street,” is basically interactive TV on the Xbox console.

Thanks Jason!

Jason Milligan/ @jmilligan718/ Sesame Workshop

Come see Jason discuss re-imagining a show’s storyline for Kinect-era television at PSFK CONFERENCE LONDON on September 13th.


Modular System Lets Musicians Create Their Own MIDI Controllers

Arts & Culture
Travel Yesterday

Mercedes Reveals Dazzle-Free LED Headlights

Digital Light offers great precision with a resolution of over two million pixels

Product Launch Yesterday

Nissan Is Testing A Digital Car Sharing Program In Europe

Nissan plans to launch their new service in Paris sometime this year to trial the profile-matching service


Get PSFK's Related Report: Future of Automotive

See All
Social Media Yesterday

Your Favorite Tweets Are Now Wearable

This temporary tattoo allows fans to wear their most favorite moments from the social platform

Health Yesterday

This Mirror Tracks Your Dark Circles And Fine Lines

HiMirror is a device snaps a photo of your face every day to provide feedback on how to care for your skin

Related Expert

Yael Maguire

Connectivity, Accessibility, Wireless Technology

Sustainability Yesterday

Biodegradable Furniture Made From Pine Needles Could Be The Next Phase Of Sustainable Living

Premiering at Dutch Design Week 2016, the collection fully utilizes an often wasted material

Syndicated Yesterday

Madrid's 'Robin Hood' Cafe Charge The Rich To Feed The Poor

The charity restaurant makes money from customers by day to offer homeless people meals at night

USA Yesterday

Amazon Launches Cashier-Free Store For Ultimate Efficiency

The retailer is looking to make shopping even faster by letting customers instantly pay as they walk out the door


Future Of Retail 2017
Transformation Strategies For Customer-First Business

PSFK Op-Ed Yesterday

Creative Director: Navigating The New World Of Founder-Brands

Richard Smith, Creative Director at Sullivan, explains how visionaries like Elon Musk and Mark Zuckerberg approach their branding and why it’s important to apply brand thinking to founders’ products

PSFK Labs december 1, 2016

Retail Spotlight: Home Depot Reimagines How Employees Conduct Tasks

The home improvement retailer puts the customer first by initiating local fulfillment centers and simplifying freight-to-shelf inventory management

Augmented & Virtual Reality Yesterday

Marble-Like Mini-Worlds Invade Miami Art Week And Your Mobile Screen

The gallery world's sphere of influence seems to be expanding into the realm of Pokémon Go—why that's a good thing

Technology Yesterday

A 'USB Stick' That Can Detect HIV Levels

The simple device developed at Imperial College London measures viral load in less than 30 minutes

Customer Retention Yesterday

Crafting The Personalized Retail Experience

Marriott International's Christopher Baer shares insights into how the hospitality company is strategizing customer service

Experiential Marketing Yesterday

Zagat's Cafe Offers Tiny Replicas Of Classic NYC Dishes

The restaurant rating service created buzz for its new guide and app by opening a tiny food cafe

Children Yesterday

Experience The White House In Augmented Reality Using A $1 Bill

1600 Pennsylvania Ave. is an AR app designed to help people learn about the history and significance of the United State's capital building


Conference Built Around Report Launch

Travel Yesterday

Portable Computer Monitor Opens Up Like An Umbrella For Travelers

This mobile screen and projector means work can go anywhere and still feel like working from home

Food Yesterday

Starbucks Is Selling An Automated Temperature-Setting Mug

For those looking to keep their coffee hot on their winter commutes, the coffee chain has created a device that keeps beverages exactly at their desire temperature

No search results found.