How Sesame Street Is Re-imagining A Show’s Storyline For Kinect-Era Television [PSFK LONDON]

How Sesame Street Is Re-imagining A Show’s Storyline For Kinect-Era Television [PSFK LONDON]

The Creative Director of the Content Innovation Lab at Sesame Workshop talks to PSFK about the future of educational programming centered on interactivity.

Timothy Ryan, PSFK Labs
  • 8 september 2012

We are excited to have J Milligan, Creative Director of the Content Innovation Lab for Sesame Workshop, come speak at PSFK CONFERENCE LONDON. Sesame Workshop has recently partnered with Microsoft to leverage the Kinect for enhanced educational programming centered on interactivity. On September 13th, Jason will discuss reimagining a show’s storyline for Kinect-era television.

Sesame Workshop has been experimenting with digital learning materials for some time. What are the challenges around designing interactive programming for kids?

If you’re interested in teaching literacy, numeracy, rhymes or anything else, you will always be fighting the platform. The first set of instructions is always about moving the cursor around, picking up the mouse, or just trying to deal with basic interface issues before we can even teach anything.

The first thing we always have to teach is how to use the platform, but that’s been falling away more and more lately. First, going to touch, which is such an intuitive thing for kids. Many children can already navigate an iPhone at a very young age. Touch is intuitive, and Kinect is even better because it’s just about moving your body and using your voice to make things happen on screen.

I’ve been designing digital curricula for a while and it has always been a challenge to design for an age group appropriately. Whenever you hear people talk about design, they talk about designing for the extreme user; if you design for the extreme instead of the middle, the logic goes, everything will work itself out. Designing for preschoolers is definitely one end of the extreme.

Recently you’ve partnered with Microsoft to develop programming leveraging the Kinect platform. What is the opportunity?

When Kinect came along we had a viable platform available at scale that millions of people already had in their homes. Bringing video to the platform and making it react to the player and what they’re actually doing finally brought together some of our thinking around creating engaging educational material in a significant and incredibly interesting way.

Sesame Workshop had always been interested in using television as a tool to teach. If a platform can react to a viewer or player in a meaningful way, it makes an experience that much more personal. We’d been looking at ways to do this for a long time: back in the 1980s we did a project with Viewmaster using VHS to build an interactive TV experience, which is kind of incredible. We did a big workshop with AFI in 2002 to use a DVD platform to create an interactive experience, and we also did some work with CableVision using the remote. At that time, a lot of people who were working the space were doing what I call ‘science experiments;’ you could make interactive TV, but you had to build your own hardware and software to make anything happen.

What specifically with respect to the Kinect is so engaging and intuitive for Sesame Workshop’s audience?

Sesame is all about making the whole thing fun and entertaining, and ‘engagement’ is exactly the right way to describe it. The more you’re engaged in the content, the more you’ll get out of the learning materials we’re presenting. For example, in the case of the Kinect-enabled game Letter Tree, what we’re really teaching you is that ‘A’ is the first letter sound. All we’re asking you to do is jump, and that’s something kids are very willing to do.

We’re focused on embedding interactivity into the narratives we’re creating, where the things we’re asking the kid to do just feel natural and organic. Now we’re able to bring that experience in to your living room. When Elmo throws you a ball through the glass of the TV, you throw it back to him and he catches it. When Cookie Monster says, “I need your help, there’s something stuck in this tree that I want to eat, jump to shake this tree,” the viewer will start jumping. It’s just incredibly natural for a kid to make a throwing motion or start jumping. We’ve seen this in testing, and I’ve seen this in my own home – it’s great.

When it comes to Kinect, I would definitely say we’re still figuring it out, but currently we have eight shows coming out; eight episodes coming out around the time of the conference and then another eight on Christmas day.

Have the goals of Sesame Workshop changed at all over the years?

Our mission is to reach kids on every available platform, and give them developmentally-appropriate, educational content to get them ready for school and life. Sesame Workshop, as you may know, is going into our 43rd season of production this fall. It was a show that Joan Ganz Cooney and Lloyd Morisett invented in the late ’60s (with help from Jim Henson and the Muppets) to harness the power of television to get kids excited and prepared for school.

What is your role in creating those platforms?

I’m the creative director of the Content Innovation Lab, a small group of six people who look three to five years out to determine what platforms are going to be relevant and what Sesame should be working on. We build capacity for the company as a whole on things like gesture and voice and my role has always been on the digital side. I’ve been designing interactive experiences for (mostly) preschool kids for almost 17 years.  I’ve designed on every major platform to come along since the mid-1990s: things like PC CD‑ROM to web, consoles like PS1, handheld, Gameboy and then lately moving to mobile apps, touch‑screen iPad, iPod, iPhone and now Kinect. Back in the day, Sesame Workshop was doing software for things like the Apple II, and so the digital side of things goes back almost as far as the television show. Now we’re bringing it all together. The project that I’m bringing to the PSFK London Conference, “Connect Sesame Street,” is basically interactive TV on the Xbox console.

Thanks Jason!

Jason Milligan/ @jmilligan718/ Sesame Workshop

Come see Jason discuss re-imagining a show’s storyline for Kinect-era television at PSFK CONFERENCE LONDON on September 13th.


Lyft Gives Free Rides To Those Who Have Had Too Much To Drink

Syndicated Yesterday

Banned Books Week Urges People To Seek Out Controversial Works

Joining the annual celebration of the right to read, US author Jessica Herthel called for 'more information, more voices' to protect diversity

Augmented & Virtual Reality Yesterday

Outdoor Camp Presented In 360° VR By X Games Gold Medalist

The video features campers riding BMX trails, zip lining through the woods, and performing big-air jumps


Get PSFK's Related Report: Future of Automotive

See All
Travel Yesterday

Boeing Wants Passengers To Control Their In-Flight Experience Through Their Phones

The airline manufacturer is embracing automation through a new generation of mobile travel apps

Augmented / Virtual Reality Yesterday

VR Surgery Videos Offer Interactive Medical Education

Dutch startup MDLinking hopes to globalize communication between students and medical care professionals with virtual reality content

Related Expert

Helen Warner

Surrealist Photographer

Culture Yesterday

Use Twitter To Learn A New Language

tDict is an app that uses the social media platform to help you search for words in local dialects

Mobile Yesterday

This Startup Wants To Digitize The Loose Coins In Your Pocket

CoinOut is a new app that lets you save your extra change from cash transactions as electronic funds

Advertising Yesterday

McDonald’s Is Accepting Trash As Currency In Exchange For Burgers

An initiative in Stockholm is trying to keep streets clean while satisfying hunger


Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry

PSFK Op-Ed september 21, 2016

Creative Agency Founder: Using VR As A Race Relationship Tool

Maurice Bernstein, CEO and Founder of Giant Step, explores the value in transforming headsets from high-tech entertainment tools into empathy machines

PSFK Labs september 22, 2016

The Future Of Work: Why Innovation Is Every Employee’s Job

PSFK Labs sits down with management at Johnson & Johnson to learn how the company comes up with their next ‘big idea’

Culture Yesterday

Google Is Using Virtual Paper Airplanes To Bring People Closer Together

The tech giant released an app that lets people throw their good wishes out into the world on the International Day of Peace

Advertising Yesterday

Get Paid For Traveling In San Francisco After Rush Hour

BART Perks rewards commuters who take early or late trains by giving them extra points to trade for money

Culture Yesterday

Artist Designs Covers For Books That Don’t Exist

Published by the fictional 'Specious Books,' the subversive works facilitate a conversation regarding the artistic integrity of graphic designers

Work Yesterday

Editorial Roundtable: The Arrival Of The People-First Workplace

Managed By Q, Soma, Workbar, Primary, AltSchool and thinkPARALLAX enumerate the reasons why companies need an employee-embracing workforce in order to exist

Food Yesterday

Bringing Food Innovation To America’s Crowded Milk Market

a2 Milk's Blake Waltrip, Chief Executive of the USA region, discusses how the distributor plans on bringing the popular drink for the dairy-sensitive to the States


Future Of Work
Cultivating The Next Generation Of Leaders

Financial Services Yesterday

Device Makes Digital Currency Feel Tangible

The concept gadget wants you to experience the highs and lows of spending money

Design & Architecture Yesterday

Why Building Better Offices Is The Key To Employee Engagement

Interaction Designer and Audio-visual Technologist at ESI Design illustrates the value in creating environments filled with surprise and delight

Augmented / Virtual Reality Yesterday

NBC Is Planning To Stream The Presidential Debates In Virtual Reality

Partnering with AltspaceVR, the broadcaster offers another way for Americans to engage with the election season

No search results found.