How Sesame Street Is Re-imagining A Show’s Storyline For Kinect-Era Television [PSFK LONDON]

How Sesame Street Is Re-imagining A Show’s Storyline For Kinect-Era Television [PSFK LONDON]

The Creative Director of the Content Innovation Lab at Sesame Workshop talks to PSFK about the future of educational programming centered on interactivity.

Timothy Ryan, PSFK Labs
  • 8 september 2012

We are excited to have J Milligan, Creative Director of the Content Innovation Lab for Sesame Workshop, come speak at PSFK CONFERENCE LONDON. Sesame Workshop has recently partnered with Microsoft to leverage the Kinect for enhanced educational programming centered on interactivity. On September 13th, Jason will discuss reimagining a show’s storyline for Kinect-era television.

Sesame Workshop has been experimenting with digital learning materials for some time. What are the challenges around designing interactive programming for kids?

If you’re interested in teaching literacy, numeracy, rhymes or anything else, you will always be fighting the platform. The first set of instructions is always about moving the cursor around, picking up the mouse, or just trying to deal with basic interface issues before we can even teach anything.

The first thing we always have to teach is how to use the platform, but that’s been falling away more and more lately. First, going to touch, which is such an intuitive thing for kids. Many children can already navigate an iPhone at a very young age. Touch is intuitive, and Kinect is even better because it’s just about moving your body and using your voice to make things happen on screen.

I’ve been designing digital curricula for a while and it has always been a challenge to design for an age group appropriately. Whenever you hear people talk about design, they talk about designing for the extreme user; if you design for the extreme instead of the middle, the logic goes, everything will work itself out. Designing for preschoolers is definitely one end of the extreme.

Recently you’ve partnered with Microsoft to develop programming leveraging the Kinect platform. What is the opportunity?

When Kinect came along we had a viable platform available at scale that millions of people already had in their homes. Bringing video to the platform and making it react to the player and what they’re actually doing finally brought together some of our thinking around creating engaging educational material in a significant and incredibly interesting way.

Sesame Workshop had always been interested in using television as a tool to teach. If a platform can react to a viewer or player in a meaningful way, it makes an experience that much more personal. We’d been looking at ways to do this for a long time: back in the 1980s we did a project with Viewmaster using VHS to build an interactive TV experience, which is kind of incredible. We did a big workshop with AFI in 2002 to use a DVD platform to create an interactive experience, and we also did some work with CableVision using the remote. At that time, a lot of people who were working the space were doing what I call ‘science experiments;’ you could make interactive TV, but you had to build your own hardware and software to make anything happen.

What specifically with respect to the Kinect is so engaging and intuitive for Sesame Workshop’s audience?

Sesame is all about making the whole thing fun and entertaining, and ‘engagement’ is exactly the right way to describe it. The more you’re engaged in the content, the more you’ll get out of the learning materials we’re presenting. For example, in the case of the Kinect-enabled game Letter Tree, what we’re really teaching you is that ‘A’ is the first letter sound. All we’re asking you to do is jump, and that’s something kids are very willing to do.

We’re focused on embedding interactivity into the narratives we’re creating, where the things we’re asking the kid to do just feel natural and organic. Now we’re able to bring that experience in to your living room. When Elmo throws you a ball through the glass of the TV, you throw it back to him and he catches it. When Cookie Monster says, “I need your help, there’s something stuck in this tree that I want to eat, jump to shake this tree,” the viewer will start jumping. It’s just incredibly natural for a kid to make a throwing motion or start jumping. We’ve seen this in testing, and I’ve seen this in my own home – it’s great.

When it comes to Kinect, I would definitely say we’re still figuring it out, but currently we have eight shows coming out; eight episodes coming out around the time of the conference and then another eight on Christmas day.

Have the goals of Sesame Workshop changed at all over the years?

Our mission is to reach kids on every available platform, and give them developmentally-appropriate, educational content to get them ready for school and life. Sesame Workshop, as you may know, is going into our 43rd season of production this fall. It was a show that Joan Ganz Cooney and Lloyd Morisett invented in the late ’60s (with help from Jim Henson and the Muppets) to harness the power of television to get kids excited and prepared for school.

What is your role in creating those platforms?

I’m the creative director of the Content Innovation Lab, a small group of six people who look three to five years out to determine what platforms are going to be relevant and what Sesame should be working on. We build capacity for the company as a whole on things like gesture and voice and my role has always been on the digital side. I’ve been designing interactive experiences for (mostly) preschool kids for almost 17 years.  I’ve designed on every major platform to come along since the mid-1990s: things like PC CD‑ROM to web, consoles like PS1, handheld, Gameboy and then lately moving to mobile apps, touch‑screen iPad, iPod, iPhone and now Kinect. Back in the day, Sesame Workshop was doing software for things like the Apple II, and so the digital side of things goes back almost as far as the television show. Now we’re bringing it all together. The project that I’m bringing to the PSFK London Conference, “Connect Sesame Street,” is basically interactive TV on the Xbox console.

Thanks Jason!

Jason Milligan/ @jmilligan718/ Sesame Workshop

Come see Jason discuss re-imagining a show’s storyline for Kinect-era television at PSFK CONFERENCE LONDON on September 13th.


IKEA Is Letting Kids Design Its New Line Of Toys

Travel Yesterday

30-Year-Old Photographs Used As Travel Guides

A new photo series revolves around tracing the origins of images from the past

Technology Yesterday

Album Turns Into Something New Each Time It’s Streamed

Bill Baird's new album explores the relationship between time and music through a website crafted by design team, One Pixel Wide


Get PSFK's Related Report: Future of Automotive

See All
Health Yesterday

VR App Prescribed For Pain Relief

A pharmacy chain in Sweden is stepping away from tradition to develop a happy place for the pain-afflicted

Retail Yesterday

Banks Are Coming Together To Create A New Payment Network That Rivals Venmo

A number of financial institutions are collaborating to make a new person-to-person monetary system called Zelle for their customers

Related Expert

Josh Kushner

Health Insurance, Private Equity

Media & Publishing Yesterday

Pocket Camera Aims To Facilitate The Struggles Of Live Streams

The Mevo helps resolve the complexities of streaming video with an intuitive setup and smart editing controls

Food Yesterday

Startup Believes Traceability Will Help Disrupt The Multivitamin Industry

Ritual is a daily supplement for women that traces every ingredient back to its source

Food Yesterday

Photo Series Brutally Murders Some Of Your Favorite Fast Food

The portraits by artist duo Ilka & Franz do away with mealtime regulars in a way that is both beautiful and humorous


Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry

PSFK Op-Ed Yesterday

General Electric: Lighting’s Impact On Sleep Is More Than The Off Switch

Jeff Patton, General Manager of Connected Home Products at GE Lighting, uncovers how lighting technologies can affect our sleep cycles

PSFK Labs october 25, 2016

The Keys For Exceptional Performance On And Off The Field

PSFK Labs' new report highlights five important insights for businesses to perform better than the competition

Mobile Yesterday

Coffeemaker Teaches You How To Make The Perfect Cup

The device comes with an accompanying app that guides novices and experts alike through the brewing process

Brand Development Yesterday

The Story Behind How LYNK & CO Created A Car Brand From Scratch

Head of Design Andreas Nilsson describes which values were most influential in determining the identity and design direction of the new auto company

Travel Yesterday

Architect’s Design Presents A Radically New Approach For New York’s Penn Station

The firm of Vishaan Chakrabarti has envisioned a bright community and travel hub in the heart of the city

Fitness & Sport Yesterday

Editorial Roundtable: Building An All-Encompassing Performance Suite

WHOOP, ShotTracker, Rithmio, PlaySight, STYR Labs, EverybodyFights and Lift / Next Level Floats on the partnership opportunities available in health and fitness

Gaming & Play Yesterday

Fantasy Game Responds To Each Player’s Emotions

The card battling venture measures responses through a Bluetooth clip to adjust the experience accordingly

Luxury Yesterday

Carry A Map Of NYC On A Handbag

The bag from Bottega Veneta has been designed exclusively for Bergdorf Goodman to celebrate New York City

Technology Yesterday

Roaming Robots Crawl Around Your Body To Do Small Jobs As You Go About Your Day

A new concept wearable developed by researchers at MIT and Stanford are fully functional bots that live on your clothing

Retail Yesterday

Runners Are Delivering Groceries To Parisians

French supermarket Carrefour partnered with a running group for fast and memorable deliveries

No search results found.