Creating The Next Google Just Became More Affordable

Creating The Next Google Just Became More Affordable

The Common Crawl Foundation has made 5 billion indexed web pages readily available for free.

Yi Chen
  • 16 november 2011

The Common Crawl Foundation has indexed 5 billion web pages and has made the data readily available to anyone for free on the Amazon EC2/S3 cloud computing infrastructure. What this essentially means is that tech innovators looking to challenge Google and create the next best search engine can do so more easily, quicker and cheaper.

To access the information, users will need to setup their own Amazon EC2 Hadoop cluster and pay for the time they use it. There are no upfront costs to use the Amazon EC2 Hadoop and it charges cost per instance hour.

Common Crawl Foundation


Fitness Advocate: Paving The Future of Workouts With Audio

Fitness & Sport
Retail Today

Exchange Your Old Razors For New Ones At This Bartershop

The subscription-based shaving company lets customers trade their unwanted razors for Harry's brand steel at a temporary New York pop-up

Food Today

A Brewer Has Created The Most Expensive Chips To Snack On

St. Eriks Brewery created crisps made from rare mushrooms to go with its artisan beer, donating all proceeds to charity


Get PSFK's Related Report: Future of Automotive

See All
Travel Today

Melbourne Hotel Lets Guests Stay In Their Own Chrome Airstream Trailers

Notel is a luxury rooftop with six guest rooms made from vintage 1970s mobile homes

Related Expert

Robin Chase

Product As Service, Customer Service, Automotive

Sustainability Today

Swedish Citizens Get A Tax Break For Repairing Old Goods

A sustainability initiative encourages people to fix, rather than replace, broken objects

Fitness & Sport Today

How Precision Data Can Make Anyone A Better Performer

The Sports Debrief from PSFK Labs looks at how analytic tools are being developed to optimize human performance across all industries

Home Today

You Can Now Buy Furniture From A Daytime TV Show

Home furnishing online retailer Wayfair is partnering with Lifetime to create a shoppable life improvement television program


Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry

PSFK Op-Ed Today

Marketing Experts: Millennials And The Power Of Cool

'Good Is The New Cool' Authors Afdhel Aziz and Bobby Jones share their 7 principles for branding with a social impact

PSFK Labs Today

The Keys For Exceptional Performance On And Off The Field

PSFK Labs' new report highlights five important insights for businesses to perform better than the competition

Social Media Today

Instagram Tool Prevents People From Harming Themselves

A new anonymous reporting option on the social media platform hopes to provide better emotional support for users

Retail Today

Apple’s New London Store Is Filled With Living Trees And Sunlight

The brand's UK flagship location has been reimagined with a focus on customer experience and community

Syndicated Today

Artificial Intelligence Could Serve As A Judge In The Court Of The Future

A new software program can weigh up legal evidence and more questions of right and wrong to predict the outcome of trials

Brand Development Yesterday

Swipe Left On A Dating World Built To Keep You Single And Disconnected

Hinge's VP of Marketing Karen Fein tells us about the service's daring ditch of the swiping culture that's designed to attract advertising revenue, not meaningful connections

Food Yesterday

Brooklyn Cafe Lets Customers Pay By The Hour, Not By The Cup

Glasshour is an establishment that provides free coffee and pastries and charges for the time guests spend there

Technology Yesterday

Electric Spoon Changes The Way Food Tastes

The Taste Buddy is being developed to manipulate your taste buds and make everything more delicious

No search results found.