menu

Creating The Next Google Just Became More Affordable

Creating The Next Google Just Became More Affordable
technology

The Common Crawl Foundation has made 5 billion indexed web pages readily available for free.

Yi Chen
  • 16 november 2011


The Common Crawl Foundation has indexed 5 billion web pages and has made the data readily available to anyone for free on the Amazon EC2/S3 cloud computing infrastructure. What this essentially means is that tech innovators looking to challenge Google and create the next best search engine can do so more easily, quicker and cheaper.

To access the information, users will need to setup their own Amazon EC2 Hadoop cluster and pay for the time they use it. There are no upfront costs to use the Amazon EC2 Hadoop and it charges cost per instance hour.

Common Crawl Foundation

technology
Trending

DIY Kit Lets You Build Your Own Wooden Bike, Boat Or Caravan

Design & Architecture
Culture Yesterday

Messaging Add-On Helps You Correct Your Friends’ Bad Grammar

An iMessage sticker pack will help you copyedit text messages

Automotive Yesterday

Mercedes-Benz Introduces A New Electric Mobility Brand

The separate entity aims to simplify the identification of Mercedes EV products to customers

Trending

Get PSFK's Related Report: Future of Automotive

See All
Mobile Yesterday

Tinder’s New Feature Makes Swiping A Group Effort

The dating app wants to democratize its gestural interaction by buying in to the social polling trend pervasive among millennials

Syndicated Yesterday

Autonomous Garbage Drone Prevents Trash From Reaching Deep Ocean

The solar-powered WasteShark collects refuse closer to the source: the harbor

Automotive Yesterday

Aston Martin Reveals Its Own Luxury Powerboat

The sleek AM37 echoes styling elements from the British brand's sports cars

Advertising Yesterday

An Escort Website Fights Violence Against Sex Workers

The advocacy campaign from McCann aims uncover the human toll of the exploitative industry

PSFK LABS REPORT

Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry
NEW

PSFK Op-Ed september 29, 2016

Digital Design Expert: Mobile First Is Dead, Think Mobile Native

Brian Cooper, chief creative officer of OLIVER Group UK, explains how some brands are still playing catch-up to new technology

PSFK Labs september 29, 2016

The 10 Steps To Discover, Hire, Develop Your Next Leader

PSFK's Future of Work report outlines key steps in the employee development path to empower next-gen leaders

Culture Yesterday

LIFE Magazine Relaunches In Pure VR

The general interest periodical, which ceased publication in 2000, has turned into a portal for virtual reality content

Mobile Yesterday

Reorder This Detox Drink With A Simple Text Message

Dirty Lemon is streamlining its communication by letting customers place orders, ask product questions and request help exclusively through chat

Op-Ed Yesterday

The Future Of The American Workforce Requires Unbundling College Education

President of JetBlue Technology Ventures: developing corporate education programs for non-traditional students

Retail Yesterday

Gilt’s Pop-Up House Is The Kind Of Store You’ll Want To Live In

The New York City townhouse plays host to the latest in retail inspiration, curation, and lifestyle activation (and some libations, too)

PSFK LABS REPORT

Future Of Work
Cultivating The Next Generation Of Leaders
NEW

Mobile Yesterday

Registering To Vote Is Now Just A Text Away

A new bot aims to mobilize underrepresented groups this election season through SMS and Facebook Messenger

Africa Yesterday

Virtual Reality Game Gives Lessons About Emergency Birth Care

LIFE is a serious tool that takes advantage of new technology to help save lives

Luxury Yesterday

Shoe Repair Has Moved Onto Your Phone

Cobbler Concierge is an on-demand service to get your footwear fixed online

No search results found.