menu

Creating The Next Google Just Became More Affordable

Creating The Next Google Just Became More Affordable
technology

The Common Crawl Foundation has made 5 billion indexed web pages readily available for free.

Yi Chen
  • 16 november 2011


The Common Crawl Foundation has indexed 5 billion web pages and has made the data readily available to anyone for free on the Amazon EC2/S3 cloud computing infrastructure. What this essentially means is that tech innovators looking to challenge Google and create the next best search engine can do so more easily, quicker and cheaper.

To access the information, users will need to setup their own Amazon EC2 Hadoop cluster and pay for the time they use it. There are no upfront costs to use the Amazon EC2 Hadoop and it charges cost per instance hour.

Common Crawl Foundation

+#technology
+amazon
+cloud computing
+Google
+Science
+search engine
+technology
+web
+Work & Business
Trending

Volvo Is Launching A Global Car-Sharing Business

Automotive
PSFK MEMBERSHIP


JOIN RETAIL INTELLIGENCE PLATFORM
Get access to retail reports and 20,000 retail insights


LEARN MORE

TREND REPORT


FORECAST 2020
The Consumer Roles That Will Shape Our Future
 

DOWNLOAD NOW

IoT january 19, 2017
Technology january 19, 2017
No search results found.