How to Find Old Sites That Google Won’t Show
The Internet as a “place” is constantly growing. We create more and more web pages every day – in fact, so many that it may seem like some corners of them have been lost in time.
It turns out they can only be lost to Google. Earlier this year, web developer bloggers Tim Bray and Marco Fioretti noted that Google appears to have completely stopped indexing the web for Google searches. As a result, some older websites that are more than 10 years old do not appear in Google search. Both authors lamented that the limitation of Google’s effective memory over the past decade, while logical when faced with the daunting task of playing information concierge for each of our bizarre questions, forces us to reckon with the fact that when you use Google for historical search, there is maybe more answers there.
According to a BoingBoing post based on Bray’s comments , DuckDuckGo and Bing continue to offer more complete records about the Internet, in particular showing web pages that Google has stopped indexing for search. If you’re looking for a specific website before 2009 and can’t find one, either one is a good first step. If that doesn’t work, it’s always possible that someone else who needed the same page you were looking for has archived it on the Wayback Machine .
But what about general questions? Questions you still don’t know the answer to? Historical research from the early web? There are other, more specialized options for this. The Hacker News forum post suggests a couple of search engines. Million Short , which allows you to trigger searches and automatically skip the most popular answers to get deeper into the Internet. Wiby.me is a “classic website search engine” designed to help people find hobbyist pages and other archaic web features.
Hacker’s news feed also mentions Pinboard , a minimalist Pocket-like bookmarking service that has a key feature for archivists: if you subscribe to its premium service – $ 11 a year – Pinboard will create a web archive of every page you save. If you are viewing older, non-indexed content, such a tool can make it easier to get back to certain parts of the old Internet that you might want or need to remember again.