Roll your own google

Google dominates contemporary access to the Internet, becoming virtually synonymous with search, online video, and through Android increasingly mobile.

BACK in the early daze, net sites were sparse isolated islands of, tethered together with webrings and a patchwork of amateur link lists and proto-portals. This exercise is at once a simple exercise in CGI scripting and an earnest effort to take back the web. Restriction: all data that your cgi uses must be local to the server -- meaning your "results" will be purely algorithmic, and/or based only on input provided to it (via the search box), and/or using either collected or crawled data you've yourself gathered. It's only fair, that's how Google works too.

CGI

• Start with a simple form (HTTP/Post/Submit!)
• Can respond with ANY type (image/audio/...)
• Respond to browser (with audio?)

Links

• Eliza and Weizenbaum's clever text and response (chatbot as a kind of search engine)
• Scrapy

Creating an index

• How does an algorithm "see" a text, a sound, a video, a webpage
• Beautiful Soup
• Lucene... intro, tokenizer
• Crawling
• Danya Vasiliev's Puppet piece ?!