User:Francg/expub/thesis/prototype

Prototype

Extracting data (in this case I just scrap URL's / web links) from:
e.g. the website where I have my thesis outline stored: Thesis Outline

Run Python (I did it from virtual environment)
from bs4 import BeautifulSoup
import requests
url = raw_input("Enter a website to extract the URL's from: ")
r = requests.get("http://" +url)
data = r.text
soup = BeautifulSoup(data)
for link in soup.find_all('a'):

   print(link.get('href'))