User:Tash/Prototyping 03
Workshop: CGI & TF-IDF search engine
#!//usr/local/bin/python3
import cgi
import cgitb; cgitb.enable()
import nltk
import re
print ("Content-type:text/html;charset=utf-8")
print ()
#cgi.print_environ()
f = cgi.FieldStorage()
submit1 = f.getvalue("submit1", "")
submit2 = f.getvalue("submit2", "")
text = f.getvalue("text", "")
### SORTING
import os
import csv
import string
import pandas as pd
import sys
### SEARCHING
#input keyword you want to search
keyword = text
print ("""<!DOCTYPE html>
<html>
<head>
<title>Search</title>
<meta charset="utf-8">
</head>
<body>
<p style='font-size: 20pt; font-family: Courier'>Search by keyword</p>
<form method="get">
<textarea name="text" style="background: yellow; font-size: 10pt; width: 370px; height: 28px;" autofocus></textarea>
<input type="submit" name="submit" value="Search" style='font-size: 9pt; height: 32px; vertical-align:top;'>
</form>
<p style='font-size: 9pt; font-family: Courier'>
webring <br>
<a href="http://145.24.204.185:8000/form.html">joca</a>
<a href="http://145.24.198.145:8000/form.html">alice</a>
<a href="http://145.24.246.69:8000/form.html">michael</a>
<a href="http://145.24.165.175:8000/form.html">ange</a>
<a href="http://145.24.254.39:8000/form.html">zalan</a>
</p>
</body>
</html>""")
x = 0
if text :
#read csv, and split on "," the line
csv_file = csv.reader(open('tfidf.csv', "r"), delimiter=",")
col_names = next(csv_file)
#loop through csv list
for row in csv_file:
#if current rows value is equal to input, print that row
if keyword == row[0] :
tfidf_list = list(zip(col_names, row))
del tfidf_list[0]
sorted_by_second = sorted(tfidf_list, key=lambda x:float(x[1]), reverse=True)
print ("<p></p>")
print ("--------------------------------------------------------------------------------------")
print ("<p style='font-size: 20pt; font-family: Courier'>Results</p>")
for item in sorted_by_second:
x = x+1
print ("--------------------------------------------------------------------------------------")
print ("<br></br>")
print(x, item)
n = item[0]
f = open("cgi-bin/texts/{}".format(n), "r")
sents = nltk.sent_tokenize(f.read())
for sentence in sents:
if re.search(r'\b({})\b'.format(text), sentence):
print ("<br></br>")
print(sentence)
f.close()
print ("<br></br>")
Workshop with Marcell Mars
On tunneling: contexts: censorship (e.g. of nation states like China, Turkey, Iran or of institutions and academia), anonymity, organizing resistance or political action rules: physical location (as far as the wifi goes),
AP: access point (for wireless routers for example) router: a networking device that forwards data packets between computer networks. Routers perform the traffic directing functions on the Internet. encryption certificates: used by websites to enable secure HTTPS connections, issued to domain and subdomain. Issued by authorities like Let’s Encrypt (recently free) and DigiCert. DNS: domain name server or system, which resolves and distributes IP adresses, and lets you get to the domain. Usually set to automatic DHCP, but you can manually choose your own conversion point, like those served by Google (8.8.8.8) ping: command line tool to send a quick byte of info to check if a domain is alive network interface: the device / card (e.g. wireless or ethernet) through which your computer is talking to the internet. IP addresses are assigned to the network interface
> so to avoid network admins from seeing your DNS requests (and tracking domain and subdomains) you can ‘tunnel’ and use things like an encrypted DNS server, or proxy servers
When talking about networks: in Unix philosophy, everything is a file, with paths which you can read and write into. Networks are streaming media, so here things become more complex. Here, ports are the sockets through which you can make connections. Over time, default conventions have been assigned – like 22 for SSH and 443 for HTTPS.
Repositories: https://gitlab.com/marcellmars/letssharebooks https://github.com/marcellmars/logan_and_jessica
Exercise: https://imgur.com/a/xuUuN https://rsync.samba.org/
Interesting projects: https://beakerbrowser.com/ https://ipfs.io/ https://zerotier.com/