16-03-2011 Laura Amy Laurier

Simple Statistics

Legal terminology used in terms and conditions policies are often ambiguous and arbitrary. We want to highlight this ambiguity by showing the incongruity of definitions dependent on other factors often not implicitly explained. e.g the limits actually meant when the word 'unlimited' is used is paradoxical and often to give a false sense of comfort/security/complacency..

Word frequency distribution

from nltk import FreqDist
from matplotlib import *
import urllib2

t = "** 20.1 ** SITE shall not be responsible for any failure to perform due to unforeseen circumstances or to causes beyond our reasonable control, including but not limited to: acts of God, such as fire, flood, earthquakes, hurricanes, tropical storms or other natural disasters; war, riot, arson, embargoes, acts of civil or military authority, or terrorism; fiber cuts; strikes, or shortages in transportation, facilities, fuel, energy, labor or materials; failure of the telecommunications or information services infrastructure; hacking, SPAM, or any failure of a computer, server or software, including Y2K errors or omissions, for so long as such event continues to delay the SITE's performance.   "

words = t.split()

fdist = FreqDist(words)

voc = fdist.keys() 

print voc[:10]

#fdist.plot(50, cumulative=True)

from nltk import FreqDist
from matplotlib import *
import urllib2
from nltk.corpus import stopwords
n
english_stops = set(stopwords.words("english"))

 
t = "** 20.1 ** SITE shall not be responsible for any failure to perform due to unforeseen circumstances or to causes beyond our reasonable control, including but not limited to: acts of God, such as fire, flood, earthquakes, hurricanes, tropical storms or other natural disasters; war, riot, arson, embargoes, acts of civil or military authority, or terrorism; fiber cuts; strikes, or shortages in transportation, facilities, fuel, energy, labor or materials; failure of the telecommunications or information services infrastructure; hacking, SPAM, or any failure of a computer, server or software, including Y2K errors or omissions, for so long as such event continues to delay the SITE's performance.   "
 
words = t.split()
words = [w for w in words if w not in english_stops]
fdist = FreqDist(words)
 
voc = fdist.keys() 
 
print voc[:10]