NLTK text analysis

Natural Language Tool Kit_141020_Michael

url = "https://git.xpub.nl/XPUB/S13-Words-for-the-Future-notebooks/raw/branch/master/txt/words-for-the-future/UNDECIDABILITY.txt" url 'https://git.xpub.nl/XPUB/S13-Words-for-the-Future-notebooks/raw/branch/master/txt/words-for-the-future/UNDECIDABILITY.txt' from urllib.request import urlopen r = urlopen(url) r <http.client.HTTPResponse at 0x7f2f424be6a0> rawtext = r.read() text = rawtext.decode() type(text) str text = urlopen(url).read().decode() len(text) 15990 text[-1] '\r' text[0] 'U' words = text.split() len(words) 2548 words[0] 'Undecidability' words[3] 'Multiplying' words[-1] '158-172.' from nltk import word_tokenize, Text tokens = word_tokenize(text) len(tokens) 3047 tokens[-1] '.' tokens[:10] ['Undecidability',

'Silvia',
'Bottiroli',
'Multiplying',
'the',
'Visible',
'The',
'word',
'[',
'i']

tokens[21:30] # not including 30th word ['Memos', 'for', 'the', 'Next', 'Millennium', '[', 'i', ']', 'written'] strengers = Text(tokens) strengers <Text: Silvia Bottiroli Multiplying the Visible The word [ i> strengers.concordance("multiplicity", width = 84, lines = 72) Displaying 11 of 11 matches:

] attempts to escape the vortex of multiplicity are useless. ” [ 6 ] In his fifth m
, he subsequently focuses on [ i ] multiplicity [ i ] as a way for literature to co

fore , let ’ s think visibility and multiplicity together , as : a multiplication of n the contrary , it is generating a multiplicity of different gazes that are all leg ed and thus incomplete and open . A Multiplicity of Gazes An undecidable artwork is ics today , is that they generate a multiplicity of gazes and of forms of spectators

positions and points of view . The multiplicity of gazes produced and gathered by u

tes a radical collectivity based on multiplicity and on conflicting positions that a ility and from its encounter with a multiplicity of gazes . Preserving it is possibl encounter between undecidable art , multiplicity of gazes , and a curatorial dimensi

ibid , p. 98 . 7 . Italo Calvino , Multiplicity , [ i ] Six Memos for the Next Mill

for line in strengers.concordance_list("the", width=82, lines=74):

   print (line.left_print, line.query, line.right_print)

idability Silvia Bottiroli Multiplying the Visible The word [ i ] undecidable [ i lvia Bottiroli Multiplying the Visible The word [ i ] undecidable [ i ] appears i e [ i ] appears in [ i ] Six Memos for the Next Millennium [ i ] written by Italo ry lectures at Harvard University . In the last months of his life Calvino worked rishly on these lectures , but died in the process . In the five memos he left be ectures , but died in the process . In the five memos he left behind , he did not i ] Visibility [ i ] , revolves around the capacity of literature to generate ima flow continuously . Calvino focuses on the imagination as “ the repertory of what alvino focuses on the imagination as “ the repertory of what is potential ; what

exist but might have existed. ” [ 2 ] The main concern that he brings forth lies

ncern that he brings forth lies within the relation between contemporary culture contemporary culture and imagination : the risk to definitely lose , in the overp ion : the risk to definitely lose , in the overproduction of images , the power o se , in the overproduction of images , the power of bringing visions into focus w g [ i ] in terms of images. ” [ 3 ] In the last pages of the lecture , he propose f images. ” [ 3 ] In the last pages of the lecture , he proposes a shift from und he proposes a shift from understanding the fantastic world of the artist , not as m understanding the fantastic world of the artist , not as indefinable , but as [ th this word , Calvino means to define the coexistence and the relation , within no means to define the coexistence and the relation , within any literary work , , between three different dimensions . The first dimension is the artist ’ s imag nt dimensions . The first dimension is the artist ’ s imagination – a world of po at no work will succeed in realizing . The second is the reality as we experience l succeed in realizing . The second is the reality as we experience it by living we experience it by living . Finally , the third is the world of the actual work

it by living . Finally , the third is the world of the actual work , made by the
. Finally , the third is the world of the actual work , made by the layers of si

the world of the actual work , made by the layers of signs that accumulate in it ns that accumulate in it ; compared to the first two worlds , it is “ also infini ctory to formulation. ” [ 4 ] He calls the link between these three worlds “ the

the link between these three worlds “ the undecidable , the paradox of an infini

these three worlds “ the undecidable , the paradox of an infinite whole that cont ino , artistic operations involve , by the means of the infinity of linguistic po c operations involve , by the means of the infinity of linguistic possibilities , infinity of linguistic possibilities , the infinity of the artist ’ s imagination uistic possibilities , the infinity of the artist ’ s imagination , and the infin ty of the artist ’ s imagination , and the infinity of contingencies . Therefore ity of contingencies . Therefore , “ [ the ] attempts to escape the vortex of mul erefore , “ [ the ] attempts to escape the vortex of multiplicity are useless. ”

as a way for literature to comprehend the complex nature of the world that for t

re to comprehend the complex nature of the world that for the author is a whole o e complex nature of the world that for the author is a whole of wholes , where th he author is a whole of wholes , where the acts of watching and knowing also inte watching and knowing also intervene in the observed reality and alter it . Calvin are readable as different narratives . The lecture revolves around some novels th ain multiple worlds and make space for the readers ’ imaginations . The common so space for the readers ’ imaginations . The common source to all these experiments all these experiments seems to rely in the understanding of the contemporary nove

seems to rely in the understanding of the contemporary novel “ as an encyclopedi
, as a network of connections between the events , the people , and the things o

rk of connections between the events , the people , and the things of the world.

between the events , the people , and the things of the world. ” [ 7 ] Therefore

vents , the people , and the things of the world. ” [ 7 ] Therefore , let ’ s thi ic production and define a context for the undecidable , or rather for undecidabi le , or rather for undecidability , as the quality of being undecidable . Calvino tion modes and doesn ’ t fade out from the scene of the ‘ real ’ world . We might d doesn ’ t fade out from the scene of the ‘ real ’ world . We might stretch this

s potentiality is that of multiplying the visible as an actual counterstrategy t

isible as an actual counterstrategy to the proliferation of images that surrounds ly articulates , redefines , or alters the complex system of links , bounds , and specific to some artworks within which the three worlds that Calvino describes me tains and under certain terms performs the possibility of its actualisation , a w into one actual form . In particular , the potentiality generated by undecidable c of ‘ and… and… and… ’ as opposite to the logic of ‘ either… or… ’ that seems to ature and just exist as such . None of the images of an artwork are being more or twork are being more or less real than the others , no matter whether they come a vidual or collective fantasies . It is the art ( work ) as such that creates a gr s such that creates a ground where all the images that come into visibility share images that come into visibility share the same gradient of reality , no matter w itors or spectators to enter into – if the invitation of art is often that of los itation of art is often that of losing the contact with known worlds in order to Here , spectators are invited to enter the work ’ s fictional world carrying with ctional world carrying with themselves the so-called real world and all their oth ll these worlds are equally welcomed . The artwork may then be navigated either b for w in strengers:

   if w.endswith("ity"):
       print (w) # but then this will show overlapping, looping..

Undecidability University visibility Visibility capacity reality infinity infinity infinity multiplicity multiplicity reality visibility multiplicity undecidability quality potentiality visibility undecidability undecidability quality possibility potentiality potentiality reality reality visibility reality undecidability reality contemporaneity possibility possibility possibility undecidability community possibility multiplicity Multiplicity multiplicity multiplicity community collectivity multiplicity reality responsibility undecidability potentiality undecidability collectivity visibility Undecidability possibility potentiality quality undecidability multiplicity intensity multiplicity Visibility University Multiplicity University

1. and now collected in a list, and squashing case, and using a "set" to remove dupliates.

ity = [] for w in strengers :

   if w.endswith("ity"):
       #print(w)
       ity.append(w.lower())
       #strengers.concordance()

ity = set(ity) with open("nami_undecidibility_Michael_NLTK_141020.text", "w") as output:

   s = 0

   for word in ity:
       #strengers.concordance(word, width = 84)
       for line in strengers.concordance_list(word, width=82, lines=74):
           t = line.left_print + " " * (2 + int(s)) + line.query + " " * (2 + int(s)) + line.right_print 
           #print(s)
           print (t[:82], file = output)#0-82 limited
           s = s + 0.3
  
       
       
       

for w in strengers:

   if w.endswith("le"):
       print (w) # but then this will show overlapping, looping..

1. and now collected in a list, and squashing case, and using a set to remove dupliates

le = [] for w in strengers :

   if w.endswith("le"):
       #print(w)
       le.append(w.lower())
       #strengers.concordance()

le = set(le) for word in le:

       strengers.concordance(word, width = 84)

strengers.concordance("multiplicity", width = 50) Displaying 11 of 11 matches: cape the vortex of multiplicity are useless. ” [ 6 y focuses on [ i ] multiplicity [ i ] as a way for ink visibility and multiplicity together , as : a it is generating a multiplicity of different gazes plete and open . A Multiplicity of Gazes An undeci at they generate a multiplicity of gazes and of fo ints of view . The multiplicity of gazes produced lectivity based on multiplicity and on conflicting s encounter with a multiplicity of gazes . Preserv

undecidable art , multiplicity of gazes , and a c
. Italo Calvino , Multiplicity , [ i ] Six Memos 

strengers.similar("multiplicity") kind world logic space practice undecidable visibility capacity images and repertory overproduction power thinking understanding means coexistence layers paradox whole strengers.common_contexts(["undecidability", "multiplicity"]) strengers.dispersion_plot(["the", "multiplicity", "performance"]) from IPython.core.pylabtools import figsize fizsize(20.0, 20.0) #make the graph more longe from nltk.probability import FreqDist freq = FreqDist(tokens) # frequency distribution freq.keys() freq["the"] freq.plot() freq.plot(50, cumulative = True) freq.plot(30)