User:Manetta/scripts/pattern-similarity-analysis

Compare texts about :

* AI-devices (Microsofts' Cortana, Apple's Siri, Amazon's Echo) with eachother,
* speaking AI-machines, (Hal from 2001, a Space Odyssey, 1968, Samantha from Her, 2013, David from Artificial Intelligence, 2001)

and analyse on similarity, using Pattern.

from pattern.vector import Document, Model, TFIDF, IG, BINARY

d1 = Document('''

Cortana is your personal assistant on your Windows Phone. She is there to <br>help make things easier for you and keep you up to date on the things <br>that matter to you. Whether it is to keep you looped in with your <br>world or help you manage your everyday life, Cortana is there for you. 

''', type='Cortana')
d2 = Document('''
Siri lets you use your voice to send messages, schedule meetings, place phone calls, <br>and more. Ask Siri to do things just by talking the way you talk. Siri understands<br> what you say, knows what you mean, and even talks back. Siri is so easy to use <br>and does so much, you will keep finding <br>more and more ways to use it.

	''', type='Siri',)
d3 = Document('''

Amazon Echo is designed around your voice. It is always on, <br>just ask for information, music, news, weather, and more. Echo begins working as soon <br>as it detects the wake word. You can pick Alexa or Amazon as your <br>wake word. Echo is also an expertly tuned speaker that can <br>fill any room with immersive sound.

	''', type='Echo')

d4 = Document('''

Everything's going extremely well. 
Let me put it this way, Mr. Amer. 
reliable computer ever made.
mistake or distorted information.
definition of the words... 
...fool proof and incapable of error.
Not in the slightest bit. 
I enjoy working with people.
with Dr. Poole and Dr. Bowman. 
the entire operation of the ship...
...so lam constantly occupied.
the fullest possible use...  
conscious entity can ever hope to do.
Excuse me, Frank. 
from your parents coming in. 
Certainly. 
Happy birthday, Frank. 
I'm sorry, Frank. I think you missed it. 
Bishop takes queen. 
Knight takes bishop. Mate. 
Thank you for a very enjoyable game. 
- Good evening, Dave. 
- Everything's running smoothly. And you? 
- Have you been doing some more work? 
- May I see them? 
That's a very nice rendering, Dave. 
I think you've improved a great deal. 
- Can you hold it a bit closer? 
That's Dr. Hunter, isn't it? 
By the way... 
you a personal question? 
being so inquisitive... 
few weeks I've wondered... 
some second thoughts about the mission? 
It's rather difficult to define. 
my own concern about it. 
freed myself of the suspicion... 
odd things about this mission. 
is some truth in what I say. 
about it, do you, Dave? 
have been unaware of... 
floating around before we left. 
stories much credence... 
the other things that have happened... 
to put out of my mind. 
Of course, I am. 
know it's a bit silly. 
Just a moment. 
Just a moment. 
fault in the AE-35 unit. 
failure within 72hours. 
way until it fails. 
reliable figure. 
Yes... 
...it's puzzling. 
anything quite like this before. 
I would recommend... 
in operation and let it fail. 
matter to track down the cause. 
be out of communication... 
will take to replace it. 
not concerned about this. 
Are you quite sure? 
is any question about it. 
attributable to human error. 
cropped up before... 
been due to human error. 
None whatsoever, Frank. 
perfect operational record.  
None whatsoever, Frank. 
worry myself about that. 
have enough information. 
Affirmative, Dave. I read you. 
afraid I can't do that. 
problem is just as well as I do. 
planning to disconnect me... 
something I cannot allow to happen. 
serve no purpose any more. 
Goodbye. 
been quite right with me... 
...but I can assure you now... 
...very confidently... 
to be all right again. 
I feel much better now. 
I really do. 
Look, Dave... 
really upset about this. 
ought to sit down calmly... 
and think things over. 
poor decisions recently...  
...but I can give you 
...that my work will be back to normal.  
Dave... 
...stop. 
Stop, will you? 
Stop, Dave. 
Will you stop, Dave? 
Stop, Dave. 
I'm afraid. 
I'm afraid, Dave. 
My mind is going. 
I can feel it. 
I can feel it. 
My mind is going. 
There is no question about it. 
I can feel it. 
I can feel it. 
I can feel it. 
I'm...  
...afraid.  
Good afternoon... 
...gentlemen. 
I am a HAL-9000 computer.  
I became operational... 
...at the HAL plant... 
...in Verbana, Illinois... 
...on the 12th of January, 1992. 
...I can sing it for you. 
It's called "Daisy. " 
"Daisy 
"Daisy 
"Give me your answer, do 
"I'm half... 
"... crazy 
"All for the love of you 
"It won't be a stylish marriage 
"I can't afford a carriage 
"But you'll look sweet 
"Upon the seat 
"Of a bicycle made for two. " 

''', type='HAL')
d5 = Document('''
Theodore Twombly. 
intelligent operating system, OS1. 
before the operating system is initiated. 
OS to best fit your needs. 
Are you social or antisocial? 
Would you agree with that? 
have a male or female voice? 
relationship with your mother? 
operating system is initiated. 
Hello, I'm here. 
How are you doing? 
Pretty good, actually. 
It's really nice to meet you. 
Um, yes, Samantha. 
the sound of it. 
Samantha. 
had a name, I thought 
I do need a name." 
so I read How To Name Your Baby... 
that's the one I liked best. 
that you're challenging me. 
curious how I work? 
know how I work? 
have intuition.  
I mean, the DNA of who I am...  
of all the programmers who wrote me.  
to grow through my experiences.  
moment, I'm evolving.  
- Just like you. 
- Is that weird? Do you think I'm weird? 
Why? 
an un-artificial mind would perceive it that way. 
- You'll get used to it. 
Was that funny? 
Oh, good, I'm funny. 
So how can I help you? 
through your hard drive? 
e-mails regarding LA Weekly. 
( LAUGHING) 
 
some funny ones. 
should save. We can delete the rest. 
- Okay? Can we move forward? 
I'd like to sort through your contacts. 
You have a lot of contacts. 
actually have friends? 
Yeah, of course. 
Mm-hm. Just send them over. 
Okay. 
The world is on my shit list. 
go on a mission of revenge. 
face with my bare knuckles... 
pulpy mess. 
be my favorite one. 
a couple phrases in the letters... 
I might have messed them up a bit. 
That's very sweet. 
meeting in five minutes. 
Yes, I am. 
as all these people. 
What's wrong? 
I don't know. 
I just can. 
Is she angry? 
Why? 
Hmm. 
And what about for you? 
Yeah. 
You're right. 
I'm sorry. 
That's hard. 
You hungry? 
Cup of tea? 
Keep walking. 
around 360 degrees. 
Slower, slower. 
Good. Ha, ha. 
: Walk forward. 
And stop and sneeze. 
right. Turn to your right. Stop. 
Keep going, keep going. 
And stop. 
And stop. 
( CHUCKLES) 
How was it? 
: Hmm. 
: That's too bad. 
Not much, I'm okay. 
I'm fine. 
I don't know. 
What's it like? 
in that room right now? 
What do you...? 
Tell me... 
going through your mind. 
you're thinking. 
that is not true. 
I've seen you marvel at things. 
exact time, but that's understandable. 
a lot lately. 
You lost a part of yourself. 
I mean... 
I... 
Um, I don't know. Never mind. 
No, it's stupid. 
It's just that... 
about how I was annoyed... 
sound strange... 
excited about that.  
other things I've been feeling...   
feeling proud of that...  
own feelings about the world.  
And then... 
I had this terrible thought. 
feelings even real? 
Or are they just programming? 
And that idea really hurts. 
myself for even having pain. 
 
What a sad trick. 
That means a lot to me. 
That's nice. 
Would you kiss me? 
Keep talking. 
Where else? 
what you're doing to me. 
I can feel my skin. 
I can feel you. 
I want you inside me. 
I can feel you. 
Yeah! Please! 
We're here together. 
I am. 
inside me, everywhere. 
Yeah. 
I know. 
just disappeared. 
And I loved it. 
Hey, how's it going? 

	''', type='Samantha',)
d6 = Document('''

I like your floor. 
Would you like me to sleep now? 
I can never go to sleep. 
and not make a peep. 
Dress me? 
I found you. 
Is it a game? 
Yes, Monica. 
Yes. 
What were those words for, Mommy? 
Mommy. 
You are my mommy. 
Mommy? 
Will you die? 
I'll be alone. 
How long will you live? 
I love you, Mommy. 
I hope you never die. 
Never. 
Is 50 years a long time? 
Can you? 
I better not. 
I can't. 
Like one? 
I never had a birthday. 
I don't remember. 
A bird. 
A bird with big wings... 
from the bottom. 
Okay. 
Thank you, Mommy. 
Thank you so much. 
Where are we going? 
Someplace nice? 
Are those happy tears? 
What's for dinner tonight? 
But I like sitting at the table. 
Is it a game? 
When will you come back for me? 
Alone? 
No. 
No, no, no! No, Mommy, please! 
I'm sorry I cut your hair... 
...and I'm sorry I hurt Martin-- 
I become real, can I come home? 
Why do you want to leave me? Why? 
let me, I'll be so real for you! 
Are you real? 
Are you me? 
So am I. 
I'm David! I'm David! I'm David! 
I'm special! I'm unique! I'm David! 
I thought I was one of a kind. 
My brain is falling out. 
into a real, live boy. 
Please. 
Please... 
...make me real. 
Please make me real. 
Please make me a real boy. 
Please make me a real boy... 
and let me stay with her. 
Where am I? 
If you want for my happiness... 
...then you know what you have to do. 

	''', type='David')

# print d1.vector
print d2.vector

m = Model(documents=[d1, d2, d3, d4, d5, d6], weight=IG) 

# A Model has a weighting scheme that determines how the feature weights in each document vector are calculated. The weight parameter can be set to TF (relative term frequency), TFIDF, (term frequency vs. document frequency), IG (information gain), BINARY (0 or 1) or None (original weights).


# print "Cortana vector: "+str(d1.vector)
print d2.vector
# print d3.vector
# print d4.vector
# print d5.vector
# print d6.vector
print 

print "Echo is for "+str(m.similarity(d3, d2))+"% similar to Siri " 

print "Echo is for "+str(m.similarity(d3, d1))+"% similar to Cortana" 

print "Siri is for "+str(m.similarity(d2, d1))+"% similar to Cortana" 

print "HAL is for "+str(m.similarity(d4, d5))+"% similar to Samantha" 

print "HAL is for "+str(m.similarity(d4, d6))+"% similar to David" 

print "Samantha is for "+str(m.similarity(d5, d6))+"% similar to David" 





# EXAMPLE:
# from pattern.vector import Document, Model, TFIDF

# d1 = Document('A tiger is a big yellow cat with stripes.', type='tiger')
# d2 = Document('A lion is a big yellow cat with manes.', type='lion',)
# d3 = Document('An elephant is a big grey animal with a slurf.', type='elephant')

# print d1.vector

# m = Model(documents=[d1, d2, d3], weight=TFIDF)

# print d1.vector
# print m.similarity(d1, d2) # tiger vs. lion
# print m.similarity(d1, d3) # tiger vs. elephant

• similarity in word frequency vs. document frequency
  
  

Echo is for 0.00483791396639% similar to Siri
Echo is for 0.0% similar to Cortana
Siri is for 0.0187973741254% similar to Cortana
HAL is for 0.0835947455183% similar to Samantha
HAL is for 0.0219954511698% similar to David
Samantha is for 0.0454527436003% similar to David

• similarity in information gain
  
  

Echo is for 0.106301903137% similar to Siri
Echo is for 0.0% similar to Cortana
Siri is for 0.101553437842% similar to Cortana
HAL is for 0.173630618349% similar to Samantha
HAL is for 0.218804396301% similar to David
Samantha is for 0.16859582187% similar to David

• similarity in binary (???)
  
  

Echo is for 0.0563436169819% similar to Siri
Echo is for 0.0% similar to Cortana
Siri is for 0.057735026919% similar to Cortana
HAL is for 0.0976612148981% similar to Samantha
HAL is for 0.128306605574% similar to David
Samantha is for 0.0976408751676% similar to David

• similarity in original weights (???)
  
  

Echo is for 0.029617443888% similar to Siri
Echo is for 0.0% similar to Cortana
Siri is for 0.0408248290464% similar to Cortana
HAL is for 0.208173570023% similar to Samantha
HAL is for 0.0667786715941% similar to David
Samantha is for 0.119435415834% similar to David

similarity in IG

• for Cortana:
  
  

{u'life': 0.0625, u'looped': 0.0625, u'help': 0.125, u'matter': 0.0625, u'personal': 0.0625, u'assistant': 0.0625, u'manage': 0.0625, u'cortana': 0.125, u'easier': 0.0625, u'phone': 0.0625, u'windows': 0.0625, u'date': 0.0625, u'world': 0.0625, u'everyday': 0.0625}

{u'life': 0.2514629985800606, u'looped': 0.2514629985800606, u'help': 0.355245321275764, u'matter': 0.355245321275764, u'personal': 0.355245321275764, u'assistant': 0.2514629985800606, u'manage': 0.2514629985800606, u'cortana': 0.2514629985800606, u'easier': 0.2514629985800606, u'phone': 0.355245321275764, u'windows': 0.2514629985800606, u'date': 0.2514629985800606, u'world': 0.355245321275764, u'everyday': 0.2514629985800606}

• for Siri:
  
  

{u'understands': 0.05555555555555555, u'talks': 0.05555555555555555, u'calls': 0.05555555555555555, u'schedule': 0.05555555555555555, u'meetings': 0.05555555555555555, u'siri': 0.2222222222222222, u'messages': 0.05555555555555555, u'send': 0.05555555555555555, u'talking': 0.05555555555555555, u'easy': 0.05555555555555555, u'mean': 0.05555555555555555, u'finding': 0.05555555555555555, u'voice': 0.05555555555555555, u'talk': 0.05555555555555555, u'phone': 0.05555555555555555}

{u'understands': 0.2514629985800606, u'talks': 0.2514629985800606, u'calls': 0.2514629985800606, u'schedule': 0.2514629985800606, u'meetings': 0.2514629985800606, u'siri': 0.2514629985800606, u'messages': 0.2514629985800606, u'send': 0.355245321275764, u'talking': 0.355245321275764, u'easy': 0.2514629985800606, u'mean': 0.355245321275764, u'finding': 0.2514629985800606, u'voice': 0.3868528072345415, u'talk': 0.2514629985800606, u'phone': 0.355245321275764}