User:Manetta/prototyping/2001-annotated-cut-ups: Difference between revisions

From XPUB & Lens-Based wiki
No edit summary
 
Line 1: Line 1:
{{#widget:YouTube|id=nuxMHelPVJw}}
= annotated cut ups =
= annotated cut ups =


Line 7: Line 10:


triple = subject + predicate + object<br>
triple = subject + predicate + object<br>
example: Dave + is located in + main room  
example: Dave + is located in + main room <br>




Line 21: Line 24:




After annotating the first 8 minutes of the Jupiter Mission (part of 2001, a Space Odyssey), i created a set of 28 clips containing all the shots in which a certain 'subject' is present. The following clips are videogrepped at the following terms: <br>
 
== 2. create subtitle file from csv file ==
 
Then, i turned the annotations from csv into a subtitle file.
 
<source lang="python">
import os
import csv
import itertools
 
# SIMPLE SRT:
with open('2001-space-odyssey-simple.srt', 'w') as txt:
with open('2001-chunks-annotations-01-simple.csv', 'rb') as f:
reader = csv.reader(f)
i=1
for row in reader:
start = row[0]
end = row[1]
x1 = row[2]
r1 = row[3]
y1 = row[4]
x2 = row[6]
r2 = row[7]
y2 = row[8]
x3 = row[10]
r3 = row[11]
y3 = row[12]
x4 = row[14]
r4 = row[15]
y4 = row[16]
x5 = row[18]
r5 = row[19]
y5 = row[20]
x6 = row[22]
r6 = row[23]
y6 = row[24]
x7 = row[26]
r7 = row[27]
y7 = row[28]
x8 = row[30]
r8 = row[31]
y8 = row[32]
x9 = row[34]
r9 = row[35]
y9 = row[36]
a = i
b = start+" --> "+end
c = x1+" "+r1+" "+y1+"\n"+x2+" "+r2+" "+y2+"\n"+<br>x3+" "+r3+" "+y3+"\n"+x4+" "+r4+" "+y4+"\n"+x5+" "+<br>r5+" "+y5+"\n"+x6+" "+r6+" "+y6+"\n"+<br>x7+" "+r7+" "+y7+"\n"+x8+" "+r8+" "+y8+"\n"+<br>x9+" "+r9+" "+y9+"\n\n\n"
# print c
 
txt.write(str(a)+"\n")
txt.write(b+"\n")
txt.write(c)   
i=i+1
</source>
 
 
 
1
00:00:00.00 --> 00:00:07.00
space is located in between stars
 
2
00:00:07.00 --> 00:01:08.00
spaceship is located in space
spaceflight flying to jupiter
spaceship performs spaceflight
3
00:01:08.00 --> 00:01:19.00
spaceship is located in space
4
00:01:19.00 --> 00:01:31.00
spaceship is located in space
5
00:01:31.00 --> 00:01:47.00
dave is located in main room
dave  running at main room
main room  is located in  spaceship
6
00:01:47.00 --> 00:01:56.00
other three astronauts is located in hybernation beds




== undefined room ==
== 2001 'element' clips ==
 
After annotating the first 8 minutes of the Jupiter Mission (part of 2001, a Space Odyssey), i created a set of 28 clips containing all the shots in which a certain 'element' is present (including the connections). To do this, i used videogrep to search for an 'element' in the subtitles i just created:
 
 
searching with regular expressions (re) for either 'dave' and 'dave)'
 
<source lang="python">
import os
import re
 
p = "python videogrep.py --input 2001-space-odyssey-jupiter-mission.avi --search 'dave\ |dave\)' --output video-dave.mp4"
print p
os.system(p)
</source>
 
 
 
The following clips are videogrepped at the following terms: <br>
 
 
=== undefined room ===
{{#widget:YouTube|id=XahNfDa2_-Y}}
{{#widget:YouTube|id=XahNfDa2_-Y}}


== Dave ==
=== Dave ===
{{#widget:YouTube|id=X4e_Y668nw4}}
{{#widget:YouTube|id=X4e_Y668nw4}}


== HAL ==
=== HAL ===
{{#widget:YouTube|id=Qv87Z0Xvbmg}}
{{#widget:YouTube|id=Qv87Z0Xvbmg}}


== button ==
=== button ===
{{#widget:YouTube|id=8z2PtN0_NC8}}
{{#widget:YouTube|id=8z2PtN0_NC8}}


== main room ==
=== main room ===
{{#widget:YouTube|id=q45OCpxy8lI}}
{{#widget:YouTube|id=q45OCpxy8lI}}


== tablet ==
=== tablet ===
{{#widget:YouTube|id=9F_y__QmQwg}}
{{#widget:YouTube|id=9F_y__QmQwg}}
== reassembled version of 2001 ==
This made it possible to later on connect the different characters and locations in the film, by doing a set of 'search and replace' queries. And so this connected for example every shot of the 'main room' with a shot of the 'spaceship', as the main room is located in the space ship.<br>
This method uses the relations that are created in the film, using the act of deduction: when the main room has a fixed truth of being in the spaceship, all the shots that contain 'main room' are connected to the 'spaceship' shots. <br>
I wanted to bring these videogrepped clips back to the film, to create a reassembled version of 2001, based on these relations. So then I created an ordering of the grepped videoclips, and exported a new videofile where the grepped clips are timestretched to the original shotduration:
<source lang="python">
from __future__ import division
import os
import csv
import re
import glob
import subprocess as sub
import datetime
import itertools
# open csv with duration column in it:
with open('../2001-chunks-annotations-01-duration.csv', 'rb') as f:
reader = csv.reader(f)
i=0
# *** GET DURATION OF CSV SHEET (ORIGINAL SHOT DURATION) ***
for row in reader:
shotduration = row[3]
print shotduration
# --> if using an integer number:
# shotduration = "".join(shotduration)
# shotduration = shotduration.split('.')
# shotduration = shotduration[0]
# SHOTDURATION / GREPPEDDURATION = SETPTS
# *** LIST OF VIDEO'S TO REPLACE ORIGINAL SHOTS ***
videofiles = [
'video-between-stars.mp4',
'video-jupiter.mp4',
'video-space.mp4',
'video-space.mp4',
'video-main-room.mp4',
'video-hybernation-beds.mpeg',
'video-main-room.mp4',
'video-main-room.mp4',
'video-main-room.mp4',
'video-undefined-room.mp4',
'video-food.mp4',
'video-button.mp4',
'video-door.mp4',
'video-frank.mp4',
'video-button.mp4',
'video-button.mp4',
'video-BBC.mp4',
'video-main-room.mp4',
'video-tablet.mp4',
'video-couch.mp4',
'video-earth.mp4',
'video-other-three-astronauts.mpeg',
'video-frank-and-dave.mp4',
'video-hybernation.mp4',
'video-tablet.mp4',
'video-earth.mp4',
'video-hybernation-beds.mpeg',
'video-food.mp4',
'video-frank-at-tablet.mp4',
'video-frank-and-dave-at-tablet.mp4',
'video-hybernation-beds.mpeg',
'video-HAL.mp4',
'video-main-room.mp4',
'video-earth.mp4',
'video-HAL.mp4',
'video-earth.mp4',
'video-BBC.mp4',
'video-eating-table.mp4',
'video-tablet.mp4',
'video-earth.mp4',
'video-frank-at-tablet.mp4',
'video-HAL.mp4',
'video-tablet.mp4',
'video-main-room.mp4',
'video-HAL.mp4'
]
# print videofiles[i]
# *** READING DURATION OF LIST OF VIDEOFILES***
v = "ffmpeg -i "+videofiles[i]+" 2>&1 | grep Duration | awk '{print $2}' | tr -d ,"
# print v
p = os.popen(v,'rb')
while 1:
line = p.readline()
if not line: break
# print line
number = line.split(':')
# print number[0]
# *** CONVERTING TIMESTAMP
# TO INTEGER (IN SECONDS) ***
hours = float(number[0])*2400
# print number[1]
minutes = float(number[1])*60
# print number[2]
seconds = float(number[2])
# --> if using integer number: 
# seconds = number[2].split('.')
# seconds = seconds[0]
clipduration = float(hours)+float(minutes)+float(seconds)
print clipduration
# set stretch value:
pts = float(shotduration) / float(clipduration)
print pts
# print i
# *** STRETCH THE VIDEOFILES TO ORIGINAL SHOT DURATION
# (DIVIDING SHOTDURATION / CLIPDURATION)
# SHOTDURATION = FROM ORIGINAL FILM SPEED
# CLIPDURATION = LENGTH OF VIDEOGREPPED VIDEOS ***
x = "ffmpeg -i "+videofiles[i]+" -vf 'setpts="+str(pts)+"*PTS' -an -y export/0"+str(i)+".mpeg"
print x
os.system(x)
# *** WRITE VIDEO FILENAME TO TXT FILE ***
with open('clipslist.txt', 'w') as txt:
m = 0
for video in videofiles:
print>>txt,"file 'export/0"+str(m)+".mpeg'"
m=m+1
print "ENDING"
i=i+1
# *** joining the stretched clips together in one file ***
a = "ffmpeg -f concat -i clipslist.txt -c copy 2001-joined-clips-01.avi"
print a
os.system(a)
</source>
== end of process ==
As the clips became quite repetitive, and the system behind it quite complex, i decided to leave the clips here, and focus on 2001 from a different perspective.<br>

Latest revision as of 09:51, 8 April 2015


annotated cut ups

(proces scripts here step 1-3)

As I was thinking to create an excercise of the semantic-web by working with film, i started to write annotations in a spreadsheat-file. There, I notated a starting and ending time, and described the actions of the characters and objects that are present in that shot. When turning these annotations in subtitle files, it produces a very descriptive layer onto the film. (and i doubt that that is bringing the viewer that much). I wrote the descriptions in the form of so called 'triples'. A triple is a description used in markup-languages as RDF, which contains three elements:


triple = subject + predicate + object
example: Dave + is located in + main room


the first lines of the annotations:

00:00:00.00,00:00:07.00,space,is located in,between stars
00:00:07.00,00:01:08.00,spaceship,is located in,space,#,spaceflight,flying to,jupiter,#,spaceship,performs,spaceflight
00:01:08.00,00:01:19.00,spaceship,is located in,space
00:01:19.00,00:01:31.00,spaceship,is located in,space
00:01:31.00,00:01:47.00,dave,is located in,main room,,dave ,running at,main room,,main room ,is located in ,spaceship
00:01:47.00,00:01:56.00,other three astronauts,is located in,hybernation beds
00:01:56.00,00:02:23.00,dave,is located in,main room,,dave ,running at,main room
00:02:23.00,00:02:30.00,dave,is located in,main room,,dave ,running at,main room


2. create subtitle file from csv file

Then, i turned the annotations from csv into a subtitle file.

import os
import csv
import itertools

# SIMPLE SRT:
with open('2001-space-odyssey-simple.srt', 'w') as txt:
	with open('2001-chunks-annotations-01-simple.csv', 'rb') as f:
			reader = csv.reader(f)
			i=1
			for row in reader:
				start = row[0]
				end = row[1]
				x1 = row[2]
				r1 = row[3]
				y1 = row[4]
				x2 = row[6]
				r2 = row[7]
				y2 = row[8]
				x3 = row[10]
				r3 = row[11]
				y3 = row[12]
				x4 = row[14]
				r4 = row[15]
				y4 = row[16]
				x5 = row[18]
				r5 = row[19]
				y5 = row[20]
				x6 = row[22]
				r6 = row[23]
				y6 = row[24]
				x7 = row[26]
				r7 = row[27]
				y7 = row[28]
				x8 = row[30]
				r8 = row[31]
				y8 = row[32]
				x9 = row[34]
				r9 = row[35]
				y9 = row[36]
				a = i
				b = start+" --> "+end
				c = x1+" "+r1+" "+y1+"\n"+x2+" "+r2+" "+y2+"\n"+<br>x3+" "+r3+" "+y3+"\n"+x4+" "+r4+" "+y4+"\n"+x5+" "+<br>r5+" "+y5+"\n"+x6+" "+r6+" "+y6+"\n"+<br>x7+" "+r7+" "+y7+"\n"+x8+" "+r8+" "+y8+"\n"+<br>x9+" "+r9+" "+y9+"\n\n\n"
				# print c

				txt.write(str(a)+"\n")
				txt.write(b+"\n")
				txt.write(c)    
				i=i+1


1
00:00:00.00 --> 00:00:07.00
space is located in between stars
 
2
00:00:07.00 --> 00:01:08.00
spaceship is located in space
spaceflight flying to jupiter
spaceship performs spaceflight

3
00:01:08.00 --> 00:01:19.00
spaceship is located in space

4
00:01:19.00 --> 00:01:31.00
spaceship is located in space

5
00:01:31.00 --> 00:01:47.00
dave is located in main room
dave  running at main room
main room  is located in  spaceship

6
00:01:47.00 --> 00:01:56.00
other three astronauts is located in hybernation beds


2001 'element' clips

After annotating the first 8 minutes of the Jupiter Mission (part of 2001, a Space Odyssey), i created a set of 28 clips containing all the shots in which a certain 'element' is present (including the connections). To do this, i used videogrep to search for an 'element' in the subtitles i just created:


searching with regular expressions (re) for either 'dave' and 'dave)'
import os
import re

p = "python videogrep.py --input 2001-space-odyssey-jupiter-mission.avi --search 'dave\ |dave\)' --output video-dave.mp4"
print p
os.system(p)


The following clips are videogrepped at the following terms:


undefined room

Dave

HAL

button

main room

tablet


reassembled version of 2001

This made it possible to later on connect the different characters and locations in the film, by doing a set of 'search and replace' queries. And so this connected for example every shot of the 'main room' with a shot of the 'spaceship', as the main room is located in the space ship.

This method uses the relations that are created in the film, using the act of deduction: when the main room has a fixed truth of being in the spaceship, all the shots that contain 'main room' are connected to the 'spaceship' shots.

I wanted to bring these videogrepped clips back to the film, to create a reassembled version of 2001, based on these relations. So then I created an ordering of the grepped videoclips, and exported a new videofile where the grepped clips are timestretched to the original shotduration:


from __future__ import division 
import os
import csv
import re
import glob
import subprocess as sub
import datetime
import itertools

# open csv with duration column in it:
with open('../2001-chunks-annotations-01-duration.csv', 'rb') as f:
	reader = csv.reader(f)
	i=0

	# *** GET DURATION OF CSV SHEET (ORIGINAL SHOT DURATION) ***
	for row in reader:
		shotduration = row[3]
		print shotduration

		# --> if using an integer number:
		# shotduration = "".join(shotduration)
		# shotduration = shotduration.split('.')
		# shotduration = shotduration[0]

		# SHOTDURATION / GREPPEDDURATION = SETPTS

		# *** LIST OF VIDEO'S TO REPLACE ORIGINAL SHOTS ***
		videofiles = [
		'video-between-stars.mp4',
		'video-jupiter.mp4',
		'video-space.mp4',
		'video-space.mp4',
		'video-main-room.mp4',
		'video-hybernation-beds.mpeg',
		'video-main-room.mp4',
		'video-main-room.mp4',
		'video-main-room.mp4',
		'video-undefined-room.mp4',
		'video-food.mp4',
		'video-button.mp4',
		'video-door.mp4',
		'video-frank.mp4',
		'video-button.mp4',
		'video-button.mp4',
		'video-BBC.mp4',
		'video-main-room.mp4',
		'video-tablet.mp4',
		'video-couch.mp4',
		'video-earth.mp4',
		'video-other-three-astronauts.mpeg',
		'video-frank-and-dave.mp4',
		'video-hybernation.mp4',
		'video-tablet.mp4',
		'video-earth.mp4',
		'video-hybernation-beds.mpeg',
		'video-food.mp4',
		'video-frank-at-tablet.mp4',
		'video-frank-and-dave-at-tablet.mp4',
		'video-hybernation-beds.mpeg',
		'video-HAL.mp4',
		'video-main-room.mp4',
		'video-earth.mp4',
		'video-HAL.mp4',
		'video-earth.mp4',
		'video-BBC.mp4',
		'video-eating-table.mp4',
		'video-tablet.mp4',
		'video-earth.mp4',
		'video-frank-at-tablet.mp4',
		'video-HAL.mp4',
		'video-tablet.mp4',
		'video-main-room.mp4',
		'video-HAL.mp4'
		]
		# print videofiles[i]

		# *** READING DURATION OF LIST OF VIDEOFILES***
		v = "ffmpeg -i "+videofiles[i]+" 2>&1 | grep Duration | awk '{print $2}' | tr -d ,"
		# print v
		p = os.popen(v,'rb')
		while 1:
			line = p.readline()
			if not line: break
			# print line
			number = line.split(':')
			# print number[0]

			# *** CONVERTING TIMESTAMP 
			# TO INTEGER (IN SECONDS) ***
			hours = float(number[0])*2400
			# print number[1]
			minutes = float(number[1])*60
			# print number[2]
			seconds = float(number[2])

			# --> if using integer number:  
			# seconds = number[2].split('.')
			# seconds = seconds[0]

			clipduration = float(hours)+float(minutes)+float(seconds)
			print clipduration	

		# set stretch value:
		pts = float(shotduration) / float(clipduration)
		print pts
		# print i

		# *** STRETCH THE VIDEOFILES TO ORIGINAL SHOT DURATION
		# (DIVIDING SHOTDURATION / CLIPDURATION)
		# SHOTDURATION = FROM ORIGINAL FILM SPEED
		# CLIPDURATION = LENGTH OF VIDEOGREPPED VIDEOS ***
		x = "ffmpeg -i "+videofiles[i]+" -vf 'setpts="+str(pts)+"*PTS' -an -y export/0"+str(i)+".mpeg"
		print x
		os.system(x)

		# *** WRITE VIDEO FILENAME TO TXT FILE ***
		with open('clipslist.txt', 'w') as txt:
			m = 0
			for video in videofiles:
				print>>txt,"file 'export/0"+str(m)+".mpeg'"
				m=m+1

		print "ENDING"
		i=i+1

# *** joining the stretched clips together in one file ***
a = "ffmpeg -f concat -i clipslist.txt -c copy 2001-joined-clips-01.avi"
print a
os.system(a)


end of process

As the clips became quite repetitive, and the system behind it quite complex, i decided to leave the clips here, and focus on 2001 from a different perspective.