User:Zuhui//Prototyping/BASH: Difference between revisions

From XPUB & Lens-Based wiki
< User:Zuhui‎ | ‎ | Prototyping
(Created page with "[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/cloudmix.html 151024] ===yt-dlp scripts=== [https://hub.xpub.nl/cerealbox/~manetta/cloudmix/yt-dlp/ 151024 yt-dlp] //go to directory I want to save the youtube file '''$yt-dlp (video url)''' //copy the name of the file '''$ffmpeg -i (name of the file) (filename+filetype)''' //this will convert to filetype that I want to download ===stream segments scripts=== [https://hub.xpub.nl/cerealbox/~manetta/cloudmix/stream-segm...")
 
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/cloudmix.html 151024]
=Scripts=
===yt-dlp scripts===
[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/cloudmix.html Tuesday with Manetta 151024]
==yt-dlp==
[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/yt-dlp/ 151024 yt-dlp]
[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/yt-dlp/ 151024 yt-dlp]
  //go to directory I want to save the youtube file
  //go to directory I want to save the youtube file
Line 7: Line 8:
  '''$ffmpeg -i (name of the file) (filename+filetype)''' //this will convert to filetype that I want to download
  '''$ffmpeg -i (name of the file) (filename+filetype)''' //this will convert to filetype that I want to download


===stream segments scripts===
==Stream segments==
[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/stream-segments/ 151024 stream segments]
[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/stream-segments/ 151024 stream segments]
==Vosk==
[https://hub.xpub.nl/cerealbox/~manetta/cloudmix/vosk/ 151024 with Manetta]
==Vosk 2==
[https://hub.xpub.nl/cerealbox/~murtaugh/si25/mixcloud/ 251124 with Michael]
<br><br>
'''Trial 1, with [https://hub.xpub.nl/cerealbox/~zuhui/transcriptt/interview/adventure.webm Adventure Time clip]'''
'''$ yt-dlp "https://www.youtube.com/watch?v=qbsvZclUXh8" --write-info-json'''
'''$ mv *.info.json *.webm interview/'''
'''$ ffmpeg -i adventure.webm adventure.wav'''
'''$ vosk-transcriber -l en-us -i adventure.wav -t srt -o adventure.srt'''
'''$ ../py_script/srt2vtt.py adventure.srt adventure.vtt'''
'''$ chmod +x ../py_script/srt2vtt.py''' //if permission denied
'''$ ffmpeg -i adventure.wav adventure.mp3''' //for html
'''[https://hub.xpub.nl/cerealbox/~zuhui/transcriptt/interview/adventuretime_transcript.html ↘︎it works]'''
<br><br>
'''Trial 2, with Json to vtt'''
'''$ vosk-transcriber -l en-us -i adventure.wav -t json -o adventure.json'''
'''$ ../py_script/voskjson2vtt.py adventure.json adventure_words.vtt'''
'''[https://hub.xpub.nl/cerealbox/~zuhui/transcriptt/interview/adventuretime_transcript_ver2.html ↘︎doesn't work and it's vtt file problem → ok now it works, used wrong python file to make vtt out of json]'''
<br><br>
===Video subtitling===
{{vimeo|1043091875}}
▲original<br>
- first create vtt file (json if necessary), running python files.<br>
- put them together on html
<video src="liberty-loop.mp4" controls style="width: 50%" >
    <track src="liberty-loop_words.vtt" kind="subtitles" srclang="en" label="English" default>
</video>
-also can style the subtitle on either vtt file or html.
'''WEBVTT'''
STYLE
::cue {
    color: yellow;
    background: rgba(0, 0, 0, 0.7);
    font-size: 20px;
    font-family: Arial, sans-serif;
}
'''HTML CSS'''
<style>
    video {
    }
    ::cue {
      color: yellow;
      background: rgba(0, 0, 0, 0.7);
      font-size: 20px;
      font-family: "Courier New", monospace;
    }
  </style>
↘︎ '''[https://hub.xpub.nl/cerealbox/~zuhui/transcriptt/yoon/liberty-loop.html subtitle created(language detection in eng)]'''
<br><br>
===Embedding subtitle to mp4 file===
ffmpeg -i ''input.mp4'' -i ''subtitles.vtt'' -c:v copy -c:a copy -c:s mov_text -metadata:s:s:0 language=eng ''output.mp4''
* '''-i input.mp4:''' input video file
* '''-i subtitles.vtt:''' input subtitle file
* '''-c:v copy:''' copy video stream without re-encoding
* '''-c:a copy:''' copy audio stream without re-encoding
* '''-c:s mov_text:''' convert to a subtitle format supported by .mp4 (mov_text)
* '''-metadata:s:s:0 language=eng:''' subtitle lang
* '''output.mp4'''
===↘︎ Force style===
mov_text doesn't support subtitle styling. If I really want to style the subtitle, one way to do it is "hardcoding"(rendering vtt file directly to video file).
ffmpeg -i liberty-loop.mp4 -vf "subtitles=liberty-loop_0.vtt:force_style='Fontsize=35,PrimaryColour=&H0000FFFF&,Fontname=\"Courier New\",Alignment=6,MarginV=150'" subtitle-embed2.mp4
* ''':force_style:'''
* '''PrimaryColour:''' 16진수 BGR code → ffmpeg code convert (basically adding &H00 on the front, & on the back): #5733FF → &H005733FF&
* '''Bold and Italic:''' Bold=1, Italic=1
* '''Alignment:''' Alignment=6(top center), 2(mid center), 10(bottom center) 
* '''MarginV:''' vertical margin, unit is pixel.
'''↘︎[https://hub.xpub.nl/cerealbox/~zuhui/SI25/transcriptt/yoon/subtitle-embed2.mp4 :force_style example]'''
<br><br>
==wget==
if the local computer doesn't have wget
$ brew install wget
$ cd (folder you want to scrape data into)
$ wget -r (url)
=Command line=
'''pwd''' outputs the name of the current working directory.
'''ls''' lists all files and directories in the working directory.
'''cd''' switches you into the directory you specify.
'''mkdir''' creates a new directory in the working directory.
'''touch''' creates a new file inside the working directory.
'''cat''' show you immediately what's inside the file
Helper commands to make navigation easier:
'''clear''' clears the terminal
'''tab''' autocompletes the name of a file or directory
'''↑ and ↓''' allow you to cycle through previous commands
'''ls -a''' lists all contents of a directory, including hidden files and directories
'''ls -l''' lists all contents in long format
'''ls -t''' orders files and directories by the time they were last modified
Multiple options can be used together, like '''ls -alt'''
From the command line, you can also copy, move, and remove files and directories:
'''cp''' copies files
'''mv''' moves and renames files
'''rm''' removes files
'''rm -r''' removes directories
'''rm -rf'''  adding -f removes all the files without having to confirm
Redirection + other commands are powerful when combined with redirection commands:
'''>''' redirects standard output of a command to a file, overwriting previous content.
'''>>''' redirects standard output of a command to a file, appending new content to old content.
'''<''' redirects standard input to a command.
'''|''' redirects standard output of a command to another command.
'''sort''' sorts lines of text alphabetically.
'''uniq''' filters duplicate, adjacent lines of text.
'''grep''' searches for a text pattern and outputs it.
'''sed''' searches for a text pattern, modifies it, and outputs it.

Latest revision as of 09:09, 8 February 2025

Scripts

Tuesday with Manetta 151024

yt-dlp

151024 yt-dlp

//go to directory I want to save the youtube file
$yt-dlp (video url)
//copy the name of the file
$ffmpeg -i (name of the file) (filename+filetype) //this will convert to filetype that I want to download

Stream segments

151024 stream segments

Vosk

151024 with Manetta

Vosk 2

251124 with Michael

Trial 1, with Adventure Time clip

$ yt-dlp "https://www.youtube.com/watch?v=qbsvZclUXh8" --write-info-json
$ mv *.info.json *.webm interview/
$ ffmpeg -i adventure.webm adventure.wav
$ vosk-transcriber -l en-us -i adventure.wav -t srt -o adventure.srt
$ ../py_script/srt2vtt.py adventure.srt adventure.vtt
$ chmod +x ../py_script/srt2vtt.py //if permission denied
$ ffmpeg -i adventure.wav adventure.mp3 //for html

↘︎it works

Trial 2, with Json to vtt

$ vosk-transcriber -l en-us -i adventure.wav -t json -o adventure.json
$ ../py_script/voskjson2vtt.py adventure.json adventure_words.vtt

↘︎doesn't work and it's vtt file problem → ok now it works, used wrong python file to make vtt out of json

Video subtitling

http://vimeo.com/1043091875 ▲original
- first create vtt file (json if necessary), running python files.
- put them together on html

<video src="liberty-loop.mp4" controls style="width: 50%" >
   <track src="liberty-loop_words.vtt" kind="subtitles" srclang="en" label="English" default>
</video>

-also can style the subtitle on either vtt file or html.

WEBVTT
STYLE
::cue {
   color: yellow;
   background: rgba(0, 0, 0, 0.7);
   font-size: 20px;
   font-family: Arial, sans-serif;
}
HTML CSS
<style>
   video {
   }
   ::cue {
     color: yellow;
     background: rgba(0, 0, 0, 0.7);
     font-size: 20px;
     font-family: "Courier New", monospace;
   }
 </style>

↘︎ subtitle created(language detection in eng)

Embedding subtitle to mp4 file

ffmpeg -i input.mp4 -i subtitles.vtt -c:v copy -c:a copy -c:s mov_text -metadata:s:s:0 language=eng output.mp4
  • -i input.mp4: input video file
  • -i subtitles.vtt: input subtitle file
  • -c:v copy: copy video stream without re-encoding
  • -c:a copy: copy audio stream without re-encoding
  • -c:s mov_text: convert to a subtitle format supported by .mp4 (mov_text)
  • -metadata:s:s:0 language=eng: subtitle lang
  • output.mp4

↘︎ Force style

mov_text doesn't support subtitle styling. If I really want to style the subtitle, one way to do it is "hardcoding"(rendering vtt file directly to video file).

ffmpeg -i liberty-loop.mp4 -vf "subtitles=liberty-loop_0.vtt:force_style='Fontsize=35,PrimaryColour=&H0000FFFF&,Fontname=\"Courier New\",Alignment=6,MarginV=150'" subtitle-embed2.mp4
  • :force_style:
  • PrimaryColour: 16진수 BGR code → ffmpeg code convert (basically adding &H00 on the front, & on the back): #5733FF → &H005733FF&
  • Bold and Italic: Bold=1, Italic=1
  • Alignment: Alignment=6(top center), 2(mid center), 10(bottom center)
  • MarginV: vertical margin, unit is pixel.

↘︎:force_style example

wget

if the local computer doesn't have wget
$ brew install wget
$ cd (folder you want to scrape data into)
$ wget -r (url)

Command line

pwd outputs the name of the current working directory.
ls lists all files and directories in the working directory.
cd switches you into the directory you specify.
mkdir creates a new directory in the working directory.
touch creates a new file inside the working directory.
cat show you immediately what's inside the file

Helper commands to make navigation easier:

clear clears the terminal
tab autocompletes the name of a file or directory
↑ and ↓ allow you to cycle through previous commands
ls -a lists all contents of a directory, including hidden files and directories
ls -l lists all contents in long format
ls -t orders files and directories by the time they were last modified
Multiple options can be used together, like ls -alt

From the command line, you can also copy, move, and remove files and directories:

cp copies files
mv moves and renames files
rm removes files
rm -r removes directories
rm -rf  adding -f removes all the files without having to confirm

Redirection + other commands are powerful when combined with redirection commands:

> redirects standard output of a command to a file, overwriting previous content.
>> redirects standard output of a command to a file, appending new content to old content.
< redirects standard input to a command.
| redirects standard output of a command to another command.
sort sorts lines of text alphabetically.
uniq filters duplicate, adjacent lines of text.
grep searches for a text pattern and outputs it.
sed searches for a text pattern, modifies it, and outputs it.