User:Pedro Sá Couto/TW/JSTOR De-watermarking

STEPS

De-watermarking is separated into 4 steps:

1. Bursting the PDF into png
2. Overlaying the cover
3. Overlaying the pages
4. OCR again

RESULTS IN EACH STEP

0. Starting with a Paper from JSTOR
File:42938075.pdf

1. Bursting the PDF into PNGs

PDF is seperated into pages

• 
• 
• 
• 
• 
• 

2. Overlaying the cover

The cover is overlayed and dewatermarked

• 

3. Overlaying the pages

The pages are overlayed and dewatermarked

• 
• 
• 
• 
• 

4. OCR again

You have a De-watermarked, searchable PDF

File:42938075 dewater.pdf

FLOW

JSTOR.SH

To activate the stream I use ./jstor.sh

It is on a bash for loop. In this way I can easily control each 5 outputs.

for i in {1..5}
do
  cd /Users/PSC/Desktop/JSTOR/jstorpaper/
  for name in *; do mv "$name" "${name// /_}"; done
  cd `ls -td -- /Users/PSC/Desktop/JSTOR/jstorpaper/* | head -n 1`
  var2=`ls -td -- /Users/PSC/Desktop/JSTOR/jstorpaper/* | head -n 1`
  for name in *; do mv "$name" "${name// /_}"; done
  cp $var2/*.pdf /Users/PSC/Desktop/JSTOR/overlay
  cd /Users/PSC/Desktop/JSTOR/overlay
  mv /Users/PSC/Desktop/JSTOR/overlay/*.pdf target.pdf
  mkdir -p split
  python3 burstpdf.py
  python3 overlaylogo_cover.py
  python3 overlaylogo_page.py
  rm target.pdf
  convert "split/*.{png,jpeg,pdf}" -quality 100 name.pdf
  var1=$var2/*.pdf
  mv name.pdf $var1
  rm -r split
  mv $var2 /Users/PSC/Desktop/JSTOR/ready
  cd /Users/PSC/Desktop/JSTOR/ready
  ocrmypdf `ls -td -- /Users/PSC/Desktop/JSTOR/ready/*/*.pdf | head -n 1` `ls -td -- /Users/PSC/Desktop/JSTOR/ready/*/*.pdf | head -n 1`
done

1. Bursting the PDF into png

#Based in the code in https://iq.opengenus.org/pdf_to_image_in_python/

import pdf2image
from PIL import Image
import time

#DECLARE CONSTANTS
PDF_PATH = "target.pdf"
DPI = 200
FIRST_PAGE = None
LAST_PAGE = None
FORMAT = 'png'
THREAD_COUNT = 1
USERPWD = None
USE_CROPBOX = False
STRICT = False

def pdftopil():
    #This method reads a pdf and converts it into a sequence of images
    #PDF_PATH sets the path to the PDF file
    #dpi parameter assists in adjusting the resolution of the image
    #first_page parameter allows you to set a first page to be processed by pdftoppm
    #last_page parameter allows you to set a last page to be processed by pdftoppm
    #fmt parameter allows to set the format of pdftoppm conversion (PpmImageFile, TIFF)
    #thread_count parameter allows you to set how many thread will be used for conversion.
    #userpw parameter allows you to set a password to unlock the converted PDF
    #use_cropbox parameter allows you to use the crop box instead of the media box when converting
    #strict parameter allows you to catch pdftoppm syntax error with a custom type PDFSyntaxError

    start_time = time.time()
    pil_images = pdf2image.convert_from_path(PDF_PATH, dpi=DPI, first_page=FIRST_PAGE, last_page=LAST_PAGE, fmt=FORMAT, thread_count=THREAD_COUNT, userpw=USERPWD, use_cropbox=USE_CROPBOX, strict=STRICT)
    print ("Time taken : " + str(time.time() - start_time))
    return pil_images

def save_images(pil_images):
    d = 1
    for image in pil_images:
        image.save(("split/page%d"%d) + ".png")
        d += 1

if __name__ == "__main__":
    pil_images = pdftopil()
    save_images(pil_images)

2. Overlaying the cover

from PIL import Image

background = Image.open("split/page1.png")

#rescaling the logo
basewidth = (background.size[0])
baseheight = (background.size[1])
finalcover = Image.open("cover.png")
wpercent = (basewidth/float(finalcover.size[0]))
hsize = int((float(finalcover.size[1])*float(wpercent)))
finalcover = finalcover.resize((basewidth,hsize), Image.ANTIALIAS)
finalcover.save("cover_rescale.png")

foreground = Image.open("cover_rescale.png")
foregroundheight = (foreground.size[1])

background.paste(foreground, (0, (baseheight-foregroundheight)), foreground.convert('RGBA'))
background.save("split/page1.png")

3. Overlaying the pages

This happens through ./jstor.sh

from PIL import Image

base = Image.open("split/page2.png")

#rescaling the logo
basewidth = (base.size[0])
baseheight = (base.size[1])
finalpage = Image.open("pages.png")
wpercent = (basewidth/float(finalpage.size[0]))
hsize = int((float(finalpage.size[1])*float(wpercent)))
finalpage = finalpage.resize((basewidth,hsize), Image.ANTIALIAS)
finalpage.save("page_rescale.png")

foreground = Image.open("page_rescale.png")
foregroundheight = (foreground.size[1])

i = 2

while True:
    try:
        background = Image.open("split/page%i.png"%i)

        background.paste(foreground, (0, (baseheight-foregroundheight)), foreground.convert('RGBA'))
        background.save("split/page%i.png"%i)

        i+=1

    except:
        print("DID MY JOB!")
        break

4. OCR again

This happens through ./jstor.sh

ocrmypdf `ls -td -- /Users/PSC/Desktop/JSTOR/ready/*/*.pdf | head -n 1` `ls -td -- /Users/PSC/Desktop/JSTOR/ready/*/*.pdf | head -n 1`