XPPL Documentation

About

What is XPPL?

XPPL is a space for potential pirate librarianship aimed at people who are studying the field of media culture, or as we like to call them: knowledge comrades. This library gathers all the books and articles floating around on PZI shelves and our hard drives and memory sticks, so that they can be shared. Its main interface hosts a curated catalogue of books and articles, and its distributed architecture allows for instances of uploading and downloading. It starts at XPUB, but can go anywhere we want it to.

See the XPPL project page for more on the concept & process.

How does it work?

The XPPL uses:

• RQLite: a distributed version of SQL
• Flask (and Jinja): for the web application
• VPN and Syncthing: for networking & file management
• 3 initial instances of the library, running on separate machines, including one read-only version on a server -----> lib.xpub.nl

The framework onto which the XPPL is built is as important as the library itself. The access to the library has two levels: the collection (books and stacks) is interfaced through a public catalogue that anyone can browse with the help of various experimental interfaces, but the files and stacks are only available for download and edit for those who find themselves in the 'comrade circle'. Meaning: librarians who are currently connected to a local instance of the XPPL (which has the book files in storage).

During specific moments, any XPPL instance can provide full access to the files and editing features. During workshops, presentations, parties anyone is invited to intervene in the library. The purpose of having a network of people around the library is that anyone in the network can contribute, either by expanding the collection, augmenting existing features, or by adding completely new ones. Thus, the library is changing and growing continuously. We aim to keep the process of building and maintaining this library as transparent as possible, so that it can also have an educational value.

At the base of the library, there is a core collection of books and stacks. We are building easy to use methods to communicate with this core (API), so you can build your own interface, or features that are part of the 'default' interface for the XPUB Library.

Setting up an instance from image

What is in the image

Requirements:

• sqlalchemy-rqlite
• appdirs==1.4.0
• click==6.7
• Flask==0.12
• Flask-SQLAlchemy==2.1
• Flask-WTF==0.14.2
• gunicorn==19.6.0
• itsdangerous==0.24
• Jinja2==2.9.4
• MarkupSafe==0.23
• packaging==16.8
• pyparsing==2.1.10
• six==1.10.0
• SQLAlchemy==1.1.5
• Werkzeug==0.11.15
• WTForms==2.1
• flask-marshmallow==0.9.0
• Wand==0.4.4
• PyPDF2==1.26.0
• flask-socketio==2.9.2
• flask-whooshalchemyplus==0.7.5
• python-dotenv==0.7.1
• autocomplete==0.0.104

rqlite

Rqlite is a lightweight, distributed relational database built on SQLite. Git link

Documentation

Syncthing

We use Syncthing to share and sync the /uploads and /cover folder among multiple nodes of the XPPL. The only node which does not have Syncthing running on both of these folders is the server, which is only showing the catalogue version of the site. The files tracked by Syncthing are not stored in the cloud and it allows for decentralized, read-write architecture (different from rsync which uses a master-slave relationship).

Configuration
Install Syncthing on your pi / laptop from: https://syncthing.net/ Once done, you will have a config.xml file, which can be edited via terminal or through the web GUI interface. Because the pi can't access the browser GUI, you can change the config file to add the GUI port address from 127... to 0.0.0.0 served on Apache web server. Then you can look at the pi's GUI remotely from your machine's browser. This way you can check the Syncthing device ID of your pi, which you will need to exchange with all the other nodes running the XPPL. Alternatively, you can add device keys via terminal in the config file.

Looking at the config file (or on the GUI), each folder/directory you want to sync is described here in separate elements. The following attributes may be set on a folder element:

id - The folder ID, must be unique. (mandatory) label The label of a folder is a human readable and descriptive local name. May be different on each device, empty, and/or identical to other folder labels. (optional)

path - The path to the directory where the folder is stored on this device; not sent to other devices. (mandatory)

type - Controls how the folder is handled by Syncthing. The default mode is readwrite - sending local and accepting remote changes. The folders we want Syncthing to track are the /uploads and /cover folder.

Once you exchange keys with the other nodes in the XPPL, you'll see these two folders appear. These folders will be synced at regular intervals. This means that any file uploads you make while on your instance of the XPPL, will be synced throughout all the nodes that are online (except the server).

More documentation on how Syncthing works here.

Tinc VPN

Tinc is a Virtual Private Network (VPN) daemon that uses tunnelling and encryption to create a secure private network between hosts on the Internet. We use tinc to provide access to the library from an external network. The client librarians can then access the library through a tunnel created by the server. You can find the documentation here.

A tunnel

Setting up an instance from scratch

To be continued

The lib.xpub.nl setup

Currently, the catalogue is running on lib.xpub.nl. Accessing the library through this URL gives read-only access to book details and the stacks that have been created.