User:Alexander Roidl/buildingxppl: Difference between revisions
(→VPN) |
|||
| (16 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
[[Flask]] cookbook | [[Flask]] cookbook | ||
===Basic Structure=== | ===Basic Structure=== | ||
[[File:Screen Shot 2018-05-25 at 00.18.56.png|thumbnail|First Prototype (during wisdomtooth-incident)]] | |||
====Flask / Webinterface==== | ====Flask / Webinterface==== | ||
I created a basic flask application that provides the groundlaying interface to plugin for access to the library. | I created a basic flask application that provides the groundlaying interface to plugin for access to the library. | ||
| Line 34: | Line 35: | ||
[[File:Screen Shot 2018-05-14 at 23.58.21.png|thumbnail]] | [[File:Screen Shot 2018-05-14 at 23.58.21.png|thumbnail]] | ||
GET all books as JSON | GET all books as JSON | ||
/api/books | /api/books | ||
GET a single book by id | GET a single book by id | ||
/api/books/<id> | /api/books/<id> | ||
| Line 47: | Line 51: | ||
This is an important part of the infrastructure of the library. | This is an important part of the infrastructure of the library. | ||
Because of its importance it is also directly integrated in the webinterface. It makes it possible to export a csv and import it later back in. (With a simple and to be further developed merge functionality) | Because of its importance it is also directly integrated in the webinterface. It makes it possible to export a csv and import it later back in. (With a simple and to be further developed merge functionality) | ||
The export / import makes it possible to potentially go offline and switch to your offline database. If you make changes there you can reimport them as soon as you are online again. | |||
[[File:Screen Shot 2018-06-10 at 12.13.35.png|frameless]] | [[File:Screen Shot 2018-06-10 at 12.13.35.png|frameless]] | ||
| Line 91: | Line 97: | ||
==rqlite== | ==rqlite== | ||
[[File:Screen Shot 2018-06-05 at 15.45.35.png|thumbnail|RQlite on 2 instances]] | [[File:Screen Shot 2018-06-05 at 15.45.35.png|thumbnail|RQlite on 2 instances]] | ||
[[Rqlite]] | |||
Installed rqlite on 2 instances. | |||
Consensus protocol, has an interesting construction. | |||
==VPN== | ==VPN== | ||
All instances are on the same network via a vpn server. This makes it possible to connect rqlite and sync files. | All instances are on the same network via a vpn server. This makes it possible to connect rqlite and sync files. | ||
For the setup tinc vpn is used. | For the setup tinc vpn is used. | ||
[[File:Screen Shot 2018-06-05 at 16.18.02.png|frameless]] | [[File:Screen Shot 2018-06-05 at 16.18.02.png|frameless]] | ||
| Line 105: | Line 118: | ||
===Instances contribution=== | ===Instances contribution=== | ||
The scape also makes the distribution over the different instances visible. With a selector you can select a instance and see (with opacity) what books are closed to this instance. | The scape also makes the distribution over the different instances visible. With a selector you can select a instance and see (with opacity) what books are closed to this instance. | ||
===New Scape=== | |||
Zoomable with pan with d3js -> lib.xpub.nl/scape_new | |||
==(Potential) Cover== | ==(Potential) Cover== | ||
| Line 121: | Line 137: | ||
==Chat== | ==Chat== | ||
To generate more dynamic communication I created the chat based on websocket. | To generate more dynamic communication on the platform. | ||
I created the chat based on websocket. (with gradients) | |||
==Context Banner== | ==Context Banner== | ||
| Line 128: | Line 144: | ||
It shows last uploaded, last viewed book and number of books online. It tries to give context for the user. | It shows last uploaded, last viewed book and number of books online. It tries to give context for the user. | ||
[[File:Screen Shot 2018-06-01 at 23.30.02.png|frameless]] | [[File:Screen Shot 2018-06-01 at 23.30.02.png|frameless]] | ||
==XPPL namegenerator== | |||
[[File:Screen Shot 2018-06-09 at 18.59.54.png|thumbnail|Generated by generator, image by Joca]] | |||
[[User:Alexander_Roidl/xppl_name_generator|XPPL Generator]] | |||
==Documentation== | ==Documentation== | ||
For a long living library there is major documentation required. | For a long living library there is major documentation required. | ||
I contributed to the [[cookbook]] for easier access to the programming part. | I contributed to the [[cookbook]] for easier access to the programming part. | ||
[[XPPL Documentation]] | |||
[https://git.xpub.nl/xpub-lib/log.html Git] | |||
Latest revision as of 08:36, 15 June 2018
All parts built for XPPL
Flask Application
Flask cookbook
Basic Structure
Flask / Webinterface
I created a basic flask application that provides the groundlaying interface to plugin for access to the library. The basic structure is based on a SQLite (now rqlite) database. The webinterface provides a catalog in different forms (List, Grid and Scape)
The network
The library distributes over multiple instances that share the same database. They all mirror the same library and they are all able to contribute. These instances are only reachable within the local network. In contrast to that there is a public accessible catalog, that doesn't allow for contribution or download.
SQL Database
- Integration of SQLite
Relationships
- One book can have many authors or tags
- Therefore: fields like authors or tags are another list itself
- So we have relational tables for:
- Authors table
- Instances table
- Stacks table (-> Alice)
Table, show books
The table lists the books with its attributes.
Forms
Forms are built with WTForm, a flask plugin. This makes it easy to POST new books or edit books.
Add book
Edit books
Takes the existing values out of the database and puts it as defaults into the form.
API
GET all books as JSON
/api/books
GET a single book by id
/api/books/<id>
Instances
Instances mirror the whole library and make the library accessible for adding, editing and downloading. The webinterface shows all instances that are part of hosting the library. The scape and the book details page make visible where uploads come from and who contributed mostly. So you can see how much each book belongs to which instance.
Im / Export
This is an important part of the infrastructure of the library. Because of its importance it is also directly integrated in the webinterface. It makes it possible to export a csv and import it later back in. (With a simple and to be further developed merge functionality) The export / import makes it possible to potentially go offline and switch to your offline database. If you make changes there you can reimport them as soon as you are online again.
Accessibity
+-------------+ +-------+ +---------+
| | +-------------------------> | | | Outside |
| lib.xpub.nl | | WWW | +---------+ | Client |
| | <-------------------------+ | | | |
+-------------+ +---+---+ +---------+
+------------------------------+CATALOG | |
| +-------------+ |
| |
| |
| +--------------------+ |
| | | |
| +------------------------------------------------------+ | |
| | | | | |
| | +------------+ +------------+ | +------------+ | | |
| | | Instance 1 | | Instance 2 | | | Instance 3 | | | |
+ | +------------+ +------------+ | +------------+ | | |
RQLITE +--------+CATALOG +----+CATALOG +----+CATALOG | | | |
| +------------+ +------------+ | +------------+ | | |
| |FILES | |FILES | | |FILES | | | |
| | | | | | | | | | |
SYNCTHING +--------+DOWNLOAD +----+DOWNLOAD +----+DOWNLOAD | | | |
| |EDIT | |EDIT | | |EDIT | | | |
| |UPLOAD | |UPLOAD | | |UPLOAD | | | |
| +------------+ +------------+ | +------+-----+ | | |
| | | | | |
+------------------------------------------------------+ | |
VPN MESH | | | |
| | | |
| +----+----+ | |
| | Comrade +---------------+
| +---------+ |
| |
+--------------------+
LOCAL NETWORK
rqlite
Installed rqlite on 2 instances. Consensus protocol, has an interesting construction.
VPN
All instances are on the same network via a vpn server. This makes it possible to connect rqlite and sync files.
For the setup tinc vpn is used.
Scape
As an alternative view for all the books I created a scape.
Visual Distribution
The position is stored in the database, so user can sort books visually, visible for anyone.
Instances contribution
The scape also makes the distribution over the different instances visible. With a selector you can select a instance and see (with opacity) what books are closed to this instance.
New Scape
Zoomable with pan with d3js -> lib.xpub.nl/scape_new
(Potential) Cover
cover extraction
While uploading the cover gets extracted with a little python script based on PyPDF2 get cover from pdf script
generated cover
For the missing covers I was looking for a way to distinguish the different books, even there file is not there / they don't have a cover. So I created a generative potential cover that is generated by the colorhash of the title of the book -> developed further by Tash (potential pdf)
PDF to Text
Next to the cover also the text of the books gets extracted. User:Alexander_Roidl/pdf2html
Chat
To generate more dynamic communication on the platform. I created the chat based on websocket. (with gradients)
Context Banner
The banner on top of the page gives recent information about the library. It's meant to disturbe and to create a certain dynamic in the library.
It shows last uploaded, last viewed book and number of books online. It tries to give context for the user.
XPPL namegenerator
Documentation
For a long living library there is major documentation required. I contributed to the cookbook for easier access to the programming part.