User:Rita Graca/trimester6/technicaldocumentation

From XPUB & Lens-Based wiki

Introduction

In the forefront of the fight against hate, there are users committed to creating better social media experiences for them and for others. These users offer support with their work on moderation, technical knowledge, emotional labour, and many others.


Technical knowledge is relevant to:

  • use moderation tools
  • adapt, fork, customise moderation tools
  • create new tools
  • adapt, fork, customise social platforms
  • create new platforms
  • use software and hardware
  • question autonomy
  • experiment with protocols
  • self-host
  • understand tutorials
  • write and send reports


These actions can be challenging, and they demand time, money and effort from a community. Networks of Care goes through some of these technical hurdles, it exposes processes and documents the experience of creating and maintaining this project.

Networks of Care is hosted on a Raspberry Pi and uses MediaWiki software to organise the content. It uses extensions such as Semantic MediaWiki and Page Forms. The main page shows a diagram that uses the API to get access to wiki features. It also uses Leaflet, a JavaScript library.


Setting up the Raspberry Pi

In my computer:

SD card

1. Format for FAT32 (I used disk utility on mac. Format to MS_DOS). Use more than 8gb, mediawiki can become heavy.


2. Flash ISO to SD card (Raspbian Buster Light, https://www.raspberrypi.org/downloads/raspbian/ ) with a mounter (I've used etcher https://www.balena.io/etcher/ )


With raspi connected to power, internet, screen and keyboard:


01 Default login

Login: pi

Password: raspberry


02 SSH config 

   $ cd /boot

   $ sudo touch ssh

   $ sudo reboot


03 Check IP address 

   $ hostname -i

Response: 192.0.0.0


04 My keyboard wasn't working properly.

I had to change language of keyboard to US instead of UK. 

   $ sudo raspi-config

   Localisation Options

   Change Keyboard Layout

   Choose US

After using the keyboard I changed back to UK to avoid errors.


05 Change hostname 

   $ ssh pi@192.0.0.0

   $ sudo raspi-config

   Choose item no. 2 on the menu: Network Options

   Rename hostname to: newhostname

   $ reboot


06 Add user 

   $ sudo adduser name


07 Add new user to sudo 

   $ sudo adduser name sudo


08 You should be able to login with: 

   $ ssh name@newhostname


09 I had an error with known hosts. I removed cached key on my computer: 

   $ ssh-keygen -R 192.000.0.000


10 Remove default pi user 

   $ sudo deluser pi


10 Use key-based authentication:


10.1 Add .ssh keys to RPI

https://www.raspberrypi.org/documentation/remote-access/ssh/passwordless.md#copy-your-public-key-to-your-raspberry-pi


10.2 Edit file 

   $ sudo nano /etc/ssh/sshd_config

   Uncomment PassowrdAuthentication and set it to no;

   Uncomment PermitRootLogin prohibit-passowrd;


10.3 Reload SSH 

   $ sudo /etc/init.d/ssh restart


10.4 Reboot RPi 

   $ sudo reboot


10.5 In my computer: 

   $ cd ~/.ssh

   $ sudo nano config


Add to the document: 

   Host newhostname
       User name
       Hostname 192.000.0.0
       Port 22
       Identityfile /Users/ritagraca/.ssh/id_rsa
       Serveraliveinterval 30


10.6 To ssh to my pi now: 

   $ ssh newhostname


11. Set static ip address for LAN 

   $ sudo nano /etc/dhcpcd.conf


Edit this part: 

   #Static IP text block
       interface eth0
       static ip_address=192.000.0.00
       static routers=192.000.0.0
       static domain_name_servers=8.8.8.8 8.8.4.4


Note:

I shut down my pi, IP changed and couldn't ssh to it. If this happened already, connect pi again to screen, find new IP with $ config command, and put new IP in ~/.ssh, step 10.5.


Installing Mediawiki on the pi

1. Install Apache 

   $ sudo apt update

   $ sudo apt install apache2 -y

I will remove Apache and install Ngnix in the next steps, see Fixing my very slow MediaWiki.


2. Download Mediawiki 

   $ mkdir Downloads

   $ cd Downloads

   $ wget https://releases.wikimedia.org/mediawiki/1.34/mediawiki-1.34.0.tar.gz (check the version)

   $ tar -xvzf mediawiki-1.34.0.tar.gz


3. Move 

   $ sudo mkdir /var/lib/mediawiki

   $ sudo mv mediawiki-*/* /var/lib/mediawiki


4. Linking 

   $ cd /var/www/html

   $ sudo ln -s /var/lib/mediawiki mediawiki


5. Updating 

   $ sudo apt-get update

   $ sudo apt-get install apache2 mysql-server php php-mysql libapache2-mod-php php-xml php-mbstring


5.1 Error: Package 'mysql-server' has no installation candidate

Fix with: 

   $ sudo apt install mariadb-server-10.3 php-mysql


5.2 Error: MySQL didn't ask for password so: 

   $ sudo mysql_secure_installation

It will ask for password now.


6. Database 

   $ sudo mysql -u root -p

Enter password. Type: 

   mysql> CREATE DATABASE wiki;
   mysql> USE wiki;

   mysql> CREATE USER 'name'@'localhost' IDENTIFIED BY 'yourpassword';
   mysql> GRANT ALL PRIVILEGES ON wiki.* TO 'name'@'localhost'; 
   mysql> FLUSH PRIVILEGES;
   mysql> quit


7. Open browser 

   http://ip_address_of_pi/mediawiki

On the website, it will ask for database connection settings: 

   localhost
   database: wiki
   user: name
   password: yourpassword

Continue installation.


8. After setting up, it will ask to save LocalSettings.php. Download to your computer.


9. Move file from your computer to pi.

In your computer: 

   $ scp LocalSettings.php name@hostname:/home/name/Downloads


10. Move from pi to the right place.

In your pi: 

   $ sudo mv ~/Downloads/LocalSettings.php /var/lib/mediawiki/


some more help to setup mediawiki http://willbeone-techtips.co.za/2018/02/12/installing-mediawiki-raspberry-pi3/


Fixing my very slow MediaWiki

My wiki is painfully slow. Things I'll try:

1. Switch from Apache to Ngnix.

2. Swap to SanDisk Ultra 64gb a2. Update — The sd card made the most difference. Amazing.


0. Backing up my raspbery pi image

First, I backed up my raspbery pi image. Tutorial for mac: https://medium.com/@ccarnino/backup-raspberry-pi-sd-card-on-macos-the-2019-simple-way-to-clone-1517af972ca5


1. Switching from Apache to Ngnix

1.1 Uninstalling Apache: 

   $ sudo apt-get --purge remove apache2

   $ sudo apt-get autoremove


1.2 Installing Ngnix: 

   $ sudo apt update

   $ sudo apt install nginx

   $ sudo /etc/init.d/nginx start


Go to ip address and see the Ngnix starting message.

1.3 Additional — Install PHP:

https://www.raspberrypi.org/documentation/remote-access/web-server/nginx.md


1.4 I uninstalled mediawiki: 

   $ rm -rf /var/www/mediawiki


1.5 Started from step Installing Mediawiki


Backing up the wiki

File system

Downloaded the folder and all contents of “mediawiki”

Database backup

Used Mysqldump from the command line. 

  mysqldump -h hostname -u userid -p --default-character-set=whatever dbname > backup.sql

  mysqldump -h localhost -u rita -p --default-character-set=binary wiki > backup.sql

XML backup

"XML dumps contain the content of the wiki (wiki pages with all their revisions), without the site-related data (they do not contain user accounts, image metadata, logs, etc)." https://www.mediawiki.org/wiki/Manual:Backing_up_a_wiki 

  php dumpBackup.php --full > dump.xml

https://www.mediawiki.org/wiki/Manual:DumpBackup.php


Extensions

Semantic MediaWiki

1. Download composer

More here https://getcomposer.org/doc/00-intro.md#installation-nix


1.1 Download the installer to the current directory 

   $ php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"


1.2 Verify the installer SHA-384 

   $ php -r "if (hash_file('sha384', 'composer-setup.php') === 'e0012edf3e80b6978849f5eff0d4b4e4c79ff1609dd1e613307e16318854d24ae64f26d17af3ef0bf7cfb710ca74755a') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"


1.3 Run the installer 

   $ php composer-setup.php


1.4 Remove the installer 

   $ php -r "unlink('composer-setup.php');"


1.5 Global Access

Now that I have composer.phar, I will put it in /usr/local/bin to have global access. 

   $ sudo mv composer.phar /usr/local/bin/composer

This means that instead of starting with $ php composer.phar, I just write $ composer.

Some more help to install composer https://www.semantic-mediawiki.org/wiki/Help:Installation/Using_Composer_with_MediaWiki_1.25%2B


2. Create and edit the file composer.local.json in /var/www/html/mediawiki. 

   $ cd /var/www/html/mediawiki

   $ sudo nano composer.local.json


paste this code: 

   {
           "require": {
                     "mediawiki/semantic-media-wiki": "~3.1"
           }
   }


3. Run the update script 

   $ composer update --no-dev


3.1 Change permissions:

Gives error: 

   file_put_contents(./composer.lock): failed to open stream: Permission denied

Give permissions: 

   $ chmod -R 777 ./


4. Maintenance script 

   $ php maintenance/update.php


5. Enable Semantic MediaWiki on LocalSetting.php

Write in the end of the file: 

   enableSemantics( ' http://192.000.0.0/mediawiki/ ' );


6. Repeat step 4.

I had a error with database. I had to run the update script again. If when running this script there's error with permissions, just change permissions, step 3.1


7. Uploading files to the server

Check first folder permissions, and change if necessary: 

   $ chmod -R 777 ./


To allow bigger files:

7.1. Edit this file 

       $ sudo nano /etc/nginx/nginx.conf


7.2 Add this in the bottom of the http part: 

       # set client body size to 2M
       client_max_body_size 2M;


7.3 Reload the server 

       $ service nginx reload


Understanding Properties

Basic way of entering semantic data: 

  [[Property + name of property::datatype]

  [[Has + name of property::datatype]]

  The creator of this text is [[Has Author:Rita]]


There are special kinds of properties: Has, Media type, Creation date, Language code, etc.

See all here: https://www.semantic-mediawiki.org/wiki/Help:Special_properties


Possible datatypes: Number, page, temperature, date, text, page, URL, telephone number, etc.

See all of 15 here: https://www.semantic-mediawiki.org/wiki/Help:Property_declaration


Choose a fitting datatype for your property.

  • For example, the datatype Telephone number only holds numbers, if you try to write something like Hello, it will give an error.
  • For me, it will be useful the datatype Text. (Used to store strings of arbitrary length, i.e. text.)
  • Default datatype is Page. (Holds names of wiki pages, and displays them as a link).


Creating semantic information

Some information I want to store for every page in Archiving:

Field — xxxx, xxxx, xxxx

Scope — online, offline


Steps:

Create property Field and Scope. 

 Special:SpecialPages > Page Forms > Create a property

Create template Archiving. A template sets the display of the data on a page. 

 Special:SpecialPages > Page Forms > Create a template.

Create form Archiving. A form allow users to easily add and edit pages. 

 Special:SpecialPages > Page Forms > Create a form

Now go to forms. You can create a page. Every time you go to forms and select "Archiving" form you can create a new page using the form. The forms asks you to fill the properties.

(more here: https://www.mediawiki.org/wiki/Extension:Page_Forms/Quick_start_guide )

Edit with form

To have the "edit with form" appear when you create a page without going to the forms page in Special Pages:

Create a category (I created one called "Archiving") and select a default form. 

 {{#default_form:Archiving}}.

Every time you create a page within the category, you'll have the option of "edit with form".


Queries

https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2017/Introduction_to_Semantic_MediaWiki


Templates

In the category Archiving, I wanted to annotate the Codes of Conduct I was collecting. I saw some common patterns in the texts, and I wanted to make them visible. The main themes were:

  • Intentions
  • Expected behaviour
  • Unacceptable behaviour
  • Behaviour to avoid
  • Enforcement
  • Support
  • Links

First, I created properties. This allowed me to annotate the text in-line. 

  [[Intentions:: We want to provide a safe place.]]

But I wanted to make the annotations visible. So, I created templates which allowed me to add style.


Template example: 

  [[Intentions::{{{1}}}|<span style="border-bottom: 2px solid DarkGrey;">{{{1}}}</span>]]


Template usage in text: 

  {{Intentions | We want to provide a safe place.}}


Result:

The sentence will appear with a grey border, the style I wanted.

It will also be added the property Intentions.


Later, I felt the colours needed a caption so the readers could understand the meaning of each colour. I used Extensions SimpleTooltip.

I also used the Extension ParserFunctions.

Page Forms

This enables forms, useful to add semantic annotations to pages.


1. Download

https://github.com/wikimedia/mediawiki-extensions-PageForms/archive/4.8.zip

Put it in /var/www/html/mediawiki/extensions.

You shoud now have a directory called PageForms in your extensions folder.


2. Add the following line to your LocalSettings.php file: 

   wfLoadExtension( 'PageForms' );


WikiEditor

This enables the edit toolbar.


1. Download

https://www.mediawiki.org/wiki/Extension:WikiEditor

Put it in /var/www/html/mediawiki/extensions.

You should now have a directory called PageForms in your extensions folder.


2. Add the following line to your LocalSettings.php file: 

   wfLoadExtension( 'WikiEditor' );

EmbedVideo

To allow video files, such as .mp4, on the wiki.

https://www.mediawiki.org/wiki/Extension:VideoPlayer


SimpleTooltip

I'm using this extension to add information on the annotated pages in Archiving. Semantic MediaWiki allows to add tooltips, without any more extensions, like this: 

 {{#info: <text> |<icon> |max-width=<width> |theme=<theme> }}

However, for me this was extremely buggy. With SimpleTooltip it works better. This extension allows for adding extra information when hovering an icon. I'm using this as a way to decode the meaning of the colours used for annotation.

Annotation preview.gif


For download and documentation go here: https://www.mediawiki.org/wiki/Extension:SimpleTooltip

ParserFunctions

I used this extension for the #if function. In my templates, it's common that the form is not filled completely. For example, the information on the number of members of a community is usually hard to get. In those cases, I didn't want the table to appear with empty lines. This function allows me to say #if there is content, add to the table, otherwise don't show anything. 

{{#if: condition | if true | if false }}
{{#if: there is members info | put in table | don't put in table) }}


In the template it looks like this: 

|-
{{ #if: {{{Members|}}} |
! Members
{{!}} [[Members::{{{Members|}}}]] | }}
|-


Note: the {{!}} is a magic word to use the pipe character for the table cell contents without messing with the #if function own pipe characters.

Creating a diagram for navigation/main page

From the beginning, I wanted to show the content of the Wiki visually, so I started imagining a landing page for the project. A visualisation of the pages inside the wiki could help to understand the structure of the project.

As the idea was developing, it became important to also show the connections between content, people and projects. For example, if I was archiving a Code of Conduct of a community and could then make questions to a moderator of that same group, I wanted to make visible that storyline. In this way, I began experimenting with different ways of visualising information.


Playing with Mediawiki Extensions

Extension "Semantic Result Formats"


Extension "Maps" (using leaflet from Maps Extension)

I couldn't find an extension that actually let me do what I wanted and have some creative freedom.


Doing the diagram outside the wiki using Mediawiki API

Steps:

  • Create my diagram on leaflet
  • Link Mediawiki Semantic wiki API to leaflet
  • Include diagram to wiki
  • Populate, populate, populate


API

The web server is using HTTP requests to communicate to a publicly available URL endpoint containing JSON data. You need to use CORS or JSONP, I'll try to use JSONP for now. (note that JSONP is inherently read-only)

A request consists of an endpoint and a set of parameters. Example to list all pages in category Archiving:

http://192.000.0.0/mediawiki/api.php?action=query&format=json&&cmtitle=Category:Archiving&list=categorymembers

Important things:


Mediawiki API Sandbox:

https://www.mediawiki.org/wiki/Special:ApiSandbox


Semantic Wiki API Sandbox:

https://www.semantic-mediawiki.org/wiki/Special:ApiSandbox

Overall the sandbox is very useful to create queries with properties.


Limit problem

The API was limiting my query to 50 pages. The solution was to add |limit=100 to the query. 

var params ="action=ask&format=json&query=[[Category%3ADiagram]] |%3FCategory |%3FConnection |%3FField|limit=100";


In case I need to query even more pages, in the LocalSettings.php I'll add: 

$smwgQMaxInlineLimit = 1500; $smwgQMaxInlineLimitSets = 1500; # maximal number of results ever retrieved by queries
Retrieved from "https://pzwiki.wdka.nl/mw-mediadesign/index.php?title=User:Rita_Graca/trimester6/technicaldocumentation&oldid=175051"