User:Alexander Roidl/neuralnetsanddeeplearning: Difference between revisions

From XPUB & Lens-Based wiki
No edit summary
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 2: Line 2:
Machine Learning is a term used to describe certain algorithms that »learn«. But learning can be seen as a quite broad term, for some applications it really comes down to a statistical calculation
Machine Learning is a term used to describe certain algorithms that »learn«. But learning can be seen as a quite broad term, for some applications it really comes down to a statistical calculation


Terms:
Common Terms:
* Generalisation
* Generalisation
** means how well the algorithm performs on unknown data, meaning how well can the algorithm generalise?
* Overfitting
* Overfitting
** To many features, so the algorithms will only work on the sample data
* Unterfitting
* Unterfitting
** To less features, so the algorithms will apply to any data.
* Features
* Features
** Mostly the input data. Certain aspects of the investigated object that is specific about the problem.
* Error
** How wrong the algorithm is, connected to:
* Loss
** function that calculates the error
* Feed-forward
** One run through the network
* Back-propagation
** adjusting the weights of the network according to the derivative function of the estimated and the actual output.


==K–Nearest Neighbour==
==K–Nearest Neighbour==
Line 23: Line 35:


==Neural Networks==
==Neural Networks==
Same as Deeplearning
''Also known as Deeplearning''


Uses different Layers and a networked structure to find the best values for the wanted result.  
Uses different Layers and a networked structure to find the best values for the wanted result.
 
It is working in »loops« using Feedforward -> Backpropagation -> FFD -> BPG in x many epochs.
Feedforward -> Backpropagation …


===Neurons===
It consists of a lot of so called neurons.
It consists of a lot of so called neurons.
Each neuron holds a value, that is being calculated according to the activation function.


There are different kind of neurons:
There are different kind of neurons:
Line 35: Line 48:
* Sigmoid: outputs a value between 0 and 1: f(x) = 1/1+e^(-x)
* Sigmoid: outputs a value between 0 and 1: f(x) = 1/1+e^(-x)
* ReLU: same as Sigmoid but cuts values: -1<x>1
* ReLU: same as Sigmoid but cuts values: -1<x>1
* … other functions, that basically map a value to -1 to +1 (tanh)
====Feed-forward====
the calculation of the neuron:
# taking all input–weights and multiple with the input
# take the sum of that (add bias)
# take that sum and run it through the activation function
====Back-propagation====
# Take the current output of the network and substract from wished value (from dataset)
# calculate the derivative of the sumfunction (see FFD) and the expected value
# map that to the values of the weights of the previous layer and substract the difference (delta)
# do the same for all the previous layers




===Architectures===
So if you take different or multiple of those networks you end up with one of the following popular concepts:
So if you take different or multiple of those networks you end up with one of the following popular concepts:
===LSTM: Long Short-Term Memory===
====Feed forward networks====
Stores last results in a neural network itself
Simple implementation of a neural network, see above.
 
====Recurrent Neural Networks====
Connects different neurons to implement some sort of memory.


====LSTM: Long Short-Term Memory====
Stores last results in a neural network itself.
Good for text-generation (prevents repeating)
Good for text-generation (prevents repeating)


===Recurrent Neural Networks===
====CNN: Convolutional Neural Networks====
Especially for images. Divides the data in smaller bits to recognise patterns in different contexts.


====GAN: Generative Adversarial Network====
Uses two neural networks that oppose each other. One generates random data, the other one figures if it is fake or real. So the first one is trained to fool the discriminator.


===CNN: Convolutional Neural Networks===
Especially for images


===GAN: Generative Adversarial Network===
 
Uses two neural networks that oppose each other. One generates random data, the other one figures if it is fake or real. So the first one is trained to fool the discriminator.
==Frameworks==
* [https://www.tensorflow.org/ Tensorflow]
* [https://keras.io/ Keras]
* http://caza.la/synaptic/
 
==Resources==  
* http://ml4a.github.io/ml4a/
* https://deeplearning4j.org/
* http://neuralnetworksanddeeplearning.com/chap1.html
* https://www.youtube.com/watch?v=aircAruvnKk Learning NN
 
Databases:
* https://www.cs.toronto.edu/~kriz/cifar.html
* http://www.image-net.org/
* http://cocodataset.org/#explore
* http://yann.lecun.com/exdb/mnist/
* http://host.robots.ox.ac.uk/pascal/VOC/
* http://academictorrents.com/details/71631f83b11d3d79d8f84efe0a7e12f0ac001460
 
Artworld:
* http://obvious-art.com/about-us.html
* https://twitter.com/DrBeef_/status/1055285640420483073 (copied from)
* https://superrare.co/market Market for selling digital art
 
Books:
* Introduction to Machine Learning with Python: A Guide for Data Scientists
* https://www.manning.com/books/deep-learning-with-python

Latest revision as of 16:38, 8 December 2018

Machine Learning

Machine Learning is a term used to describe certain algorithms that »learn«. But learning can be seen as a quite broad term, for some applications it really comes down to a statistical calculation

Common Terms:

  • Generalisation
    • means how well the algorithm performs on unknown data, meaning how well can the algorithm generalise?
  • Overfitting
    • To many features, so the algorithms will only work on the sample data
  • Unterfitting
    • To less features, so the algorithms will apply to any data.
  • Features
    • Mostly the input data. Certain aspects of the investigated object that is specific about the problem.
  • Error
    • How wrong the algorithm is, connected to:
  • Loss
    • function that calculates the error
  • Feed-forward
    • One run through the network
  • Back-propagation
    • adjusting the weights of the network according to the derivative function of the estimated and the actual output.

K–Nearest Neighbour

K = Variable

Plots Features as Vectors and sorts them in space > Clusters them

The new datapoint is also put in this space and the K nearest points are used to categorise the new point.

Linear models

Draw a linear border between 2 or more classes

Decision Tree

Cain of if / else statements


Neural Networks

Also known as Deeplearning

Uses different Layers and a networked structure to find the best values for the wanted result. It is working in »loops« using Feedforward -> Backpropagation -> FFD -> BPG … in x many epochs.

Neurons

It consists of a lot of so called neurons. Each neuron holds a value, that is being calculated according to the activation function.

There are different kind of neurons:

  • Perceptron: outputs 0 or 1
  • Sigmoid: outputs a value between 0 and 1: f(x) = 1/1+e^(-x)
  • ReLU: same as Sigmoid but cuts values: -1<x>1
  • … other functions, that basically map a value to -1 to +1 (tanh)

Feed-forward

the calculation of the neuron:

  1. taking all input–weights and multiple with the input
  2. take the sum of that (add bias)
  3. take that sum and run it through the activation function

Back-propagation

  1. Take the current output of the network and substract from wished value (from dataset)
  2. calculate the derivative of the sumfunction (see FFD) and the expected value
  3. map that to the values of the weights of the previous layer and substract the difference (delta)
  4. do the same for all the previous layers


Architectures

So if you take different or multiple of those networks you end up with one of the following popular concepts:

Feed forward networks

Simple implementation of a neural network, see above.

Recurrent Neural Networks

Connects different neurons to implement some sort of memory.

LSTM: Long Short-Term Memory

Stores last results in a neural network itself. Good for text-generation (prevents repeating)

CNN: Convolutional Neural Networks

Especially for images. Divides the data in smaller bits to recognise patterns in different contexts.

GAN: Generative Adversarial Network

Uses two neural networks that oppose each other. One generates random data, the other one figures if it is fake or real. So the first one is trained to fool the discriminator.


Frameworks

Resources

Databases:

Artworld:

Books: