User:Alexander Roidl/neuralnetsanddeeplearning: Difference between revisions
No edit summary |
No edit summary |
||
(2 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
Machine Learning is a term used to describe certain algorithms that »learn«. But learning can be seen as a quite broad term, for some applications it really comes down to a statistical calculation | Machine Learning is a term used to describe certain algorithms that »learn«. But learning can be seen as a quite broad term, for some applications it really comes down to a statistical calculation | ||
Terms: | Common Terms: | ||
* Generalisation | * Generalisation | ||
** means how well the algorithm performs on unknown data, meaning how well can the algorithm generalise? | |||
* Overfitting | * Overfitting | ||
** To many features, so the algorithms will only work on the sample data | |||
* Unterfitting | * Unterfitting | ||
** To less features, so the algorithms will apply to any data. | |||
* Features | * Features | ||
** Mostly the input data. Certain aspects of the investigated object that is specific about the problem. | |||
* Error | |||
** How wrong the algorithm is, connected to: | |||
* Loss | |||
** function that calculates the error | |||
* Feed-forward | |||
** One run through the network | |||
* Back-propagation | |||
** adjusting the weights of the network according to the derivative function of the estimated and the actual output. | |||
==K–Nearest Neighbour== | ==K–Nearest Neighbour== | ||
Line 23: | Line 35: | ||
==Neural Networks== | ==Neural Networks== | ||
''Also known as Deeplearning'' | |||
Uses different Layers and a networked structure to find the best values for the wanted result. | Uses different Layers and a networked structure to find the best values for the wanted result. | ||
It is working in »loops« using Feedforward -> Backpropagation -> FFD -> BPG … in x many epochs. | |||
Feedforward -> Backpropagation … | |||
===Neurons=== | |||
It consists of a lot of so called neurons. | It consists of a lot of so called neurons. | ||
Each neuron holds a value, that is being calculated according to the activation function. | |||
There are different kind of neurons: | There are different kind of neurons: | ||
Line 35: | Line 48: | ||
* Sigmoid: outputs a value between 0 and 1: f(x) = 1/1+e^(-x) | * Sigmoid: outputs a value between 0 and 1: f(x) = 1/1+e^(-x) | ||
* ReLU: same as Sigmoid but cuts values: -1<x>1 | * ReLU: same as Sigmoid but cuts values: -1<x>1 | ||
* … other functions, that basically map a value to -1 to +1 (tanh) | |||
====Feed-forward==== | |||
the calculation of the neuron: | |||
# taking all input–weights and multiple with the input | |||
# take the sum of that (add bias) | |||
# take that sum and run it through the activation function | |||
====Back-propagation==== | |||
# Take the current output of the network and substract from wished value (from dataset) | |||
# calculate the derivative of the sumfunction (see FFD) and the expected value | |||
# map that to the values of the weights of the previous layer and substract the difference (delta) | |||
# do the same for all the previous layers | |||
===Architectures=== | |||
So if you take different or multiple of those networks you end up with one of the following popular concepts: | So if you take different or multiple of those networks you end up with one of the following popular concepts: | ||
=== | ====Feed forward networks==== | ||
Simple implementation of a neural network, see above. | |||
====Recurrent Neural Networks==== | |||
Connects different neurons to implement some sort of memory. | |||
====LSTM: Long Short-Term Memory==== | |||
Stores last results in a neural network itself. | |||
Good for text-generation (prevents repeating) | Good for text-generation (prevents repeating) | ||
=== | ====CNN: Convolutional Neural Networks==== | ||
Especially for images. Divides the data in smaller bits to recognise patterns in different contexts. | |||
====GAN: Generative Adversarial Network==== | |||
Uses two neural networks that oppose each other. One generates random data, the other one figures if it is fake or real. So the first one is trained to fool the discriminator. | |||
=== | |||
==Frameworks== | |||
* [https://www.tensorflow.org/ Tensorflow] | |||
* [https://keras.io/ Keras] | |||
* http://caza.la/synaptic/ | |||
==Resources== | |||
* http://ml4a.github.io/ml4a/ | |||
* https://deeplearning4j.org/ | |||
* http://neuralnetworksanddeeplearning.com/chap1.html | |||
* https://www.youtube.com/watch?v=aircAruvnKk Learning NN | |||
Databases: | |||
* https://www.cs.toronto.edu/~kriz/cifar.html | |||
* http://www.image-net.org/ | |||
* http://cocodataset.org/#explore | |||
* http://yann.lecun.com/exdb/mnist/ | |||
* http://host.robots.ox.ac.uk/pascal/VOC/ | |||
* http://academictorrents.com/details/71631f83b11d3d79d8f84efe0a7e12f0ac001460 | |||
Artworld: | |||
* http://obvious-art.com/about-us.html | |||
* https://twitter.com/DrBeef_/status/1055285640420483073 (copied from) | |||
* https://superrare.co/market Market for selling digital art | |||
Books: | |||
* Introduction to Machine Learning with Python: A Guide for Data Scientists | |||
* https://www.manning.com/books/deep-learning-with-python |
Latest revision as of 16:38, 8 December 2018
Machine Learning
Machine Learning is a term used to describe certain algorithms that »learn«. But learning can be seen as a quite broad term, for some applications it really comes down to a statistical calculation
Common Terms:
- Generalisation
- means how well the algorithm performs on unknown data, meaning how well can the algorithm generalise?
- Overfitting
- To many features, so the algorithms will only work on the sample data
- Unterfitting
- To less features, so the algorithms will apply to any data.
- Features
- Mostly the input data. Certain aspects of the investigated object that is specific about the problem.
- Error
- How wrong the algorithm is, connected to:
- Loss
- function that calculates the error
- Feed-forward
- One run through the network
- Back-propagation
- adjusting the weights of the network according to the derivative function of the estimated and the actual output.
K–Nearest Neighbour
K = Variable
Plots Features as Vectors and sorts them in space > Clusters them
The new datapoint is also put in this space and the K nearest points are used to categorise the new point.
Linear models
Draw a linear border between 2 or more classes
Decision Tree
Cain of if / else statements
Neural Networks
Also known as Deeplearning
Uses different Layers and a networked structure to find the best values for the wanted result. It is working in »loops« using Feedforward -> Backpropagation -> FFD -> BPG … in x many epochs.
Neurons
It consists of a lot of so called neurons. Each neuron holds a value, that is being calculated according to the activation function.
There are different kind of neurons:
- Perceptron: outputs 0 or 1
- Sigmoid: outputs a value between 0 and 1: f(x) = 1/1+e^(-x)
- ReLU: same as Sigmoid but cuts values: -1<x>1
- … other functions, that basically map a value to -1 to +1 (tanh)
Feed-forward
the calculation of the neuron:
- taking all input–weights and multiple with the input
- take the sum of that (add bias)
- take that sum and run it through the activation function
Back-propagation
- Take the current output of the network and substract from wished value (from dataset)
- calculate the derivative of the sumfunction (see FFD) and the expected value
- map that to the values of the weights of the previous layer and substract the difference (delta)
- do the same for all the previous layers
Architectures
So if you take different or multiple of those networks you end up with one of the following popular concepts:
Feed forward networks
Simple implementation of a neural network, see above.
Recurrent Neural Networks
Connects different neurons to implement some sort of memory.
LSTM: Long Short-Term Memory
Stores last results in a neural network itself. Good for text-generation (prevents repeating)
CNN: Convolutional Neural Networks
Especially for images. Divides the data in smaller bits to recognise patterns in different contexts.
GAN: Generative Adversarial Network
Uses two neural networks that oppose each other. One generates random data, the other one figures if it is fake or real. So the first one is trained to fool the discriminator.
Frameworks
Resources
- http://ml4a.github.io/ml4a/
- https://deeplearning4j.org/
- http://neuralnetworksanddeeplearning.com/chap1.html
- https://www.youtube.com/watch?v=aircAruvnKk Learning NN
Databases:
- https://www.cs.toronto.edu/~kriz/cifar.html
- http://www.image-net.org/
- http://cocodataset.org/#explore
- http://yann.lecun.com/exdb/mnist/
- http://host.robots.ox.ac.uk/pascal/VOC/
- http://academictorrents.com/details/71631f83b11d3d79d8f84efe0a7e12f0ac001460
Artworld:
- http://obvious-art.com/about-us.html
- https://twitter.com/DrBeef_/status/1055285640420483073 (copied from)
- https://superrare.co/market Market for selling digital art
Books:
- Introduction to Machine Learning with Python: A Guide for Data Scientists
- https://www.manning.com/books/deep-learning-with-python