L1. Deep Learning and Financial Data Analysis: Intro

TOC

1. Overview 1.1 Deep learning, machine learning and AI 1.2 Why go deep 2. From TLU to Perceptron 2.1 M-P unit 2.2 Perceptron

1. Overview

1.1 Deep learning, machine learning and AI

1.2 Why go deep

Handle Complicated Problems

Data are often high-dimensional

There is a huge amount of structure in the data, but the structure is too complicated to be represented by a simple model

Insufficient depth can require more computational elements than architectures depth matches the task

Deep nets provide simpler but more descriptive model of many problems

Compared with shallow&wide networks which are good at memorization, deep networks are good at generalization

Hierarchical Structures and Modularization

Do not put anything in the main function

2. From TLU to Perceptron

2.1 M-P unit

If at least one of is 1, the unit is inhibited and

Otherwise the total excitation is computed and compared with the threshold of the unit (if then )

If the unit fires a 1
If the result is 0

Note that inputs of M-P units are binaries.

Synthesis of Boolean functions using B-P units

Boolean function:

Some other logical functions are shown as below

All logical functions can be implemented with a network composed of units which exclusively compute the AND, OR and NOT functions. An example is shown as below

2.2 Perceptron

Modify M-P units to process real valued inputs. Add weights to the input connections of the M-P unit.

Further, a supervised learning algorithm is proposed: For each data points and the corresponding labels

Calculate the actual output

Update the weights: , where is the learning rate

Convergence

Proposition. If the training set is linearly separable, then the perceptron is guaranteed to converge. Furthermore, there is an upper bound on the number of times the perceptron will adjust its weights during the training.