2017-07-04

RNN

首先了解了RNN，有很多的教程，这里大概贴几个网址
https://www.yunaitong.cn/understanding-lstm-networks.html
http://feisky.xyz/machine-learning/rnn/
还有莫烦的教程

我自己尝试着用keras写RNN的例子，参考了莫烦的教程，但是用莫烦的程序得到的结果特别不理想

然后就尝试着用开发者fchollet的例子
https://github.com/fchollet/keras/blob/8f4d6fc3fa6cd35a36de190a5e44ab4817cc68e8/examples/mnist_irnn.py

用f大神的例子运行速度特别慢，运行结果如图所示，结果也不理想

然后我看了下代码，代码用的是784个timestep，一个量一个量传的。

X_train = X_train.reshape(X_train.shape[0], -1, 1)
X_test = X_test.reshape(X_test.shape[0], -1, 1)

于是我把输入进行了修改， 变成28个timestep, 每次输入的量是28个量

x_train = x_train.reshape(-1, 28, 28)
x_test = x_test.reshape(-1, 28, 28)

这样运行的速度就快多了，但是运行效果还是不理想

我百思不得其解，然后我自己重新写代码，发现在f大神的代码中有learning rate, 有了LR，运行结果总是60~70%，当我去掉LR时，运行结果就在96%左右

附上代码

#!/usr/bin/python
# -*- coding:utf8 -*-

from __future__ import print_function

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.layers import SimpleRNN
from keras import initializers
from keras.optimizers import RMSprop

batch_size = 50
num_classes = 10
epochs = 100
hidden_units = 50




# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = x_train.reshape(-1, 28, 28)
x_test = x_test.reshape(-1, 28, 28)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')


# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

print('Evaluate RNN...')
model = Sequential()
model.add(SimpleRNN(hidden_units,
                    kernel_initializer=initializers.RandomNormal(stddev=0.001),
                    recurrent_initializer=initializers.Identity(gain=1.0),
                    activation='relu',
                    input_shape=x_train.shape[1:]))
model.add(Dense(num_classes))
model.add(Activation('softmax'))
rmsprop = RMSprop()
model.compile(loss='categorical_crossentropy',
              optimizer=rmsprop,
              metrics=['accuracy'])

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test))

scores = model.evaluate(x_test, y_test, verbose=0)
print('RNN test score:', scores[0])
print('RNN test accuracy:', scores[1])

PhysioNet

下载了几篇文章

https://www.researchgate.net/publication/266963784_EEG-Based_Classification_of_Imagined_Fists_Movements_using_Machine_Learning_and_Wavelet_Transform_Analysis