超参数调优

Keras 深度学习入门

Miguel Esteban

Data Scientist & Founder

神经网络超参数

层数
每层神经元数
层顺序
层激活函数
批大小
学习率
优化器
…

Sklearn 回顾

# Import RandomizedSearchCV
from sklearn.model_selection import RandomizedSearchCV

# Instantiate your classifier
tree = DecisionTreeClassifier()

# Define a series of parameters to look over
params = {'max_depth':[3,None], "max_features":range(1,4), 'min_samples_leaf': range(1,4)}

# Perform random search with cross validation
tree_cv = RandomizedSearchCV(tree, params, cv=5)

tree_cv.fit(X,y)

# Print the best parameters
print(tree_cv.best_params_)

{'min_samples_leaf': 1, 'max_features': 3, 'max_depth': 3}

将 Keras 模型转为 Sklearn 估计器

# Function that creates our Keras model
def create_model(optimizer='adam', activation='relu'):
    model = Sequential()
    model.add(Dense(16, input_shape=(2,), activation=activation))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(optimizer=optimizer, loss='binary_crossentropy')
    return model


# Import sklearn wrapper from keras
from tensorflow.keras.wrappers.scikit_learn import KerasClassifier


# Create a model as a sklearn estimator
model = KerasClassifier(build_fn=create_model, epochs=6, batch_size=16)

交叉验证

# Import cross_val_score
from sklearn.model_selection import cross_val_score

# Check how your keras model performs with 5 fold crossvalidation
kfold = cross_val_score(model, X, y, cv=5)


# Print the mean accuracy per fold
kfold.mean()

0.913333

# Print the standard deviation per fold
kfold.std()

0.110754

神经网络超参数调优技巧

随机搜索优于网格搜索
不要用太多 epoch
使用数据集的较小样本
试验批大小、激活函数、优化器和学习率

在 Keras 模型上进行随机搜索

# Define a series of parameters
params = dict(optimizer=['sgd', 'adam'], epochs=3, 
              batch_size=[5, 10, 20], activation=['relu','tanh'])

# Create a random search cv object and fit it to the data
random_search = RandomizedSearchCV(model, params_dist=params, cv=3)

random_search_results = random_search.fit(X, y)

# Print results
print("Best: %f using %s".format(random_search_results.best_score_,
random_search_results.best_params_))

Best: 0.94 using {'optimizer': 'adam', 'epochs': 3, 'batch_size': 10, 'activation': 'relu'}

调其他超参数

def create_model(nl=1,nn=256):
    model = Sequential()
    model.add(Dense(16, input_shape=(2,), activation='relu'))

    # Add as many hidden layers as specified in nl
    for i in range(nl):
        # Layers have nn neurons
        model.add(Dense(nn, activation='relu'))
    # End defining and compiling your model...

调其他超参数

# Define parameters, named just like in create_model()
params = dict(nl=[1, 2, 9], nn=[128,256,1000])

# Repeat the random search...

# Print results...

Best: 0.87 using {'nl': 2,'nn': 128}

来调一些网络吧！

Keras 深度学习入门