하이퍼파라미터 튜닝

Keras로 시작하는 딥 러닝

Miguel Esteban

Data Scientist & Founder

신경망 하이퍼파라미터

층 수
층당 뉴런 수
층 순서
활성화 함수
배치 크기
학습률
옵티마이저
...

Sklearn 복습

# Import RandomizedSearchCV
from sklearn.model_selection import RandomizedSearchCV

# Instantiate your classifier
tree = DecisionTreeClassifier()

# Define a series of parameters to look over
params = {'max_depth':[3,None], "max_features":range(1,4), 'min_samples_leaf': range(1,4)}

# Perform random search with cross validation
tree_cv = RandomizedSearchCV(tree, params, cv=5)

tree_cv.fit(X,y)

# Print the best parameters
print(tree_cv.best_params_)

{'min_samples_leaf': 1, 'max_features': 3, 'max_depth': 3}

Keras 모델을 Sklearn 추정기로 변환

# Function that creates our Keras model
def create_model(optimizer='adam', activation='relu'):
    model = Sequential()
    model.add(Dense(16, input_shape=(2,), activation=activation))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(optimizer=optimizer, loss='binary_crossentropy')
    return model


# Import sklearn wrapper from keras
from tensorflow.keras.wrappers.scikit_learn import KerasClassifier


# Create a model as a sklearn estimator
model = KerasClassifier(build_fn=create_model, epochs=6, batch_size=16)

교차 검증

# Import cross_val_score
from sklearn.model_selection import cross_val_score

# Check how your keras model performs with 5 fold crossvalidation
kfold = cross_val_score(model, X, y, cv=5)


# Print the mean accuracy per fold
kfold.mean()

0.913333

# Print the standard deviation per fold
kfold.std()

0.110754

신경망 하이퍼파라미터 튜닝 팁

그리드 서치보다 랜덤 서치를 권장합니다
에폭 수를 과하게 늘리지 마십시오
데이터셋의 일부 샘플만 사용하십시오
배치 크기, 활성화 함수, 옵티마이저, 학습률을 바꿔 보십시오

Keras 모델에서 랜덤 서치

# Define a series of parameters
params = dict(optimizer=['sgd', 'adam'], epochs=3, 
              batch_size=[5, 10, 20], activation=['relu','tanh'])

# Create a random search cv object and fit it to the data
random_search = RandomizedSearchCV(model, params_dist=params, cv=3)

random_search_results = random_search.fit(X, y)

# Print results
print("Best: %f using %s".format(random_search_results.best_score_,
random_search_results.best_params_))

최고: 0.94, 사용된 설정: {'optimizer': 'adam', 'epochs': 3, 'batch_size': 10, 'activation': 'relu'}

다른 하이퍼파라미터 튜닝

def create_model(nl=1,nn=256):
    model = Sequential()
    model.add(Dense(16, input_shape=(2,), activation='relu'))

    # Add as many hidden layers as specified in nl
    for i in range(nl):
        # Layers have nn neurons
        model.add(Dense(nn, activation='relu'))
    # End defining and compiling your model...

다른 하이퍼파라미터 튜닝

# Define parameters, named just like in create_model()
params = dict(nl=[1, 2, 9], nn=[128,256,1000])

# Repeat the random search...

# Print results...

최고: 0.87, 사용된 설정: {'nl': 2,'nn': 128}

네트워크를 튜닝해 봅시다!

Keras로 시작하는 딥 러닝