기본 특성 추출

Python으로 배우는 NLP 피처 엔지니어링

Rounak Banik

Data Scientist

문자 수

"I don't know." # 13 characters

# Compute the number of characters 
text = "I don't know."
num_char = len(text)

# Print the number of characters
print(num_char)

# Create a 'num_chars' feature
df['num_chars'] = df['review'].apply(len)

단어 수

# Split the string into words
text = "Mary had a little lamb."
words = text.split()

# Print the list containing words
print(words)

['Mary', 'had', 'a', 'little', 'lamb.']

# Print number of words
print(len(words))

단어 수

# Function that returns number of words in string
def word_count(string):
    # Split the string into words
    words = string.split()

    # Return length of words list
    return len(words)

# Create num_words feature in df
df['num_words'] = df['review'].apply(word_count)

평균 단어 길이

#Function that returns average word length
def avg_word_length(x):

    # Split the string into words
    words = x.split()

    # Compute length of each word and store in a separate list
    word_lengths = [len(word) for word in words]

    # Compute average word length
    avg_word_length = sum(word_lengths)/len(words)

    # Return average word length
    return(avg_word_length)

평균 단어 길이

# Create a new feature avg_word_length
df['avg_word_length'] = df['review'].apply(doc_density)

특수 특성

Datacamp 트윗

해시태그와 멘션

# Function that returns number of hashtags
def hashtag_count(string):

    # Split the string into words
    words = string.split()

    # Create a list of hashtags
    hashtags = [word for word in words if word.startswith('#')]

    # Return number of hashtags
    return len(hashtags)

hashtag_count("@janedoe This is my first tweet! #FirstTweet #Happy")

기타 특성

문장 수
문단 수
대문자로 시작하는 단어
전부 대문자 단어
숫자 표현

연습해 봅시다!

Python으로 배우는 NLP 피처 엔지니어링