二、什么是文本分類？

• 文本分類是指將一段文本（例如，客戶評論、電子郵件、網(wǎng)頁、新聞文章）分類為一些預(yù)定義的類別或類。正面、負(fù)面或中立的評論評論、垃圾郵件或非垃圾郵件、作為個(gè)人或商業(yè)頁面的網(wǎng)頁、有關(guān)政治、體育或金融的新聞文章）
• 即，文本分類是為給定文本分配標(biāo)簽或類的任務(wù)。例如，將電子郵件歸類為垃圾郵件。出于分類的目的，通常從輸入文本中識別出一些信息量很大的特征。
• 監(jiān)督機(jī)器學(xué)習(xí)和無監(jiān)督學(xué)習(xí)可用于 NLP 中的文本分類。監(jiān)督學(xué)習(xí)涉及在標(biāo)記的數(shù)據(jù)集上訓(xùn)練分類器。無監(jiān)督學(xué)習(xí)不需要標(biāo)記的數(shù)據(jù)集，它使用數(shù)據(jù)的固有屬性（相似性）將數(shù)據(jù)聚類到組中。高質(zhì)量的標(biāo)記數(shù)據(jù)，通常來自人類注釋者，對于監(jiān)督機(jī)器學(xué)習(xí)非常重要。標(biāo)記的數(shù)據(jù)通常分為 3 個(gè)部分，訓(xùn)練集、驗(yàn)證集和測試集。分類器的性能使用準(zhǔn)確率、精確度、召回率和 F1 分?jǐn)?shù)等指標(biāo)進(jìn)行評估。

三、文本分類的重要用例：

1. 情緒分析
2. POS 標(biāo)簽
3. 自然語言推理 — 推斷兩段文本之間的關(guān)系——前提和假設(shè)。這種關(guān)系的類型——蘊(yùn)涵性、矛盾性和中性性。

• 蘊(yùn)涵：假設(shè)由前提支持
• 矛盾：假設(shè)被前提否定
• 中性：假設(shè)和前提之間沒有關(guān)系 例如，
• 前提：我現(xiàn)在正在看電影。
• 假設(shè)：我現(xiàn)在正在打板球。關(guān)系標(biāo)簽：矛盾

4. 檢查語法正確性：可接受/不可接受。

四、使用 NLTK 進(jìn)行文本分類

• 對于文本分類的第一個(gè)示例，我們將使用?nltk?庫中內(nèi)置的movie_reviews語料庫。
• 您可以使用 nltk.download 函數(shù)下載 movie_reviews 包：?import nltk nltk.download("movie_reviews")

????????fileids（） 方法允許我們訪問 nltk.corpus 中數(shù)據(jù)集中所有文件的列表。movie_reviews數(shù)據(jù)集有 2000 個(gè)文本文件，每個(gè)文件都有一個(gè) fileid。這些文件中的每一個(gè)都包含對電影的評論。其中 1000 個(gè)文件包含負(fù)面評論，1000 個(gè)包含正面評論。負(fù)面文件位于名為“neg”的文件夾中，所有包含正面評論的文件都位于名為“pos”的文件夾中。

#Required imports
import nltk
import random
from nltk.corpus import movie_reviews#Total no. of review files in corpus
##There are 1000 negative reviews, and 1000 positive reviews (one review per file)
len(movie_reviews.fileids())#separating filenames in two lists one for positive reviews, one for negative reviews(based on which folder they exists in corpus)
negative_fileids = movie_reviews.fileids('neg')
positive_fileids = movie_reviews.fileids('pos')# Now we will load all reviews and their labels (i.e., folder name pos or neg in which the review file is present)reviewswithcategory = [(list(movie_reviews.words(fileid)), category) for category in movie_reviews.categories()for fileid in movie_reviews.fileids(category)]
random.shuffle(reviewswithcategory)

接下來，讓我們做一些文本預(yù)處理：

# Text pre-processing - lower casing, removing stop words and punctuation marks
import string
from nltk.corpus import stopwords
stop_words = stopwords.words('english')
def text_preprocessing(review):review = [w.lower() for w in review]review = [w.translate(str.maketrans('', '', string.punctuation)) for w in review]review = [w for w in review if w not in stop_words]review = list(filter(None, review)) #Remove empty stringsreturn reviewcleaned_reviewswithcategory = []
for review, cat in reviewswithcategory:cleanreview = text_preprocessing(review)cleaned_reviewswithcategory.append((cleanreview, cat))# Our variable cleaned_reviewswithcategory is a list of tuples, 
# each tuple in it has a list of words and category label# here we all getting all words from all tuples into a single iterable
allcleanwords = list(itertools.chain(*cleaned_reviewswithcategory))
allcleanwordslist = []
for m in range(len(allcleanwords)):# traversing the inner listsfor n in range (len(allcleanwords[m])):# Add each element to the result listallcleanwordslist.append(allcleanwords[m][n])

接下來，我們從電影評論數(shù)據(jù)中清理過的單詞列表中確定 5000 個(gè)最常見的單詞

# Using NLTK FreqDist for computing word frequencies
freqd = nltk.FreqDist(allcleanwordslist)
# Identifying 5000 most frequent words from Frequency Distribution
frequent_words = list(freqd.keys())[:5000]

現(xiàn)在，我們將只使用這 5000 個(gè)單詞。對于正面和負(fù)面類別中的每條評論，特征向量將包含這些常用詞和一個(gè)布爾值 True（如果該詞存在于該評論中），否則為 False。

# Identify the presence of these most frequent words in out positive and negative reviews.
# This function returns word and True if word is present, else it returns word and False. def extract_frequentwordfeatures(text):words = set(text) # computing all unique words (vocabulary) in input textfeatures = {}for w in frequent_words:features[w] = (w in words)return featuresreview_features = [(extract_frequentwordfeatures(review), category) for (review, category) in cleaned_reviewswithcategory]

????????現(xiàn)在，每個(gè)評論都由其特征表示，該特征是一個(gè)單詞列表和一個(gè)布爾值 True 或 False，指示該單詞是否存在于評論中。列表中共有 2000 條評論。接下來，我們將評審功能拆分為訓(xùn)練和測試部分。在 2000 條評論中，我們將使用 1800 條來訓(xùn)練來自 NLTK 的樸素貝葉斯分類器，并使用 200 條來測試其性能。

# Splitting the documents into training and test portions
train_data = review_features[:1800]
# set that we'll test against.
test_data = review_features[1800:]# use Naive Bayes classifier from NLTK to train 
clf_nb = nltk.NaiveBayesClassifier.train(train_data)# After training, let us see the accuracy
print(" Accuracy:",(nltk.classify.accuracy(clf_nb, test_data)))

五、使用 sklearn 分類器對上述評論數(shù)據(jù)進(jìn)行分類

from nltk.classify.scikitlearn import SklearnClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import MultinomialNB
from sklearn.svm import SVC
from sklearn.metrics import classification_report, accuracy_score, confusion_matrixnames = ['Logistic Regression','Multinomial Naive Bayes', 'Support Vector Machine']classifiers = [LogisticRegression(),MultinomialNB(),SVC(kernel='linear')]
models = zip(names, classifiers)text_features, labels = zip(*test_data)for name, model in models:nltk_clf = SklearnClassifier(model)nltk_clf.train(train_data)accuracy = nltk.classify.accuracy(nltk_clf, test_data)print("\n{} Classifier Accuracy: {}".format(name, accuracy))  

六、使用 Keras 進(jìn)行文本分類

????????在這部分，我們將使用來自 UCI 存儲庫的 Sentiment Labelled Sentences 數(shù)據(jù)集。此數(shù)據(jù)集包含標(biāo)有正面或負(fù)面情緒的句子。情緒是分?jǐn)?shù)的形式，分?jǐn)?shù)是 1 表示積極情緒，0 表示消極情緒。這些句子來自三個(gè)不同的網(wǎng)站：imdb.com、amazon.com yelp.com 對于每個(gè)網(wǎng)站，存在 500 個(gè)正面句子和 500 個(gè)負(fù)面句子。在這里的示例中，我們將僅使用文件amazon_cells_labelled.txt中的亞馬遜評論。

????????首先，我們將使用 pandas 將數(shù)據(jù)讀入 pandas 數(shù)據(jù)幀。此數(shù)據(jù)有兩列 — 句子和標(biāo)簽。句子是產(chǎn)品評論，標(biāo)簽是 0 或 1。標(biāo)簽 1 表示積極情緒，0 表示消極情緒。

import pandas as pd
df = pd.read_csv('amazon_cells_labelled.txt', names=['sentence', 'label'], sep='\t')
df.head()

接下來，我們對句子列中的文本進(jìn)行預(yù)處理

# This process_text() function returns list of cleaned tokens of the text
import numpy
import re
import string
import unicodedata
from nltk.corpus import stopwords
from nltk.stem import WordNetLemmatizer
stop_words = stopwords.words('english')
lemmatizer = WordNetLemmatizer()def process_text(text):text = unicodedata.normalize('NFKD', text).encode('ascii', 'ignore').decode('utf-8', 'ignore')text = re.sub(r'[^a-zA-Z\s]', '', text)text = text.translate(str.maketrans('', '', string.punctuation))text = text.lower()text = " ".join([word for word in str(text).split() if word not in stop_words])text = " ".join([lemmatizer.lemmatize(word) for word in text.split()])return text
df['sentence'] = df['sentence'].apply(process_text)
df['sentence']

????????現(xiàn)在我們將數(shù)據(jù)分為訓(xùn)練和測試部分，在此之前，讓我們將“句子”列中的文本和“標(biāo)簽”列中的標(biāo)簽分為兩個(gè)pandas系列——“句子”和“標(biāo)簽”。

# Taking cleaned text and sentiment labels in two separate variables
sentences = df['sentence']
labels = df['label']# Splitting into train-test portions
from sklearn.model_selection import train_test_split
sentences_train, sentences_test, labels_train, labels_test = train_test_split(sentences, labels, test_size=0.25, random_state=1000)

接下來，我們使用 sklearn 中帶有 CountVectorizer 的詞袋模型以向量形式表示句子中的文本

# Converting sentences to vectors using Bag of words model with CountVectorizer
from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer()
cv.fit(sentences_train)vc_traindata = cv.transform(sentences_train)
vc_testdata  = cv.transform(sentences_test)
vc_traindata

????????現(xiàn)在我們將使用 Keras 進(jìn)行分類，因此應(yīng)該在我們的計(jì)算機(jī)上安裝 TensorFlow 和 Keras 庫。可以使用 pip 命令從 Jupyter Notebook 中安裝它們

!pip install tensorflow
!pip install keras

首先，我們將使用 Keras Tokenizer 來計(jì)算文本的單詞嵌入

# Using Keras Tokenizer to compute embeddings for text
from keras.preprocessing.text import Tokenizer
tokenizer = Tokenizer(num_words=5000)
tokenizer.fit_on_texts(df['sentence'])# Take the sentence text in a variable X and labels in y.
X = df['sentence']
y = df['label']#Splitting X and y into train and test portions
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=1000)#converting sentences into vector sequences
X_train = tokenizer.texts_to_sequences(X_train)
X_test = tokenizer.texts_to_sequences(X_test)# Total number of unique words in our sentences
vocab_size = len(tokenizer.word_index) + 1  # Adding 1 because of reserved 0 index
print('Vocabulary size:' , vocab_size)

????????然后，我們將填充所有向量（代表產(chǎn)品評論的文本）的長度相同 — 50

# padding vector sequences to make them all of same length
from keras_preprocessing.sequence import pad_sequences
maxlen = 50
X_train = pad_sequences(X_train, padding='post', maxlen=maxlen)
X_test = pad_sequences(X_test, padding='post', maxlen=maxlen)
print(X_train[4, :])

七、使用 Keras 的嵌入層

????????我們現(xiàn)在將使用 Keras 的嵌入層，它采用先前計(jì)算的整數(shù)并將它們映射到嵌入的密集向量。它需要以下參數(shù)：

• input_dim：詞匯量的大小
• output_dim: the size of the dense vector
• input_length: the length of the sequence

????????我們可以獲取嵌入層的輸出并將其插入密集層。為此，我們需要在中間添加一個(gè) Flatten 層，為 Dense 層準(zhǔn)備順序輸入。

????????在訓(xùn)練期間，要訓(xùn)練的參數(shù)數(shù)vacab_size乘以embedding_dim。嵌入層的權(quán)重是隨機(jī)初始化的，然后在訓(xùn)練期間使用反向傳播進(jìn)行微調(diào)。該模型將按句子順序出現(xiàn)的單詞作為輸入向量

????????在下面的代碼中，我們使用嵌入層 GlobalMaxPool1D 進(jìn)行扁平化，以及由 15 個(gè)神經(jīng)元和一個(gè)輸出層組成的密集層。我們編譯了 keras 模型。

from keras.models import Sequential
from keras import layersembedding_dim = 100
maxlen = 50
model = Sequential()
model.add(layers.Embedding(input_dim=vocab_size, output_dim=embedding_dim, input_length=maxlen))
model.add(layers.GlobalMaxPool1D())
model.add(layers.Dense(15, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
model.summary()

????????接下來，我們將模型擬合在 50 個(gè) epoch 的訓(xùn)練數(shù)據(jù)上，并評估其性能。使用 matplotlib 繪制模型的準(zhǔn)確性

import matplotlib.pyplot as plt
plt.figure(figsize=(9, 5))history = model.fit(X_train, y_train,epochs=50, verbose=False,validation_data=(X_test, y_test), batch_size=10)
loss, accuracy = model.evaluate(X_train, y_train, verbose=False)
print("Training Accuracy: {:.4f}".format(accuracy))
loss, accuracy = model.evaluate(X_test, y_test, verbose=False)
print("Testing Accuracy:  {:.4f}".format(accuracy))acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']
x = range(1, len(acc) + 1)
plt.plot(x, acc, 'b', label='Training acc')
plt.plot(x, val_acc, 'r', label='Validation acc')
plt.legend()
plt.title('Training and validation accuracy')
plt.show()

????????在下一篇文章中，我們將看看變形金剛。代碼出處尼姆里塔·庫爾

參考和引用：

1. https://developers.google.com/machine-learning/guides/text-classification
2. Text Classification with Python and Scikit-Learn
3. https://towardsdatascience.com/a-practitioners-guide-to-natural-language-processing-part-i-processing-understanding-text-9f4abfd13e72
4. Text Classification using NLTK | Foundations of AI & ML
5. Practical Text Classification With Python and Keras – Real Python
6. Text Classification with NLTK | Chan`s Jupyter
7. https://www.analyticsvidhya.com/blog/2018/04/a-comprehensive-guide-to-understand-and-implement-text-classification-in-python/
8. https://medium.com/analytics-vidhya/nlp-tutorial-for-text-classification-in-python-8f19cd17b49e
9. Practical Text Classification With Python and Keras – Real Python