In [1]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID";
os.environ["CUDA_VISIBLE_DEVICES"]="0";   

In this example, we will apply ktrain to the dataset employed in the scikit-learn Working with Text Data tutorial. As in the tutorial, we will sample 4 newgroups to create a small multiclass text classification dataset. Let's fetch the 20newsgroups dataset using scikit-learn.

In [3]:
categories = ['alt.atheism', 'soc.religion.christian',
             'comp.graphics', 'sci.med']
from sklearn.datasets import fetch_20newsgroups
train_b = fetch_20newsgroups(subset='train',
   categories=categories, shuffle=True, random_state=42)
test_b = fetch_20newsgroups(subset='test',
   categories=categories, shuffle=True, random_state=42)

# # inspect
print(train_b.keys())
print(train_b['target_names'])
print(train_b['filenames'][:5])
print(train_b['target'][:5])
print(train_b['data'][0][:300])
print(train_b['target'][0])
#print(set(train_b['target']))

x_train = train_b.data
y_train = train_b.target
x_test = test_b.data
y_test = test_b.target
dict_keys(['data', 'filenames', 'target_names', 'target', 'DESCR'])
['alt.atheism', 'comp.graphics', 'sci.med', 'soc.religion.christian']
['/home/amaiya/scikit_learn_data/20news_home/20news-bydate-train/comp.graphics/38440'
 '/home/amaiya/scikit_learn_data/20news_home/20news-bydate-train/comp.graphics/38479'
 '/home/amaiya/scikit_learn_data/20news_home/20news-bydate-train/soc.religion.christian/20737'
 '/home/amaiya/scikit_learn_data/20news_home/20news-bydate-train/soc.religion.christian/20942'
 '/home/amaiya/scikit_learn_data/20news_home/20news-bydate-train/soc.religion.christian/20487']
[1 1 3 3 3]
From: [email protected] (Michael Collier)
Subject: Converting images to HP LaserJet III?
Nntp-Posting-Host: hampton
Organization: The City University
Lines: 14

Does anyone know of a good way (standard PC application/PD utility) to
convert tif/img/tga files into LaserJet III format.  We would also li
1
In [4]:
import ktrain
from ktrain import text
Using TensorFlow backend.
In [5]:
(x_train,  y_train), (x_test, y_test), preproc = text.texts_from_array(x_train=x_train, y_train=y_train,
                                                                       x_test=x_test, y_test=y_test,
                                                                       class_names=train_b.target_names,
                                                                      ngram_range=1, 
                                                                       maxlen=350, 
                                                                       max_features=35000)
Word Counts: 36393
Nrows: 2257
2257 train sequences
Average train sequence length: 321
x_train shape: (2257,350)
y_train shape: (2257,4)
1502 test sequences
Average test sequence length: 342
x_test shape: (1502,350)
y_test shape: (1502,4)
In [6]:
model = text.text_classifier('nbsvm', train_data=(x_train, y_train), preproc=preproc)
Is Multi-Label? False
compiling word ID features...
max_features is 27645
maxlen is 350
building document-term matrix... this may take a few moments...
rows: 1-2257
computing log-count ratios...
done.
In [7]:
learner = ktrain.get_learner(model, train_data=(x_train, y_train), val_data=(x_test, y_test))
In [8]:
learner.lr_find()
learner.lr_plot()
simulating training for different learning rates... this may take a few moments...
Epoch 1/1024
2257/2257 [==============================] - 1s 293us/step - loss: 1.4131 - acc: 0.3549
Epoch 2/1024
2257/2257 [==============================] - 0s 147us/step - loss: 1.4120 - acc: 0.3567
Epoch 3/1024
2257/2257 [==============================] - 0s 150us/step - loss: 1.4098 - acc: 0.3589
Epoch 4/1024
2257/2257 [==============================] - 0s 152us/step - loss: 1.4052 - acc: 0.3624
Epoch 5/1024
2257/2257 [==============================] - 0s 150us/step - loss: 1.3961 - acc: 0.3753
Epoch 6/1024
2257/2257 [==============================] - 0s 152us/step - loss: 1.3781 - acc: 0.4072
Epoch 7/1024
2257/2257 [==============================] - 0s 146us/step - loss: 1.3429 - acc: 0.4657
Epoch 8/1024
2257/2257 [==============================] - 0s 148us/step - loss: 1.2772 - acc: 0.5782
Epoch 9/1024
2257/2257 [==============================] - 0s 150us/step - loss: 1.1628 - acc: 0.7297
Epoch 10/1024
2257/2257 [==============================] - 0s 146us/step - loss: 0.9845 - acc: 0.8666
Epoch 11/1024
2257/2257 [==============================] - 0s 147us/step - loss: 0.7499 - acc: 0.9526
Epoch 12/1024
2257/2257 [==============================] - 0s 145us/step - loss: 0.5081 - acc: 0.9809
Epoch 13/1024
2257/2257 [==============================] - 0s 147us/step - loss: 0.3162 - acc: 0.9880
Epoch 14/1024
2257/2257 [==============================] - 0s 153us/step - loss: 0.1874 - acc: 0.9951
Epoch 15/1024
2257/2257 [==============================] - 0s 156us/step - loss: 0.1073 - acc: 0.9973
Epoch 16/1024
2257/2257 [==============================] - 0s 154us/step - loss: 0.0584 - acc: 0.9982
Epoch 17/1024
2257/2257 [==============================] - 0s 148us/step - loss: 0.0292 - acc: 0.9996
Epoch 18/1024
2257/2257 [==============================] - 0s 156us/step - loss: 0.0140 - acc: 1.0000
Epoch 19/1024
2257/2257 [==============================] - 0s 152us/step - loss: 0.0065 - acc: 1.0000
Epoch 20/1024
2257/2257 [==============================] - 0s 156us/step - loss: 0.0030 - acc: 1.0000
Epoch 21/1024
2257/2257 [==============================] - 0s 156us/step - loss: 0.0014 - acc: 1.0000
Epoch 22/1024
2257/2257 [==============================] - 0s 145us/step - loss: 6.4536e-04 - acc: 1.0000
Epoch 23/1024
2257/2257 [==============================] - 0s 149us/step - loss: 3.0513e-04 - acc: 1.0000
Epoch 24/1024
2257/2257 [==============================] - 0s 148us/step - loss: 1.4576e-04 - acc: 1.0000
Epoch 25/1024
2257/2257 [==============================] - 0s 153us/step - loss: 6.8675e-05 - acc: 1.0000
Epoch 26/1024
2257/2257 [==============================] - 0s 159us/step - loss: 3.2571e-05 - acc: 1.0000
Epoch 27/1024
2257/2257 [==============================] - 0s 158us/step - loss: 1.5664e-05 - acc: 1.0000
Epoch 28/1024
2257/2257 [==============================] - 0s 154us/step - loss: 7.4376e-06 - acc: 1.0000
Epoch 29/1024
2257/2257 [==============================] - 0s 150us/step - loss: 3.5614e-06 - acc: 1.0000
Epoch 30/1024
2257/2257 [==============================] - 0s 151us/step - loss: 1.7064e-06 - acc: 1.0000
Epoch 31/1024
2257/2257 [==============================] - 0s 156us/step - loss: 8.4131e-07 - acc: 1.0000
Epoch 32/1024
2257/2257 [==============================] - 0s 156us/step - loss: 4.3300e-07 - acc: 1.0000
Epoch 33/1024
2257/2257 [==============================] - 0s 153us/step - loss: 2.4621e-07 - acc: 1.0000
Epoch 34/1024
2257/2257 [==============================] - 0s 153us/step - loss: 1.6358e-07 - acc: 1.0000
Epoch 35/1024
2257/2257 [==============================] - 0s 148us/step - loss: 1.3281e-07 - acc: 1.0000
Epoch 36/1024
2257/2257 [==============================] - 0s 151us/step - loss: 1.2280e-07 - acc: 1.0000
Epoch 37/1024
2257/2257 [==============================] - 0s 155us/step - loss: 1.1990e-07 - acc: 1.0000
Epoch 38/1024
2257/2257 [==============================] - 0s 160us/step - loss: 1.1939e-07 - acc: 1.0000
Epoch 39/1024
2257/2257 [==============================] - 0s 158us/step - loss: 1.1921e-07 - acc: 1.0000
Epoch 40/1024
1760/2257 [======================>.......] - ETA: 0s - loss: 1.2209e-07 - acc: 1.0000

done.
Please invoke the Learner.lr_plot() method to visually inspect the loss plot to help identify the maximal learning rate associated with falling loss.
In [9]:
learner.autofit(0.01)
early_stopping automatically enabled at patience=5
reduce_on_plateau automatically enabled at patience=2


begin training using triangular learning rate policy with max lr of 0.01...
Train on 2257 samples, validate on 1502 samples
Epoch 1/1024
2257/2257 [==============================] - 1s 234us/step - loss: 0.2529 - acc: 0.9313 - val_loss: 0.2315 - val_acc: 0.9301
Epoch 2/1024
2257/2257 [==============================] - 0s 214us/step - loss: 0.0155 - acc: 0.9978 - val_loss: 0.2289 - val_acc: 0.9294
Epoch 3/1024
2257/2257 [==============================] - 0s 221us/step - loss: 0.0085 - acc: 0.9996 - val_loss: 0.2292 - val_acc: 0.9294
Epoch 4/1024
2257/2257 [==============================] - 0s 221us/step - loss: 0.0064 - acc: 0.9996 - val_loss: 0.2285 - val_acc: 0.9288
Epoch 5/1024
2257/2257 [==============================] - 1s 222us/step - loss: 0.0050 - acc: 1.0000 - val_loss: 0.2288 - val_acc: 0.9288
Epoch 6/1024
2257/2257 [==============================] - 0s 217us/step - loss: 0.0041 - acc: 1.0000 - val_loss: 0.2274 - val_acc: 0.9294
Epoch 7/1024
2257/2257 [==============================] - 0s 214us/step - loss: 0.0035 - acc: 1.0000 - val_loss: 0.2277 - val_acc: 0.9294
Epoch 8/1024
2257/2257 [==============================] - 0s 209us/step - loss: 0.0031 - acc: 1.0000 - val_loss: 0.2276 - val_acc: 0.9308

Epoch 00008: Reducing Max LR on Plateau: new max lr will be 0.005 (if not early_stopping).
Epoch 9/1024
2257/2257 [==============================] - 0s 208us/step - loss: 0.0028 - acc: 1.0000 - val_loss: 0.2275 - val_acc: 0.9301
Epoch 10/1024
2257/2257 [==============================] - 0s 213us/step - loss: 0.0026 - acc: 1.0000 - val_loss: 0.2274 - val_acc: 0.9301

Epoch 00010: Reducing Max LR on Plateau: new max lr will be 0.0025 (if not early_stopping).
Epoch 11/1024
2257/2257 [==============================] - 0s 215us/step - loss: 0.0025 - acc: 1.0000 - val_loss: 0.2274 - val_acc: 0.9301
Epoch 12/1024
2257/2257 [==============================] - 0s 218us/step - loss: 0.0024 - acc: 1.0000 - val_loss: 0.2274 - val_acc: 0.9314
Epoch 13/1024
2257/2257 [==============================] - 0s 214us/step - loss: 0.0024 - acc: 1.0000 - val_loss: 0.2275 - val_acc: 0.9314

Epoch 00013: Reducing Max LR on Plateau: new max lr will be 0.00125 (if not early_stopping).
Epoch 14/1024
2257/2257 [==============================] - 0s 215us/step - loss: 0.0023 - acc: 1.0000 - val_loss: 0.2275 - val_acc: 0.9314
Epoch 15/1024
2257/2257 [==============================] - 0s 218us/step - loss: 0.0023 - acc: 1.0000 - val_loss: 0.2276 - val_acc: 0.9314

Epoch 00015: Reducing Max LR on Plateau: new max lr will be 0.000625 (if not early_stopping).
Epoch 16/1024
2257/2257 [==============================] - 0s 201us/step - loss: 0.0022 - acc: 1.0000 - val_loss: 0.2276 - val_acc: 0.9314
Restoring model weights from the end of the best epoch
Epoch 00016: early stopping
Weights from best epoch have been loaded into model.
Out[9]:
<keras.callbacks.History at 0x7f166cdd7908>
In [10]:
learner.validate()
              precision    recall  f1-score   support

           0       0.93      0.86      0.89       319
           1       0.95      0.95      0.95       389
           2       0.95      0.92      0.94       396
           3       0.90      0.97      0.93       398

    accuracy                           0.93      1502
   macro avg       0.93      0.93      0.93      1502
weighted avg       0.93      0.93      0.93      1502

Out[10]:
array([[274,   4,   8,  33],
       [  4, 371,  10,   4],
       [ 10,  13, 366,   7],
       [  6,   4,   2, 386]])
In [12]:
predictor = ktrain.get_predictor(learner.model, preproc)
In [13]:
predictor.get_classes()
Out[13]:
['alt.atheism', 'comp.graphics', 'sci.med', 'soc.religion.christian']
In [14]:
predictor.predict(test_b.data[0:3])
Out[14]:
['sci.med', 'sci.med', 'sci.med']
In [15]:
test_b.target[:3]
Out[15]:
array([2, 2, 2])