PyCaret 2 NLP Example¶

This notebook is created using PyCaret 2.0. Last updated : 28-07-2020

In [1]:

# check version
from pycaret.utils import version
version()

pycaret-nightly-0.39

1. Loading Dataset¶

In [2]:

from pycaret.datasets import get_data
data = get_data('kiva')

	country	en	gender	loan_amount	nonpayment	sector
0	Dominican Republic	"Banco Esperanza" is a group of 10 women looki...	F	1225	partner	Retail
1	Dominican Republic	"Caminemos Hacia Adelante" or "Walking Forward...	F	1975	lender	Clothing
2	Dominican Republic	"Creciendo Por La Union" is a group of 10 peop...	F	2175	partner	Clothing
3	Dominican Republic	"Cristo Vive" ("Christ lives" is a group of 10...	F	1425	partner	Clothing
4	Dominican Republic	"Cristo Vive" is a large group of 35 people, 2...	F	4025	partner	Food

In [3]:

from pycaret.nlp import *
nlp1 = setup(data, target = 'en', session_id=123, log_experiment=True, log_plots = True, experiment_name='kiva1')

In [4]:

models()

Out[4]:

	Name	Reference
ID
lda	Latent Dirichlet Allocation	gensim/models/ldamodel
lsi	Latent Semantic Indexing	gensim/models/lsimodel
hdp	Hierarchical Dirichlet Process	gensim/models/hdpmodel
rp	Random Projections	gensim/models/rpmodel
nmf	Non-Negative Matrix Factorization	sklearn.decomposition.NMF

In [5]:

lda = create_model('lda')

In [6]:

nmf = create_model('nmf', num_topics = 6)

In [7]:

lda_results = assign_model(lda)
lda_results.head()

Out[7]:

	country	en	gender	loan_amount	nonpayment	sector	Topic_0	Topic_1	Topic_2	Topic_3	Dominant_Topic	Perc_Dominant_Topic
0	Dominican Republic	group woman look receive small loan take small...	F	1225	partner	Retail	0.443424	0.170630	0.001865	0.384082	Topic 0	0.44
1	Dominican Republic	walk forward group entrepreneur seek second lo...	F	1975	lender	Clothing	0.335674	0.416064	0.001612	0.246650	Topic 1	0.42
2	Dominican Republic	group people hope start business group look re...	F	2175	partner	Clothing	0.568597	0.153035	0.002369	0.275999	Topic 0	0.57
3	Dominican Republic	live group woman look receive first loan young...	F	1425	partner	Clothing	0.285612	0.225983	0.001858	0.486547	Topic 3	0.49
4	Dominican Republic	vive large group people hope take loan many se...	F	4025	partner	Food	0.383666	0.345023	0.001839	0.269472	Topic 0	0.38

In [8]:

plot_model(lda)

In [14]:

plot_model(lda, plot = 'bigram')

In [15]:

plot_model(lda, plot = 'tsne')

In [11]:

evaluate_model(lda)

interactive(children=(ToggleButtons(description='Plot Type:', icons=('',), options=(('Frequency Plot', 'freque…

In [12]:

get_logs()

Out[12]:

	run_id	experiment_id	status	artifact_uri	start_time	end_time	metrics.TT	params.solver	params.init	params.l1_ratio	...	tags.mlflow.runName	tags.USI	tags.mlflow.user	tags.URI	tags.mlflow.source.type	tags.mlflow.source.name	tags.Run ID	tags.Size KB	tags.Source	tags.Run Time
0	66f0e52994d245beb6d8a18c7cf5022e	4	FINISHED	file:///C:/Users/moezs/pycaret-demo-td/mlruns/...	2020-07-29 13:57:04.059000+00:00	2020-07-29 13:57:04.336000+00:00	2.18	cd	nndsvd	0.0	...	Non-Negative Matrix Factorization	2c3d	moezs	c7c29a21	LOCAL	C:\Users\moezs\Anaconda3\envs\pycaret-nightly-...	66f0e52994d245beb6d8a18c7cf5022e	240.69	create_model	2.22
1	9deb5a3e41424e5da94f16e28c67fb8a	4	FINISHED	file:///C:/Users/moezs/pycaret-demo-td/mlruns/...	2020-07-29 13:57:01.450000+00:00	2020-07-29 13:57:01.801000+00:00	55.55	None	None	None	...	Latent Dirichlet Allocation	2c3d	moezs	18a25e3b	LOCAL	C:\Users\moezs\Anaconda3\envs\pycaret-nightly-...	9deb5a3e41424e5da94f16e28c67fb8a	764.39	create_model	55.62
2	086a171759314b55b8a8fc38b3d50512	4	FINISHED	file:///C:/Users/moezs/pycaret-demo-td/mlruns/...	2020-07-29 13:55:31.945000+00:00	2020-07-29 13:56:05.706000+00:00	NaN	None	None	None	...	Session Initialized 2c3d	2c3d	moezs	d9929f46	LOCAL	C:\Users\moezs\Anaconda3\envs\pycaret-nightly-...	086a171759314b55b8a8fc38b3d50512	None	setup	266.11

3 rows × 56 columns

In [13]:

!mlflow ui

Running the mlflow server failed. Please see the logs above for details.

Thank you. For more information / tutorials on PyCaret, please visit https://www.pycaret.org