{"id":2891,"date":"2022-09-02T10:37:30","date_gmt":"2022-09-02T03:37:30","guid":{"rendered":"http:\/\/international.binus.ac.id\/computer-science\/?p=2891"},"modified":"2022-09-02T10:37:30","modified_gmt":"2022-09-02T03:37:30","slug":"pyldavis-topic-modelling-exploration-tool-that-every-nlp-data-scientist-should-know","status":"publish","type":"post","link":"https:\/\/international.binus.ac.id\/computer-science\/2022\/09\/02\/pyldavis-topic-modelling-exploration-tool-that-every-nlp-data-scientist-should-know\/","title":{"rendered":"pyLDAvis: Topic Modelling Exploration Tool That Every NLP Data Scientist Should Know"},"content":{"rendered":"

Have you ever wanted to classify news, papers, or tweets based on their topics? Knowing how to do this can help you filter out irrelevant documents, and save time by reading only what you\u2019re interested in.<\/p>\n

That\u2019s what text classification is for \u2013 allows you to train your model to recognize topics. This technique allows you to use data labels to train your model, and it\u2019s supervised learning.<\/p>\n

\n
\"text<\/figure>\n<\/div>\n

In real life, you might not have data labels for text classification. You can go through each document to label them, or hire somebody else to do it, but that\u2019s a lot of time and money, especially when you have more than 1000 data points.<\/p>\n

Can you find the topics of your documents without training data? Yes, you can use topic modeling to do it.<\/p>\n

What is topic modeling?<\/h2>\n

With\u00a0topic modeling<\/a>, you can cluster words for a set of documents. This is unsupervised learning, because it automatically groups words without a predefined list of labels.<\/p>\n

If you feed the model data, it will give you different sets of words, and each set of words describes the topic.<\/p>\n

(0<\/span>, ‘0.024*<\/span>“ban”<\/span> + 0.017<\/span>*“order”<\/span> + 0.015<\/span>*“refugee”<\/span> + 0.015<\/span>*“law”<\/span> + 0.013<\/span>*“trump”<\/span> ‘ ‘+<\/span> 0.011<\/span>*“kill”<\/span> + 0.011<\/span>*“country”<\/span> + 0.010<\/span>*“attack”<\/span> + 0.009<\/span>*“state”<\/span> + ‘ ‘0.009*<\/span>“immigration”<\/span>‘) (1<\/span>, ‘0.020*<\/span>“student”<\/span> + 0.020<\/span>*“work”<\/span> + 0.019<\/span>*“great”<\/span> + 0.017<\/span>*“learn”<\/span> + ‘ ‘0.017*<\/span>“school”<\/span> + 0.015<\/span>*“talk”<\/span> + 0.014<\/span>*“support”<\/span> + 0.012<\/span>*“community”<\/span> + ‘ ‘0.010*<\/span>“share”<\/span> + 0.009<\/span>*“event”<\/span>)<\/div>\n

When you look at the first set of words, you would guess the topic is military and politics.\u00a0Looking at the second set of words, you might guess the topic is public events, or school.<\/p>\n

This is quite useful. Your texts are automatically categorized, without the need to label them!<\/p>\n

Visualize topic modeling with pyLDAvis<\/h2>\n

Topic modeling is useful, but it\u2019s difficult to understand it just by looking at a combination of words and numbers like above.<\/p>\n

One of the most effective ways to understand data is through visualization. Is there a way that we can visualize the results of LDA? Yes, we can with\u00a0pyLDAvis<\/a>.<\/p>\n

PyLDAvis allows us to interpret the topics in a topic model like below:<\/p>\n

\n
\"PyLDAvis<\/figure>\n<\/div>\n

Pretty cool, isn\u2019t it?\u00a0Now we will learn how to use topic modeling and pyLDAvis to categorize tweets and visualize the results.<\/strong><\/p>\n

 <\/p>\n

Original Article: https:\/\/neptune.ai\/blog\/pyldavis-topic-modelling-exploration-tool-that-every-nlp-data-scientist-should-know<\/p>\n","protected":false},"excerpt":{"rendered":"

Have you ever wanted to classify news, papers, or tweets based on their topics? Knowing how to do this can help you filter out irrelevant documents, and save time by reading only what you\u2019re interested in. That\u2019s what text classification is for \u2013 allows you to train your model to recognize topics. This technique allows […]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[112],"tags":[],"class_list":["post-2891","post","type-post","status-publish","format-standard","hentry","category-article"],"_links":{"self":[{"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/posts\/2891"}],"collection":[{"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/comments?post=2891"}],"version-history":[{"count":1,"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/posts\/2891\/revisions"}],"predecessor-version":[{"id":2892,"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/posts\/2891\/revisions\/2892"}],"wp:attachment":[{"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/media?parent=2891"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/categories?post=2891"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/international.binus.ac.id\/computer-science\/wp-json\/wp\/v2\/tags?post=2891"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}