Table of Contents
- What is AI text mining?
- Two benefits of using AI text mining
- (1) Labor reduction
- ②Analysis of big data is possible
- How to use AI text mining | Explanation with four use cases
- ①Market analysis
- (2) Analyze big data
- (3) Analysis of customer feedback
- ④ Questionnaire analysis
- 3 recommended AI text mining free tools!
- ①AI text mining by user local
- ②KH Coder
- ③ Statistics software R
- Free AI text mining tool comparison
- How to choose an AI text mining tool
- Practice using tools!
- ① Just enter in the text box!
- (2) View analysis results
- summary
What is AI text mining?
Text mining is the extraction of useful information from large amounts of text data such as sentences and conversations. Conventional text mining methods have focused on simple tasks such as “counting the number of occurrences” and “analyzing morphology”.
Attention is focused on introducing AI into this field of text mining. By using AI, it becomes possible to add ambiguity peculiar to natural language, such as differences in phrasing, and to extract more useful information.
Two benefits of using AI text mining
Using AI for text mining has the following two advantages.
- labor reduction
- Big data can be analyzed
I will explain each.
(1) Labor reduction
Analyzing text mining by hand takes a lot of time.
For example, it takes a lot of human cost to check each one of the customer feedback questionnaires that are often found in department stores.
However, text mining using tools can be analyzed automatically in a short time. As a result, significant operational efficiency can be achieved.
②Analysis of big data is possible
By using big data, which is a huge data group, it is possible to analyze on a much larger scale than manual work.
We present the following four cases.
- market analysis
- Analyze big data
- Analyze voice of the customer
- Questionnaire analysis
①Market analysis
A system that predicts economic trends is being researched by combining machine learning methods and text mining. In the past, economic market forecasts were mainly analyzed based on numerical indicators such as stock prices.
However, it is difficult to analyze the market properly with such data alone, so research was conducted to convert the Nihon Keizai Shimbun’s economic information into text.
Improvements in natural language parsing technology have enabled AI to analyze unformatted text data to some extent.
(2) Analyze big data
It can be used not only for customer needs, but also as a method for analyzing big data accumulated in the information systems of general companies.
For example, there is analysis of manually created documents such as design documents and daily reports.
From such documents, AI is expected to contribute to workplace improvement by discovering connections between data and causal relationships that cannot be grasped by human analysis.
(3) Analysis of customer feedback
By combining AI ( artificial intelligence ) with voice, text mining can also be used for inquiries made by phone.
In fact, in recent years, it has been introduced to call centers. AI takes care of transcription and data analysis, which leads to a reduction in work time.
The accuracy of AI analysis has also increased, and it is possible to aggregate the history and check the trend of inquiries.
④ Questionnaire analysis
Email surveys are one of the main marketing techniques.
Traditional approaches typically employ the form of alternatives. As a result, the burden of manual aggregation and analysis was reduced. However, the true voice of the customer cannot be investigated by doing so.
Therefore, by installing AI, it became possible to analyze free-form fields that collect customers’ real opinions. It is also possible to use AI to read customer emotions with a special algorithm.
3 recommended AI text mining free tools!
This chapter introduces recommended AI text mining tools. All of the tools introduced here are free, so feel free to try them out.
- AI Text Mining by User Local
- KH Coder
- Statistics software R
①AI text mining by user local
A text mining tool provided by User Local Co., Ltd. Just paste your text and click to analyze.
The main features provided by this service are:
| ・Word cloud creation ・Co-occurring keywords ・2D map・Dependency analysis ・Hierarchical cluster analysis ・Sentiment analysis ・Sentence summarization |
AI text mining by user local will introduce analysis examples later, so please take a look!
②KH Coder
Multivariate analysis allows you to analyze text features from groups of words that occur frequently in the text, or from groups of texts that contain the same words.
Since the source code is open to the public, you can also customize it for your own purposes.
The five main features provided by this service are:
| ・KWIC Concordance You can search for sentences by specifying a word or part of speech. ・Correspondence analysis It maps words. ・Co-occurrence network Shows the relationships in which words commonly appear with circles and lines. ・Classification by Bayesian learning Training data can be created and classified by a naive Bayesian classifier. ・You can create your own aggregation by combining coding rules such as and and or. |
③ Statistics software R
It has comprehensive basic statistical processing functions, and text mining is also possible.
It is possible to perform fairly specialized analysis, but in order to use this service, knowledge of statistics and the acquisition of a programming language called “R” are required.
The official website explains everything from simple tutorials to analysis methods, so you can deepen your understanding of text mining while actually moving your hands.
Free AI text mining tool comparison
The following table summarizes the performance and information of the leading free AI text mining tools. Please use it as a reference for comparison.
| tool name | AI Text Mining by User Local | KH Coder | Statistics software R |
| environment | WEB | computer | computer |
| Character limit | 10,000 characters(200,000 characters for member registration) | No limit | No limit |
| sentence comparison | up to two | No limit | No limit |
| difficulty | ☆ | ☆☆ | ☆☆☆☆ |
How to choose an AI text mining tool
When choosing an AI text mining tool, be aware of the indicators such as whether it is easy to use (does it suit you) and whether the analysis accuracy is high (does it have the analysis items you want)? is recommended!
If you want even higher accuracy and versatility, consider paying tools as well.
Practice using tools!
Here, we will introduce how to use the AI text mining by user local tool and how to view the analysis results.
① Just enter in the text box!
The method of text analysis is quite simple.
Enter the desired text in the box. Here, let’s enter the sample Osamu Dazai’s “Run, Melos” on the site and see the analysis results.
(2) View analysis results
word cloud
A word cloud is a method of displaying words in different font sizes according to their importance in context.
Also, here the colors show the differences in the parts of speech of the words.
word frequency
Word frequency is a graphical representation of words by frequency of occurrence.
Here we use an index called ” score ” to indicate the importance of each word. For example, words that tend to appear in any sentence, such as “say” and “think,” will have a low score even if they appear frequently.
co-occurring keyword
A co-occurring keyword is a graphical representation of words that appear simultaneously or frequently with a certain keyword.
2D map
A two-dimensional map is a graphical representation of the tendency of words appearing in a set to appear close to each other.
In the figure, words that are close together are words that frequently appear together, and words that are far apart are words that appear together less frequently.
Dependency parsing
Dependency parsing is a parsing method for finding “modifier-modified” relationships between words.
Dependency analysis displays the analysis results for “adjectives”, “verbs”, and “nouns” related to “nouns”.
The “score” on this site is an independently calculated numerical value based on multiple judgments such as the number of occurrences and the ratio of the dependency relationship among all combinations. A higher “score” indicates that the dependency is more important.
If a word is followed by “(not: 50%)”, it means that 50% of the aggregated dependencies are used as negative expressions (e.g. “high” → “not high”). doing.
Negapositive indicates whether the adjective applied to the noun is a positive (negative) word.
In this way, with AI text mining by user local, you can obtain various analysis results simply by entering the text you want to analyze in the text box.
summary
This time, I explained the points about AI text mining, such as what kind of merits it has, what kind of situations it is used in, and how to do AI text mining!
I would appreciate it if you could deepen your understanding of AI text mining even a little.
