3.1. Data

A total of 10,286 posts were extracted using the crawler from Lis Links. The present study’s analysis is focused on the text dataset, so it is essential to harvest only text content. Therefore, the crawling program was written in R using the package “rvest” to extract the text data.

The extracted data was stored in CSV format. The extracted dataset consists of 10 attributes, namely PostID, PostUser, PostDate, PostTime, PostCategory, PostTitle, PostBody, PostView, PostReplyCount, and PostUrl. However, the goal of this study will suffice with the analysis of text data of PostBody. Therefore, we have considered the content of PostBody as it contains detailed information about the post. The sample of the dataset is shown in Table 1.

3.2. Data Pre-Processing

Data pre-processing is one of the key steps in all data-based research to remove unwanted noise from the data. However, this is essential in enhancing text data quality (Hariharakrishnan et al., 2017). Many studies have shown that the pre-processing of data is a critical and most time-consuming process (Press, 2016). The standard methods for pre-processing text data include lower casing of text data, stemming, lemmatization, and removal of punctuation, numbers, HTML tags, special characters, stopwords, etc. However, the type of methods used for pre-processing the data is application-dependent (Ignatow & Mihalcea, 2017). For example, we called two additional ways in this study before going for general pre-processing. The process was divided into two stages. Firstly, the blank posts and non-English posts were removed. The Lis Links also provide the functionality of posting images, documents, etc., in the discussion forum. The developed program for data extraction from Lis Links is limited to working with text data only. It does not extract anything other than text from the post, so the program will not harvest anything from the post containing only an image, document, etc., resulting in blank posts. Fig. 2 shows the example of a post where the image was embedded in the post body. Also, there is a possibility that somebody has posted blank posts without any messages. Fig. 3 shows the example of one such post without any content.

Fig. 2

Example of post containing image and extracted blank.

Fig. 3

Blank post.

The character count of the PostBody of each post was calculated using R to identify the blank posts. The character size of each post was stored in the primary data frame. Analysis found that the posts with a character count from 0 to 7 were blank posts, which have no relevant content, except for the one case where the post only has one word. Therefore, we have removed all these blank posts that were either blank or having less informative text content.

India is a diverse country with multi-language-speaking people. The 22 languages (including Hindi and English) are officially listed as per the Eighth Schedule of the Constitution of India (The Editors of Encyclopaedia Britannica, 2019). Lis Links supports the features of posting content in regional languages with the help of Unicode. It makes Lis Links a national platform to cater to the needs of all LIS professionals in India. Therefore, there is a possibility that posts may contain regional languages also. Each post of a regional language on the Lis Links is stored as a sequence of Unicode. The original post posted in the Hindi language is shown in Fig. 4. The table shows the same post extracted from the Lis Links server data.

Fig. 4

Regional language post.

The one word of Hindi language is stored as a sequence of Unicode character tags. The R program has fetched the text content and the Unicode of non-English posts.

The word “िदपावली” is made up of seven Unicode character tags: द (U+0926), “ि” (U+093F), “प” (U+092A), “ा” (U+093E), “व” (U+0935), “ल” (U+0932), and “ी” (U+0940). This is how each post in regional language can easily be identified consisting of Unicode characters. The presence of pattern <U+ was searched and stored as a separate data frame using the pattern matching function in R. These posts were checked manually to avoid any English posts and have some Unicode characters due to some unwanted space or character. The sample of one such post is shown in Table 2. Finally, those posts were considered for the final dataset, which was in English but had one or two Unicode characters. The other posts of regional language were identified and excluded from the final dataset.

In the second stage, URLs were removed from the data. Firstly, a regex was defined for all possible pattern matching of URLs. The URLs were extracted and removed based on this regex. The regex used for URL matching is defined as “(http[^ ]*)|(ftp[^ ]*)|(www\\.[^ ]*).” Similarly, the email mention was removed and stored as a separate CSV file. The regex used for email matching is defined as “\\S*@\\S*.”

The removal of URL and email strings may again leave some of the posts empty. These posts were identified and removed from the primary dataset. After removing URLs and emails from the posts, punctuation and white spaces were removed from the text corpus. The stopwords were removed from the dataset in the next step. The stopwords are words that do not contain much information about the analysis. The list of stop words of SMART (System for the Mechanical Analysis and Retrieval of Text) information retrieval system is the most popular in text analysis. This system was developed in the 1960s at Cornell University (Lewis et al., 2004). The SMART list consists of 571 stop words removed from the dataset (Hvitfeldt & Silge, 2021). The other popular stopword list is snowball stopwords given by Porter (n.d.). Apart from these, there can be some domain and platform-specific stopwords. The removal of these stopwords depends on the purpose of the study. Words like “dear,” “greetings,” “hello,” “hi,” “please,” “kindly,” etc., are commonly used words in writing any correspondence to a person or organization. These are also known as salutations. The use of salutations is done to establish strong communication. Previous studies have found that salutations were common practice in writing an email (Hiranburana, 2017; Lee, 2004). The web-based discussion forum interface and workflow are different from email-based forums. Therefore, we have not excluded these words from exploring the use of salutations on the discussion forums of Lis Links.

The text corpus of Lis Links was converted to lowercase to avoid the issue of similar words considered as different tokens. For example, the system finds Book, BOOK, book, BooK as four tokens, while the literal meaning of all these words is the same. After the lower-casing of the words, the system will identify the word “book” with a frequency of four.

The transformation of different words into root words is an important phenomenon in text analysis. It helps identify the similar token used differently as per the language’s grammar. For example, “Libraries” and “Library” convey the same meaning of context related to the library. It can be achieved by applying either stemming or lemmatization. Lemmatization is mapping the word with the root word with the dictionary meaning, while stemming is just matching to words after removal of smaller suffixes like ies, y, etc. This is why we have used lemmatization in this study to consider dictionary words for the analysis. It will also help to interpret the results quickly with the right words. The removal of numbers is a common practice in pre-processing of online data. However, we did not remove the numbers, as numbers also contain helpful information in the context of this study. The year of examination, admission, or question paper is the key information element to classify the discussion of LIS professionals. This is why we have not removed the numbers from the dataset.

3.3. Post-Pre-Processing

The pre-processed data was considered to visualize the most frequent terms and modeling of the topics. In this research, we have used LDA for extracting the topics from the text corpus. LDA is the most popular topic modeling technique in big data analysis (Garg & Rangra, 2022; Li & Lei, 2021).

The “topic models” package was used for LDA analysis (Grün & Hornik, 2011). The document term matrix (DTM) was created from the pre-processed corpus of the Lis Links discussion forum posts. As the term suggests, DTM is a matrix that consists of the number of occurrences of a term in the document.

The above two posts are pre-processed posts. The sample DTM of five terms from these two posts is shown in Table 3.

The above matrix shows that the term “exam” appeared once in both posts. The words “clear,” “focus,” “format,” and “humble” appeared only in the first post, while the second post has a zero count for these terms.

Now, this DTM was used as an argument for the LDA function of the package “topic models.” The value of k was set to 10. The k represents the number of topics to be extracted from the dataset. The value of k is the essential parameter in LDA analysis, describing the number of topics to be extracted from the text corpus. The smaller value of k represents the lesser number of general topics, while the larger value of k represents more specific topics. There is no best value of k for all studies, and its value varies for different datasets (Kahani et al., 2016).

Previous studies on mailing lists, an online forum of LIS professionals, have categorized discussions into 7 to 12 categories (Pujar et al., 2014; Siddique et al., 2020). Also, at present, Lis Links has nine (9) pre-defined categories for structuring discussions on the forums. Therefore, we have assumed the value of k to 10 based on the evidence of such previous studies on LIS domain and existing categories on Lis Links.

The extracted topics with their corresponding words were visualized as a Wordcloud. Finally, based on these ten topics, the posts were labeled. The frequency distribution of these topics was calculated.

3.4. Ethical Considerations

The analysis of content posted by people on online and social media platforms has become an emerging research topic among scholars of different academic discipline. These research efforts are supported by the easy availability of vast amounts of data and advanced tools for the whole analysis process. Popular platforms like Facebook, Twitter, and YouTube even provide APIs for accessibility of bulk content posted on these platforms. One essential question that arises here is, who has the right to the content posted on these platforms. Is it the user who has posted the content or the organization that owns the platform?

Sometimes online researchers misunderstand this. If they have access to a tool (data scraping tool) or know how to program, then no other permission is required to access the data. But this is not always the case. The users who posted their experience, reviews, etc., on the platform are generally unaware that the content will be analyzed in different research contexts. The user’s consent is a key aspect of any research involving human participants. Before starting research, mutual agreement between researchers and participants should be done regarding how their data will be used. It is common to declare this agreement when collecting data through questionnaires/surveys/interviews, etc. to maintain the confidentiality of the respondent. However, this is quite impossible in online spaces, as the number of users is large compared to traditional survey-based research.

Therefore, it is important for researchers to first understand privacy and data concerns before starting any research based in an online environment. A good number of studies have considered the analysis of public data as per ethical standards. The data is publicly available if the content on the online platforms is accessible without any restrictions or requirements of login/passwords (Betts et al., 2014; Buck & Ralston, 2021; Eastham, 2011).

The discussion forum of Lis Links is publicly available and does not require any password to see content posted by any of the users. However, a login is required to start a new post or reply to existing posts. Therefore, the discussion forum of Lis Link can be considered a publicly accessible platform. In this study, we have extracted only the text of the discussions posted on Lis Links. No personal information or images were collected for this research.

Because of the above discussion it was not deemed necessary to obtain consent from each user who posted content on the Lis Link discussion forum. However, written permission has been taken from the creator of Lis Links to pursue this research.

4.1. Wordcloud

Wordcloud is an easy-to-understand and powerful visual representation of the most frequent words in a text dataset (Bashri & Kusumaningrum, 2017; Dewi et al., 2020). The word cloud originated from the tag cloud used for Flickr (a social networking site) for multiple grouping of photographs (Miley & Read, 2011). In text analytics, word clouds are commonly used for presenting the overall picture of the text content, also known as text summarization (Heimerl et al., 2014). The font size of a word in the wordcloud is the indicator of the frequency of a word (Miley & Read, 2011). The most frequent word is displayed in larger font sizes than other words. This helps get an intuitive idea about the topics discussed in the text.

Fig. 5 illustrates the wordcloud of the 800 most frequent words in the Lis Links posts. The minimum frequency of the words was set to 20. Fig. 6 shows the frequency chart of the 25 most common words. The term “library” appeared in a larger size in pink color, followed by the word “information” in green. This shows the highest occurrence of the words “library” and “information” in the postings of Lis Links. This is not surprising, as Lis Links is a library domain-specific platform. The existence of the words “book,” “university,” and “librarian” shows that users have discussed institutional or profession-related things in the posts. The terms “UGC,” “question,” and “answer” indicate posts related to UGC examinations. The word “2012” in a significant font size refers to one of the issues related to the 2012 UGC NET (University Grant Commission National Eligibility Test) Exam. This was the year when Lis Links had the maximum number of posts. Upon further analysis of the dataset, the words “research,” “journal,” and “paper” were mainly used as a means of posting the discussion related to research. Some posts also appeared to be related to call for papers, data collections using surveys, etc. Thus the frequent words gave an intuitive idea that discussions related to examinations were common in the forums. The most frequent words related to examinations were “application,” “question,” and “answer.” The salutation words “dear” and “kindly” show that LIS professionals post discussions with due gratitude.

Fig. 5

Wordcloud of frequent words in discussions on Lis Links.

Fig. 6

Twenty-five most frequent words.

4.2. Topic Modeling

The basic assumption of the topic modeling algorithm is that each text document is distributed over a certain fixed number of topics and these topics consist of a specific group of words. The analysis retrieves a group of terms of 10 topics from the text corpus. The most prominent ten words of each topic are listed in Table 4. These groups of words for each topic were selected based on the beta score. The beta score is per topic per word probability. It estimates the occurrence of a word in each of the topics. The table shows the probability of the words “koha” and “software” in all ten topics. Analysis shows that both words have the highest probability for Topic 10 compared to other topics. Thus, the words “koha” and “software” were selected for Topic 10. Similarly, the words of other topics were considered based on high beta score.

Fig. 7 shows the comparative word cloud of the most probable words of all ten topics. The font size of the word is dependent on the beta score. The higher the beta score, the bigger the font size of the word will be. The analysis found that the words “librarian,” “post,” “college,” and “pay” belong to Topic 1; “2012,” “India,” and “current” are related to Topic 2; “dear,” “professional,” and “kindly” belong to Topic 3; “code,” “information,” and “classification” relate to Topic 4; “journal,” “research,” and “paper” relate to Topic 5; “ugc,” “net,” and “candidate” form Topic 6; Topic 7 consists of the words “library,” “book,” and “public”; “university,” “science,” and “department” are generated from Topic 8; “librarian,” “post,” and “pay” belong to Topic 9; and “software,” “koha,” and “online” form Topic 10.

Fig. 7

Comparative Wordcloud of words in each ten topics.

LDA analysis identifies the per-document-per-topic probability of representation of all ten topics in the post. It is a probability of estimating the proportion of words from the post that formed the given topic. In the next step, the important task is to select the best topic discussed in the post. This can be achieved in multiple ways based on the type of application, i.e., either by the highest probability or by setting the threshold value (Barravecchia et al., 2022). Each one of the approaches has its limitations. But in the literature, the standard approach for selecting a topic is based on the highest probability (Jelodar et al., 2019). All the posts were classified based on the high per-document-per-topic probability, also known as a gamma score.

Table 5 shows the assignments of ten topics from one sample post. The score of 0.2482 of Topic 1 of the post indicated that Topic 1 has a 24.82% representation in the post. The score of 0.5721 shows that 57.21% of the proportions of the post are related to Topic 10; similarly, 14.93% for Topic 3. A minor percentage of representation was observed for Topic 2 and from Topic 4 to Topic 9. This shows that the post consists mainly of Topic 10, Topic 1, and Topic 3, with a major representation of Topic 10. Therefore, Topic 10 has been assigned to the post. Similarly, the topic with a high probability (gamma score) was assigned as a topic with each post. Table 5 shows an example of all ten topics and their related probability of one post.

In the final step, it is necessary to transform these topics into human understandable topics. The title of the topics as Topic 1, Topic 2,...., and Topic 10 make no sense unless it is known what these topics are broadly discussing. Therefore, we have assigned the human-readable topics based on the constituents of words of each topic. The categorization of the posts by earlier studies on LIS discussions was also considered (Pujar et al., 2014; Siddique et al., 2020). The posts’ ten categories or topic names are shown in column 3 of Table 6.

Topic 1 is described by the words “librarian,” “post,” “college,” “assistant,” “pay,” “library,” “university,” “ugc,” “interview,” “school,” etc. It shows that Topic 1 is related to the “LIS Recruitment” post. Recruitment is the process of finding suitable employees for the organization. Generally, the discussion of recruitment is surrounded by the advertisement of the post, pay or remuneration of the post, specification/profile of the job, etc. Hence Topic 1 was named LIS Recruitment, consisting of messages where professionals discussed job-related matters. The sample post labeled as “Topic 1” or “LIS Recruitment” is shown in Table 7. The post discusses an advertisement for school librarians by DSSSB.

Topic 2 talks about the “LIS Issues” through the words “2012,” “India,” “download,” “current,” “affair,” “Indian,” “week,” “jun,” “world,” “bank,” etc. The LIS issues are all about the problems related to the LIS domain faced by people. The sample of the posts labeled as Topic 2 in Table 7 shows the concern of professionals.

Topic 3 represents the topic of “Other Discussion,” which consists of the words “dear,” “professional,” “lis,” “link,” “friend,” “kindly,” “library,” “information,” “member,” “share,” etc. It consists of posts related to multiple subtopics of LIS.

The fourth topic highlights the post related to “LIS Education” which consists of the words “library,” “information,” “answer,” “correct,” “ans,” “code,” “classification,” “TRUE,” “system,” etc. This means the discussions are related to LIS courses, theoretical concepts of LIS, etc.

The words “journal,” “library,” “information,” “research,” “paper,” “science,” “publish,” “international,” “author,” and “management” suggest that Topic 5 discusses “LIS Research.” The components of research include data collection, analysis, and publication. Therefore, we concluded that Topic 5 is related to LIS research.

Topic 6 consists of the words “ugc,” “net,” “candidate,” “paper,” “exam,” “answer,” “mark,” “question,” “result,” “test,” etc. These words revealed that Topic 6 talks about the post related to LIS exams. The examination process involves the candidate, question paper, answer key, marks of the question paper, exam name like UGC NET, exam results, etc.

The words “library,” “book,” “journal,” “college,” “public,” “year,” “professional,” “school,” “student,” or “dear” highlight that the posts were broadly discussing “General Information related to the Library.” It includes information about some special libraries and their services, or achievements of any particular library.

Topic 8 discusses admission notification and its related subtopics. The words include “university,” “library,” “science,” “information,” “India,” “research,” “librarian,” “department,” “phd,” or “prof.”

The analysis found that in Topic 9, library and professional activities were widely discussed. The top 10 words in this category are “library,” “work,” “information,” “make,” “read,” “librarian,” “time,” “knowledge,” “book,” and “professional.” The professional has a keen interest in organizing activities in the library to enhance usage.

In Topic 10, the generated words contain “library,” “software,” “user,” “koha,” “service,” “open,” “datum,” “access,” “system,” and “online.” A professional posted a post related to information and communication technology on this broader topic. The terms “software” and “koha” signify discussions related to open source software for library management systems like koha.

Table 8 shows the distribution of posts as per the revised category. The analysis found that most of the discussions were in the category of “Lis Exams.” It shows that LIS professionals are keen to seek or share information related to LIS exams. LIS exams is one of the key topics where study material, question papers, and proper guidance are required for passing the exam. In India, the popular LIS exams include UGC NET, UGC JRF (University Grant Commission Junior Research Fellowship), SET (State Eligibility Test), KVS (Kendriya Vidyalaya Sangathan), NVS (Navodaya Vidyalaya Samiti), MOOC, and university entrance exams. The UGC NET exam is a national level exam for qualifying the eligibility test for an assistant professor, assistant librarian, and jobs of the same cadre. These jobs are the highest job level at the entry level in India. UGC JRF is the national level exam for getting a fellowship for pursuing the MPhil/Ph.D. in India. Under this scheme, Rs. 25,000/- per month is given to each qualified candidate for the initial two years and Rs. 28,000 per month for the next two years with the annual contingency of Rs. 10,000/- per annum. It was also found that professionals actively interact on topics related to “General Information related to the Library” (14.50%, 1,442), LIS Recruitment (10.73%, 1,067), ICT (9.51%, 946), and LIS Admission (9.42%, 937). Few discussions were observed on the theme “LIS Issues” (1.86%, 185).