3.1. Data Collection

In the previous study by Na (2015), which this paper seeks to extend, articles from the top 70 journals in psychology were first chosen according to their 2013 impact factors in the Thomson Reuters Social Science Citation Index. Due to the large amount of tweet data, further data selection was done. First, all the articles were ranked by the number of mentions on Twitter. Then, recent tweets of the top ranked articles (around 133 articles) were collected from altmetric.com. For each article, up to 20 tweets in English were collected. Identical repeated tweets were removed while retweets were included. In the final selected dataset, there were a total of 2,016 tweets for analysis.

Data from altmetric.com also includes Twitter screen names of all the 2,016 tweets. Therefore, for this study, additional data, i.e. profile details of each user, were collected using the available screen names from Twitter through API. In the final dataset, name, screen name, description, profile image, time zone, location, statuses count, followers count, and friends count were included.

3.2. User Motivations

Motivations of users were the primary focus of the previous study by Na (2015). Therefore, various types of motivations were reported in a comprehensive manner with 10 main categories and 33 subcategories. Our intention in this current study, however, is to identify key motivations and their relationship with different user types. Therefore, we revised previous categories into 4 main categories (Discussion, Sharing, Promotion, and Access) and 15 sub-categories, as shown in Table 8. We maintained the coding of tweets done in the previous study by Na (2015) while we renamed and revised categories in this study. As in the original study, tweets were not mutually exclusive and a tweet can appear in more than one category or subcategory. In the table, count column shows the number of tweets in the specified motivation category or subcategory for the specified user type. Percentages in percentage column were calculated based on the total number of tweets in the respective user type as shown in Tables 4 and 5. Below is a brief description of each category and their subcategories.

Discussion: Tweets that express personal preference, personal interpretation, opinion, thoughts, criticism, and doubt were categorized under Discussion. We also kept tweets that raised questions, invited further discussion, and displayed criticism in this category. Tweets in the Discussion category were mostly different types of conversation and showed a deeper level of engagement with information in the article compared to those in the ‘Sharing’ category, where the intentions of tweets were more to share the article and related resources or to spread the findings in the article without much personal interpretations. There are six subcategories in Discussion:

D1: Expressing insights or personal interpretation of the article Sample tweet: “Pressure’s off. Study: NO connection btw family meals & kids beh/academics. Grab a slice, mama ain’t cooking 2night! http:// t.co/9Lfjly1r”

D2: Expressing personal experience or thoughts in relation to the article Sample tweet: “Godammit they published first. I started this research shortly after puberty and now I’m beaten to the punch? WTF! http://t.co/ Bau09RBo”

D3: Expressing personal preference, approval, or recommending the article Sample tweet: “This is great social science & its free! Family meals and children’s behavioral outcomes http://t.co/oBRCRyad @WBPsychology”

D4: Raising a question to think Sample tweet: “Sacrificing sleep time for study can backfire (diary study) - a lesson for our students? http://t.co/OLt5Dxzy”

D5: Inviting further discussion Sample tweet: “Arrest followed by first contact with mental health for a good minority - prodromal symptoms? http://t.co/BeGLVsxz via @ Offender_Health”

D6: Criticizing or questioning full or part of the article Sample tweet: “@bimadew Report is wishywashy, but analysis of how praise impacts learning is important, e.g. effort vs attainment. http://t.co/FjTvw2XKwy”

Sharing: Tweets in this category did not convey any opinion or interpretation but instead, the main motivation seems mainly to share the article or related resources, or spread the research findings. There are four subcategories in Sharing:

S1: Highlighting the summary or findings of the article Sample tweet: “Prevalence of mental illness among offenders (Australian) is only 11.1%; violence=/=mental illness http://t.co/21gfpQax”

S2: Simple sharing of the article with title or part of title with the link Sample tweet: “To Study or to Sleep? http://t.co/ZhWnUugT (Also check my Blog for more on the importance of sleep for all of us.) http://t.co/H570RY95...”

S3: Sharing to specific friends, specific groups of people, or directing the article to their followers. Sample tweet: “@PhillipAdamsABC You might be interested in this boost for secularism— protects against depression http://t.co/VNtXlr5b”

S4: Retweeting / Heard Through (HT) / MT / via Sample tweet: “RT @MentalHealthCop: Australian study finds offending predates onset of illness for most people with schizophrenia - http://t.co/BeGLVsxz via @Offender_Health”

S5: Sharing resources, posts, talks, activities, etc. related to the article Sample tweet: “Due to the relevance for understanding Karl Friston’s talk at #ik2014: Clark, A. (2013) - http://t.co/kxEjE04Gr8”

Promotion: Tweets under the Promotion category were those by authors and organizations or institutions tweeting their own publications.

P1: Sharing of author’s own work Sample tweet: “Our #NYT paper is up http://t.co/Mycamqnc”

P2: Sharing of own publications by a publisher or an institution Sample tweet: “Should we believe that 1 in 68 children has #autism? Maybe not. New in our journal http://t.co/G8B6BeOeGv @DSMandell”

Access: Tweets under the Access category were those inquiring about or providing access to articles.

A1: Asking for information to access papers Sample tweet: “Alright, this high heel study...can anyone get the paper? http://t.co/cdqu8lm4”

A2: Providing information to access papers Sample tweet: “Here is the full article: http://t.co/ydKfsU8o”

3.3. User Types

Based on the name, description, and profile image from their Twitter profile, users were categorized into four main types (academic vs. non-academic, and individual vs. organization). The nature of users in each type is shown in Table 1. Users were categorized as unknown if the information in their profile is not clear or is missing. Similar to the classification of motivations, classifications of users into various types were not mutually exclusive and a user could be classified into more than one type. For example, a user who describes himself as a psychologist as well as researcher will appear in both academic (researcher) and non-academic (a psychologist here is considered a professional) types.

In addition to four main types, we also explored gender and geographic distribution. Individual users were categorized into male or female categories based on their first name and profile picture. In Twitter profiles, self-reported location information is available. However, we found that information is less reliable and sometimes ambiguous, such as “Earth,” “Mare Incognitum,” and “Middle of the media bubble.” Therefore, geographic information was determined based on time zone information in their Twitter profile.

3.4. Distribution of Users in Each Motivation Category

In addition to analyzing user profiles and their motivations separately, we also looked into distribution of individual vs. organization and academic vs. non-academic users in each motivation category. A chi-square test of independence was performed to examine the relation between motivations in different user types.

4.1. Geographic Distribution

We began by examining geographic distribution of users sharing academic articles on Twitter. We found that the majority of users came from regions where English speaking countries are located. As shown in Table 2, about 86% of the users are from North America and Europe. It is most likely due to the fact that tweets in English language only were included in our sample. However, the general Twitter population at global level, as reported by Kulshrestha et al. (2012), also revealed the same pattern. In their study, with 57% of the total Twitter population, the US has the highest percentage of Twitter users, followed by the UK, the second highest with 7.3%.

This points out that, if tweets for research articles are in fact mainly in the English language, Twitter as an altmetric measure has a Western bias. Since geographic information of users and followers has important implications, more comprehensive studies are needed to further investigate this area.

4.2. Distribution of User Types

Out of the total 2,016 tweets, 1,968 tweets were identifiable as tweeted by either individuals or organizations. Table 3 shows the number of tweets and the number of unique users respectively. Out of 1,968 tweets, about 77% were tweeted by individuals while 23% were by organizations. We further divided individuals and organizations into academic and non-academic types as shown in Table 4 and Table 5. In the individual category, the number of non-academic users and tweets by non-academic users are twice more than that of academic users and tweets (Table 4).

On the other hand, in the organization category, the number of tweets by academic organizations, including publishers, was higher compared to the number by non-academic organizations, with 61% and 39% respectively (Table 5). Despite the lower number of tweets by non-academic organizations, the number of non-academic organizations is higher. Overall, the number of both tweets and Twitter users of non-academic type is higher than that of the academic type (Table 6), with 35% academic and 65% non-academic users, and 43% tweets by academic and 57% tweets by non-academic users. These findings lead to the conclusion that Twitter as an altmetric source better exposes the impact of research on the general public than on academia. In the context of this paper, the general public includes professionals, scientists, and journalists as shown in Table 1.

4.3. Gender

Table 7 shows distribution of male and female Twitter users and the number of tweets they share. It was found that the number of males and the number of tweets by males are twice more than that of females and tweets by females. This finding is in line with the result from a Twitter demographic study by Murthy, Gross and Pensavalle (2016). In their study of 275 million tweets from December 2011 to March 2013 from American cities, a dominance of male users was found.

4.4. Distribution of Users in Each Motivation Category

Table 8 shows the distributions of individual vs. organization and academic vs. non-academic users in each motivation category. Our results show that, out of four main motivation categories, a large majority of tweets fell under the category of Sharing. Among the subcategories of Sharing, “S1 highlighting the summary or findings of the article” had the highest number of tweets, with around 43% to 57% of tweets in each user types. S1 is also significantly higher than the similar subcategory “S2 - simple sharing of the article with title or part of title with the link,” which had only around 13% to 21% of tweets in each user type.

This finding revealed that the majority of users did not just copy the title of the paper but instead, they did try to find out the most significant parts (summary and findings) of the article and try to communicate it with their followers within Twitter’s limit of 140 characters. It also pointed out the fact that tweeting a paper on Twitter in most cases might not just be a mindless sharing of the article, but it carried the indication of the consumption of information, and the attempt to communicate it through a channel, which is often touted as one of the fastest sources of news. In this sense, tweets seems to be a new way of communicating research findings.

After S1 and S2, the next subcategories in rank were “S4 - Retweeting / Heard Through (HT) / MT / via” with 16% to 23% of tweets in each user type. Retweets on Twitter, if retweeted multiple times, become “a community- driven phenomenon” and, if accompanied by an appropriate hashtag, become a trending topic (Nations, 2017; Doctor, 2012). This is a way of spreading discussions in our current social media age. Retweets of academic papers therefore may be considered as indications of interest and the significance of information in the paper which is worth spreading. It is important to note here that sentiment analysis was also done on the same data set by Na (2015) and found that negative and partially negative sentiments are only around 5% in all tweets. Therefore, as with the case of traditional citation, tweet citations mainly reflect the positive impact of the paper. The last two subcategories in Sharing covered only 2% to 12% of the tweets in each user type.

A significant number of tweets were found in the Discussion category as well. The most common subcategory in Discussion is “D3 – Expressing personal preference, approval, or recommending the article,” which covered around 21% to 23% of tweets by academic users in both individuals and organizations and around 16% to 19% of tweets by non-academic users in both types. A similar subcategory is “D1- Expressing insights or personal interpretation of the article” with 3% to 7% in each user type. The remaining three subcategories covered only 1% to 7% of tweets in each user type. The least popular subcategory is “D2 - Expressing personal experience or thoughts in relation to the article” with around 0.4% to 4% users in each user type. In short, the Discussion category showed that a fair number of tweets displayed various types of engagement with articles and the most common form is expressing a positive attitude towards the articles.

The Promotion category revealed that 18% of tweets were from academic organizations, out of which about 5% were from publishers. Other than that only 0% - 5% of tweets were from individual authors promoting their own publications. Gaming and self-citation are widespread concerns when it comes to social media based metrics such as tweets. Results from this study indicated that such cases were not common enough to significantly reduce the value of tweets as a measure of impact. However, more comprehensive studies are necessary to confirm this aspect. The last category of Access covered only 0% - 5% of all tweets in each user type. While sharing information about the articles is common, as shown in the Sharing category, it seems that sharing and asking for full text articles are not common motivations when tweeting articles on Twitter.

Turning to the results of chi-square tests in Table 9, we found that there were significant differences in motivations between individual and organization users in both Discussion and Sharing categories. The main areas of difference in the Discussion category seemed to be in “D2 - expressing personal experience or thoughts in relation to the article” and “D6 - criticizing or questioning full or part of the article.” Out of all individual users, 6% (2.6%+3.8% = 6.2%) were found to express personal experiences or thoughts in relation to the article compared to only about 1.5% (0.36%+1.13%=1.49%) of organization users. Individuals also criticized or questioned full or part of the article nearly twice more. In the Sharing category, it was found that, when sharing articles, a higher proportion of individuals highlighted summaries or findings, compared to organization users. On the other hand, a higher proportion of organization users used only the article title, either full or part, when tweeting articles. Based on these findings, we can conclude that individual users demonstrated a more critical use of articles than organization users do.

Chi-square results also indicated significant differences in academic and non-academic organizations in the Discussion category. Compared to academic organizations, a higher percentage of non-academic organizations were involved in all discussion types except “D3 - expression personal preference, approval ,or recommending the article.” Between academic and non-academic individual users, academic individuals were slightly more involved in “D3 - expressing personal preference, approval, or recommending the article,” “D4 - raising a question to think,” and “D5 - inviting for further discussion,” while non-academic individuals were in “D1 - expressing insights or personal interpretation of the article,” “D2 - expressing personal experience or thoughts in relation to the article,” and “D6 - criticizing or questioning full or part of the article.” In the Sharing category, a slightly higher percentage of non-academic individuals were involved in “S3 - sharing to specific friends, groups or their followers” and “S4 - Retweeting / Heard Through (HT) / MT / via.” These differences were significant at a 10% level. In summary, there are significant but small differences in motivations between academic and non-academic individual users.