Correlation of Social Networks with Traditional Metrics of Impact on Scientific Journals in Nursing

Different bibliometric indexes allow researchers to evaluate the impact of scientific journals based on the number of citations received by their publications. However, the correlation of these indexes with alternative metrics that evaluate the presence of journals on social networks has not been evaluated in nursing journals. The objective of this study was to evaluate the correlation between the SCImago Journal Ranking Indicator (SJR) and alternative metrics of presence in four social networks (Twitter, Facebook, YouTube, Instagram) on indexed nursing journals. A correlation study was conducted in March 2019 through which the nursing journals included in the SJR were identified. Out of a total of 131 journals, 67 were excluded because they were active on social networks that were not their own. 64 were included for analysis. The most frequently used social networks were Twitter (75%) and Facebook (75%). The journals with presence on social networks had higher values for H Index (36.5 vs 12.0, p = 0.00037) and SJR (0.452 vs 0.268, p = 0.0069), and a higher number of publications (≥ 500 publications in 3 years, p = 0.03 ) than those without social networks. The correlation between the SJR and the number of followers on Twitter (r: - 0.067) and the number of followers on Facebook (- 0.18) were poor and in some cases negative. We concluded that the use of social networks as a means of dissemination and interaction by nursing journals is high, being used as a strategy of visibility and dissemination of journal contents.

Introduction

One of the stages of the research process is disclosure of results. Journals are the leading medium chosen by researchers for this task, since their purpose is to disseminate scientific knowledge. However, most journals are unaware of the importance of using social networks as complementary tools for increasing visibility and number of citations in less time, to facilitate feedback between researchers, and increase open access to information. Arcila, Calderín and Sánchez (2019) pointed out that the diffusion of knowledge through social networks accelerates the process of reaching wider audiences with results, increases visibility and allows open learning.

Research processes are becoming increasingly collaborative. Thus, communication between teachers, researchers, academicians and scientists becomes relevant. It is necessary to explore the importance of using social networks as part of the research and editorial processes of journals and researchers, so that these studies could serve as benchmarks for assessing the projection and visibility of journals, according to activity on social networks. Nonetheless, according to Soler-Tovar (2015), the analysis of the use of social networks should be accompanied by a review of the impact this use generates on the citation index, since the use of social networks is expected to increase the journals’ impact within the target audience.

The relationship between the calculation of bibliometric indexes of production and impact has become popular as a measure of comparison between different journals. Even so, according to Sarthou (2016), there are limitations and results may present some biases. Thus, the utility for evaluating certain areas of knowledge decreases. As The Universidad de Huelva (2018) mentioned, alternative metrics (altmetrics) are alternative indicators based on the 2.0 web or the social web that are useful for analyzing the activity and visibility of scientific and academic information on social networks.

Social networks have become important tools for the post-publication process of scientific articles, since these networks allow for the generation of reports with indicators on use, downloads, and views, among other relevant information. For Haustein, Peters, Sugimoto, R., Thelwall, & Larivière (2014), these tools reflect social media usage, data which are frequently overlooked for calculating academic impact.

Currently, neither traditional metrics of scientific impact based on the number of citations, nor metrics of presence in social networks, have been studied for nursing journals. The few studies aiming to assess this topic have addressed the use of social networks by researchers and the networks’ usefulness in clinical practice. Therefore, the objective of this article is to describe the presence of nursing journals in social networks and to determine if there is a correlation between the SJR (a traditional metric based on the number of citations) and metrics of presence in social media.

Methods

We performed a correlation study in March 2019. We identified journals categorized as nursing journals on the SCImago Journal & Country Rank website. This is an openly available, free-of-charge tool, which calculates the impact of scientific journals based on the number of citations a specific journal appearing in the Scopus database of published articles receives.

Regarding journal characteristics, the following variables were evaluated:

• the geographical area of publication;
• the SCimago Journal Ranking Indicator (SJR), a traditional metric based on the number of citations adjusted by the number of self-citations and the journal where said citations appeared;
• the quartile (indicator used to evaluate the relative importance of a journal within the total number of journals in its area);
• the H Index (based on the distribution of the citations that scientific papers have received);
• access to information (Open Access vs Non Open Access); and
• the number of publications in the previous 3 years.

To assess the presence of journals on social networks, the accounts of each of the analyzed journals were identified by accessing links on the official pages or by searching directly on the social network. The degree of activity in each of the social networks was evaluated through the following indicators:

• Facebook: Number of followers, date of creation of the account.
• Twitter: Number of followers, number of tweets, date of creation of the account.
• YouTube: Number of subscriptors, number of videos, number of views of the most seen video, date of creation of the account.
• Instagram: Number of followers, number of publications, date of creation of the account.

The differences between journals with or without social networks were evaluated through a non-parametric U Mann Whitney test. The global correlation and correlation within subgroups were evaluated by calculating a Spearman correlation coefficient. Statistical significance was defined by a p value below 0.05. Data processing was conducted through the StataCorp. 2017. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC.

Results

We identified 131 nursing journals. Of these, 64 were present on social media, and 67 journals were excluded for not having a social network or because they could not be identified as the official accounts for the journal.
The journals that were present on social media had higher H index (36.5 vs 12.0, p = 0.00037) and SJR values (0.452 vs 0.268, p = 0.0069) than journals without presence on social media. Regarding the quartile (Q) classification, most were indexed as Q1 (Q1 corresponds to 25% of the journals of the nursing area with the highest citation index in SJR) and more frequently had social networks (p = 0.05). The greater proportion of journals with social networks was found in Europe. It is striking that journals without a social media presence and a greater number of publications within the previous three-year period (n = 249) are most frequently Open Access journals. (Table 1).

Table 1. Sample characteristics according to presence of social networks

Regarding social networks, the most frequently used networks by journals were Twitter (75%) and Facebook (75%), YouTube was less used, and none of the evaluated journals had an active account on Instagram. Of the social networks with the largest number of followers, Twitter predominated (median: 1041,5 - IQR: 3273,35). (Table 2).

Table 2. Description of activity on social networks

The correlation is negative in most cases; this means that as the number of followers increases, the citation index tends to decrease, except for journals classified as Q3. This could be explained by a small number of journals classified as Q3 and the variability in the use of social networks, specifically Twitter. The overall correlation was very low and statistically not significant. A positive correlation was found despite the smaller amount of journals with a Twitter account as compared to those classified within Q1 and Q2. It is interesting that most journals with Twitter accounts are non Open Access. (Table 3).

Table 3. Correlation between the SJR & different alternative metrics of presence on Twitter

Unlike journals with presence on Twitter, those with accounts on Facebook are mostly Open Access. It is important to note that there is a negative association between the number of followers per year and the citational index (Table 4).

Table 4. Correlation between the SJR & different alternative metrics of presence on Facebook

Discussion

Our study shows that nursing journals are present on social networks in a high percentage, and that journals with social media obtained higher values in traditional impact metrics based on the number of citations. However, no positive correlation was found between the number of followers and the citation index.

Our results are in agreement with the survey carried out by Nature Publishing Group, which reported that 26% of respondents used the service for professional purposes and that most researchers use Twitter for personal purposes (Davis 2019). Tweets, for instance, tend to contain data that may be of benefit for researchers by making their academic work visible. The latter is due to tweets frequently including links to the articles; this may modify the classification of scientific literature in search engines, thus increasing the researcher’s visibility (Eysenbach, 2011).

According to Smith & Watson (2016), social media and alternative metrics should no longer be overlooked. Twitter makes it easier for research nurses to connect with communities in a quick, direct and affordable manner, thus promoting the use of evidence in daily practice (Archibald & Clark, 2014, p.19).

The ranking of journals may exert an influence on the citation index in an independent manner regarding the number or type of social networks where journals are present (Orduña & Martin, 2019). While there are many studies that explore the association between the citation index and alternative metrics, most have methodologically relevant limitations.

Journals classified within the third quartile (Q3) were the only ones that displayed a positive correlation despite fewer Q3 journals having a Twitter account in comparison with Q1 and Q2 journals. This positive correlation could be due to the ongoing wish of Q3 journals to continue to ascend in their classifications, since this ascension might lead to increased visibility. Since citations need time to accumulate, they are not the best indicator for defining the importance of a recently published manuscript. In response, some editors have turned to alternative metrics, which reflect the number of citations or mentions on specific social websites (Thelwall, Haustein, Larivière, and Sugimoto, 2013).

The most used social networks are Twitter by non open access journals, and Facebook for open access journals. This situation could be due to non open access journals being more restricted and their target audiences more specific. Even so, according to Castro, Ponce, Taype, Palma and Palacios (2014), not all social networks might be useful for scientific journals, since some have greater academic prestige. Networks such as Twitter are more visible among academics and researchers than others like Instagram, which emphasizes photography and interaction with people within close circles and users who share the same interests (Oropesa and Sanchez, 2016, p. 29).

Archibald & Clark (2014) suggested that metrics derived from social media usage in nursing are not currently recognized as a means for assessing impact. However, these metrics may predict citations, with some limitations, which may be interesting since citation counts usually take years to accumulate.

According to Alonso, Cordón and Maltrás (2016) the purpose of alternative metrics is not to replace traditional ones, but to complement them by adding a new perspective. For this reason, publishers and scientific societies should pay attention and make an effort to create specific social media accounts for their journals and to use other altmetrics.

Holmberg and Thelwall (2017) mentioned that social networks allow authors to take actions that aim to increase visibility of articles soon after publication. Journals should consider Twitter as an important tool to expand the target public, as well as to monitor the impact of their publications on social networks (Ortega, 2017, p.1).

Currently, contemporary nurses are taking into account which social networks journals have when choosing the best option for submitting their research. There is a special interest in Twitter, due to its reach and visibility potential (Dardas, Woodward, Scott, Xu, & Sawair, 2019).

Limitations

Some of the limitations of this study include evaluation according to a single source of information (Scimago Journal Rank) and the period of time established for the collection of the information. The study could yield limited data because the behavior of the number of followers is dynamic and can change constantly. Another important aspect is the selection of citation index by journals, and not by articles. Furthermore, the study’s design and methods do not allow assessments of the type of users of social media, the number or the frequency of use of publications, or videos or tweets containing specific information on a particular article.

Conclusions

Our data suggested that the use of social networks by nursing journals is not associated with an increase in the citation index, so both metrics provide complementary information. According to Davis (2019), the presence on social networks becomes a strategy of visibility and increased reception on the web, expanding the scope of a publication’s impact beyond the scientific community. For this reason, it is necessary that nursing journals and researchers in the area know academic social networks and their usefulness in scientific activity. The utility of social networks is highlighted, especially Twitter as a post-publication visibility tool.

Abstract

Different bibliometric indexes allow researchers to evaluate the impact of scientific journals based on the number of citations received by their publications. However, the correlation of these indexes with alternative metrics that evaluate the presence of journals on social networks has not been evaluated in nursing journals. The objective of this study was to evaluate the correlation between the SCImago Journal Ranking Indicator (SJR) and alternative metrics of presence in four social networks (Twitter, Facebook, YouTube, Instagram) on indexed nursing journals. A correlation study was conducted in March 2019 through which the nursing journals included in the SJR were identified. Out of a total of 131 journals, 67 were excluded because they were active on social networks that were not their own. 64 were included for analysis. The most frequently used social networks were Twitter (75%) and Facebook (75%). The journals with presence on social networks had higher values for H Index (36.5 vs 12.0, p = 0.00037) and SJR (0.452 vs 0.268, p = 0.0069), and a higher number of publications (≥ 500 publications in 3 years, p = 0.03 ) than those without social networks. The correlation between the SJR and the number of followers on Twitter (r: - 0.067) and the number of followers on Facebook (- 0.18) were poor and in some cases negative. We concluded that the use of social networks as a means of dissemination and interaction by nursing journals is high, being used as a strategy of visibility and dissemination of journal contents.

Introduction

One of the stages of the research process is disclosure of results. Journals are the leading medium chosen by researchers for this task, since their purpose is to disseminate scientific knowledge. However, most journals are unaware of the importance of using social networks as complementary tools for increasing visibility and number of citations in less time, to facilitate feedback between researchers, and increase open access to information. Arcila, Calderín and Sánchez (2019) pointed out that the diffusion of knowledge through social networks accelerates the process of reaching wider audiences with results, increases visibility and allows open learning.

Research processes are becoming increasingly collaborative. Thus, communication between teachers, researchers, academicians and scientists becomes relevant. It is necessary to explore the importance of using social networks as part of the research and editorial processes of journals and researchers, so that these studies could serve as benchmarks for assessing the projection and visibility of journals, according to activity on social networks. Nonetheless, according to Soler-Tovar (2015), the analysis of the use of social networks should be accompanied by a review of the impact this use generates on the citation index, since the use of social networks is expected to increase the journals’ impact within the target audience.

The relationship between the calculation of bibliometric indexes of production and impact has become popular as a measure of comparison between different journals. Even so, according to Sarthou (2016), there are limitations and results may present some biases. Thus, the utility for evaluating certain areas of knowledge decreases. As The Universidad de Huelva (2018) mentioned, alternative metrics (altmetrics) are alternative indicators based on the 2.0 web or the social web that are useful for analyzing the activity and visibility of scientific and academic information on social networks.

Social networks have become important tools for the post-publication process of scientific articles, since these networks allow for the generation of reports with indicators on use, downloads, and views, among other relevant information. For Haustein, Peters, Sugimoto, R., Thelwall, & Larivière (2014), these tools reflect social media usage, data which are frequently overlooked for calculating academic impact.

Currently, neither traditional metrics of scientific impact based on the number of citations, nor metrics of presence in social networks, have been studied for nursing journals. The few studies aiming to assess this topic have addressed the use of social networks by researchers and the networks’ usefulness in clinical practice. Therefore, the objective of this article is to describe the presence of nursing journals in social networks and to determine if there is a correlation between the SJR (a traditional metric based on the number of citations) and metrics of presence in social media.

Methods

We performed a correlation study in March 2019. We identified journals categorized as nursing journals on the SCImago Journal & Country Rank website. This is an openly available, free-of-charge tool, which calculates the impact of scientific journals based on the number of citations a specific journal appearing in the Scopus database of published articles receives.

Regarding journal characteristics, the following variables were evaluated:

• the geographical area of publication;
• the SCimago Journal Ranking Indicator (SJR), a traditional metric based on the number of citations adjusted by the number of self-citations and the journal where said citations appeared;
• the quartile (indicator used to evaluate the relative importance of a journal within the total number of journals in its area);
• the H Index (based on the distribution of the citations that scientific papers have received);
• access to information (Open Access vs Non Open Access); and
• the number of publications in the previous 3 years.

To assess the presence of journals on social networks, the accounts of each of the analyzed journals were identified by accessing links on the official pages or by searching directly on the social network. The degree of activity in each of the social networks was evaluated through the following indicators:

• Facebook: Number of followers, date of creation of the account.
• Twitter: Number of followers, number of tweets, date of creation of the account.
• YouTube: Number of subscriptors, number of videos, number of views of the most seen video, date of creation of the account.
• Instagram: Number of followers, number of publications, date of creation of the account.

The differences between journals with or without social networks were evaluated through a non-parametric U Mann Whitney test. The global correlation and correlation within subgroups were evaluated by calculating a Spearman correlation coefficient. Statistical significance was defined by a p value below 0.05. Data processing was conducted through the StataCorp. 2017. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC.

Results

We identified 131 nursing journals. Of these, 64 were present on social media, and 67 journals were excluded for not having a social network or because they could not be identified as the official accounts for the journal.
The journals that were present on social media had higher H index (36.5 vs 12.0, p = 0.00037) and SJR values (0.452 vs 0.268, p = 0.0069) than journals without presence on social media. Regarding the quartile (Q) classification, most were indexed as Q1 (Q1 corresponds to 25% of the journals of the nursing area with the highest citation index in SJR) and more frequently had social networks (p = 0.05). The greater proportion of journals with social networks was found in Europe. It is striking that journals without a social media presence and a greater number of publications within the previous three-year period (n = 249) are most frequently Open Access journals. (Table 1).

Table 1. Sample characteristics according to presence of social networks

Regarding social networks, the most frequently used networks by journals were Twitter (75%) and Facebook (75%), YouTube was less used, and none of the evaluated journals had an active account on Instagram. Of the social networks with the largest number of followers, Twitter predominated (median: 1041,5 - IQR: 3273,35). (Table 2).

Table 2. Description of activity on social networks

The correlation is negative in most cases; this means that as the number of followers increases, the citation index tends to decrease, except for journals classified as Q3. This could be explained by a small number of journals classified as Q3 and the variability in the use of social networks, specifically Twitter. The overall correlation was very low and statistically not significant. A positive correlation was found despite the smaller amount of journals with a Twitter account as compared to those classified within Q1 and Q2. It is interesting that most journals with Twitter accounts are non Open Access. (Table 3).

Table 3. Correlation between the SJR & different alternative metrics of presence on Twitter

Unlike journals with presence on Twitter, those with accounts on Facebook are mostly Open Access. It is important to note that there is a negative association between the number of followers per year and the citational index (Table 4).

Table 4. Correlation between the SJR & different alternative metrics of presence on Facebook

Discussion

Our study shows that nursing journals are present on social networks in a high percentage, and that journals with social media obtained higher values in traditional impact metrics based on the number of citations. However, no positive correlation was found between the number of followers and the citation index.

Our results are in agreement with the survey carried out by Nature Publishing Group, which reported that 26% of respondents used the service for professional purposes and that most researchers use Twitter for personal purposes (Davis 2019). Tweets, for instance, tend to contain data that may be of benefit for researchers by making their academic work visible. The latter is due to tweets frequently including links to the articles; this may modify the classification of scientific literature in search engines, thus increasing the researcher’s visibility (Eysenbach, 2011).

According to Smith & Watson (2016), social media and alternative metrics should no longer be overlooked. Twitter makes it easier for research nurses to connect with communities in a quick, direct and affordable manner, thus promoting the use of evidence in daily practice (Archibald & Clark, 2014, p.19).

The ranking of journals may exert an influence on the citation index in an independent manner regarding the number or type of social networks where journals are present (Orduña & Martin, 2019). While there are many studies that explore the association between the citation index and alternative metrics, most have methodologically relevant limitations.

Journals classified within the third quartile (Q3) were the only ones that displayed a positive correlation despite fewer Q3 journals having a Twitter account in comparison with Q1 and Q2 journals. This positive correlation could be due to the ongoing wish of Q3 journals to continue to ascend in their classifications, since this ascension might lead to increased visibility. Since citations need time to accumulate, they are not the best indicator for defining the importance of a recently published manuscript. In response, some editors have turned to alternative metrics, which reflect the number of citations or mentions on specific social websites (Thelwall, Haustein, Larivière, and Sugimoto, 2013).

The most used social networks are Twitter by non open access journals, and Facebook for open access journals. This situation could be due to non open access journals being more restricted and their target audiences more specific. Even so, according to Castro, Ponce, Taype, Palma and Palacios (2014), not all social networks might be useful for scientific journals, since some have greater academic prestige. Networks such as Twitter are more visible among academics and researchers than others like Instagram, which emphasizes photography and interaction with people within close circles and users who share the same interests (Oropesa and Sanchez, 2016, p. 29).

Archibald & Clark (2014) suggested that metrics derived from social media usage in nursing are not currently recognized as a means for assessing impact. However, these metrics may predict citations, with some limitations, which may be interesting since citation counts usually take years to accumulate.

According to Alonso, Cordón and Maltrás (2016) the purpose of alternative metrics is not to replace traditional ones, but to complement them by adding a new perspective. For this reason, publishers and scientific societies should pay attention and make an effort to create specific social media accounts for their journals and to use other altmetrics.

Holmberg and Thelwall (2017) mentioned that social networks allow authors to take actions that aim to increase visibility of articles soon after publication. Journals should consider Twitter as an important tool to expand the target public, as well as to monitor the impact of their publications on social networks (Ortega, 2017, p.1).

Currently, contemporary nurses are taking into account which social networks journals have when choosing the best option for submitting their research. There is a special interest in Twitter, due to its reach and visibility potential (Dardas, Woodward, Scott, Xu, & Sawair, 2019).

Limitations

Some of the limitations of this study include evaluation according to a single source of information (Scimago Journal Rank) and the period of time established for the collection of the information. The study could yield limited data because the behavior of the number of followers is dynamic and can change constantly. Another important aspect is the selection of citation index by journals, and not by articles. Furthermore, the study’s design and methods do not allow assessments of the type of users of social media, the number or the frequency of use of publications, or videos or tweets containing specific information on a particular article.

Conclusions

Our data suggested that the use of social networks by nursing journals is not associated with an increase in the citation index, so both metrics provide complementary information. According to Davis (2019), the presence on social networks becomes a strategy of visibility and increased reception on the web, expanding the scope of a publication’s impact beyond the scientific community. For this reason, it is necessary that nursing journals and researchers in the area know academic social networks and their usefulness in scientific activity. The utility of social networks is highlighted, especially Twitter as a post-publication visibility tool.

Online Journal of Nursing Informatics

Powered by the HIMSS Foundation and the HIMSS Nursing Informatics Community, the Online Journal of Nursing Informatics is a free, international, peer reviewed publication that is published three times a year and supports all functional areas of nursing informatics.

Read the Latest Edition