Skip to Main Content

Research Impact

How to measure impact

An author's impact on their field or discipline is usually measured by the number of academic publications and the number of times these publications were cited by other researchers. To demonstrate your impact as an author or a research group, you can create a list of your publications and the number of times they have been cited.

Different metrics are available that calculate author impact, using the data of the author's publications: H-Index, G-Index, M-Index, E-Index, and i10-Index. It is important to use metrics in combination with each other and not single out one specific metric, such as the H-Index.

H-Index

The h-index (short for highly cited index) was developed in 2005 by Professor Hirsch, a condensed matter physicist at the University of California in San Diego, to qualify the impact and quantity of an individual’s research performance.

The index is a measure of the number of highly impactful papers a scientist has published. The larger the number of important papers, the higher the h-index, regardless of where the work was published.

The h-index can therefore be regarded as a measure of the number of publications published (productivity) as well as how often they are cited. (impact).

Formula:

A scientist has index h if h of his or her NP (number of papers) papers have at least h citations each and the other (NP – h) papers have fewer than h citations each. 

E.g. a h-index of 20 means the researcher has 20 papers each of which has been cited 20+ times.

(Image source: Benchfly weblog, 20 Oct 2010)

Advantages of using the H-Index

  • It relies on citations to individual papers, not the journals, which is a truer measure of quality.
  • It is not skewed by a single well-cited, influential paper (unlike total number of citations would be).
  • It is not increased by a large number of poorly cited papers (unlike total number of papers would be).
  • It minimizes the politics of publication.  A high-impact paper counts regardless of whether it was published in the top-tier journals.
  • It’s good for comparing scientists within a field at similar stages in their careers.
  • It may be used to compare not just individuals, but also departments, programs or any other group of scientists.

Limitations of using the H-Index

  • It counts a highly-cited paper regardless of why it’s being referenced e.g, for negative reasons.
  • It doesn’t account for variations in average number of publications and citations in various fields (some disciplines traditionally publish and cite less than others).
  • It ignores the number and position of authors on a paper.
  • It limits authors by the total number of publications, so shorter careers are at a disadvantage.
  • It has relatively low resolution in that many scientists end up in the same range since it gets increasingly difficult to increase the h-index the higher it gets (an h-index of 100 corresponds to a minimum of 10,000 citations).
  • It, like all metrics, is based on data from the past and may not be a valid predictor of future performance. However, in a follow-up publication, Jorge Hirsch demonstrated that the h-index is better than other indicators (total papers, total citations, citations per paper) at predicting future scientific achievement.
  • Also read the following article about why the H-Index doesn't matter so much anymore: https://blog.impactstory.org/four-great-reasons-to-stop-caring-so-much-about-the-h-index/
  • Jorge Hirsch himself criticizes the overreliance on the H-Indexhttps://www.natureindex.com/news-blog/whats-wrong-with-the-h-index-according-to-its-inventor

G-Index

The g-index, proposed by Leo Egghe in 2006 aims to overcome a bias against highly cited papers inherent in the h-index. The g-index is the "highest number of papers of a scientist that received g or more citations, on average". It gives credit to researchers who have published high-quality papers and therefore tried to improve the h-index by giving more weight to highly-cited articles.

M-Index

The m-index, also proposed by Hirsch, is defined as h-index divided by the number of years since the researcher’s first publication.  The index is meant to normalize the h-index so that early- and late-stage scientists can be compared.  The m-index averages periods of high and low productivity throughout a career, which may or may not be reflective of the current situation of the scientist.

i10-Index

Used in Google Scholar since 2011, this index counts the number of publications with at least 10 citations.

E-Index

The e-index aims to address the number of “excess” citations above and beyond the h-index.  The e-index is defined as the square root of the sum of the “excess” citations in the papers that contributed to the h-index.

Metrics beyond indices

It is important to also include other metrics than only the different indices in your author analysis, such as the following metrics that might help you to track the relevant impact of your work:

  • The average number of citations per publication
  • Outputs in top citation percentiles
  • Outputs in top journal quartiles.

You will find these metrics in sources such as SciVal and some of them are included in the Researcher Impact Report that the Library can create for you.

Add sources and tools here

https://library.leeds.ac.uk/info/1406/researcher_support/17/research_metrics/2

Authors: https://library.leeds.ac.uk/info/1406/researcher_support/17/research_metrics/4

Leaders/groups: https://library.leeds.ac.uk/info/1406/researcher_support/17/research_metrics/5

Bath: https://library.bath.ac.uk/research-analytics/about

SciVal

  • Use the Overview module
  • Add Researcher
  • Explore all the metrics available in the Overview module.
  • For more metrics go to the Benchmark module and add all the metrics you need
    (A separate tab with more guidance about using SciVal will be available soon)

 

Scopus

  • Go to the Authors search option
  • It is possible to search under author name, ORCID and add an affiliation if necessary to refine the search.
  • The Scopus Author Profile page is generated by Scopus.
  • See Analyse results to view in-depth analysis.

 

Web of Science

  • Make sure you use the Core Collection Search
  • Go to Researcher search option
  • Search either under Name, Author Identifier (ResearcherID or ORCID) or Organisation
  • The Author's profile page includes all published articles indexed by Web of Science, author metrics, citation report, author beam plot, peer review metrics and an analyse results page.
  • If you click on the citation report and then Analyse Results, you will find the following very useful information on the author's metrics:
    - Co-authors
    - Document types
    - Categories
    - Affiliations
    - Publishers
    - Funding agencies
    - Countries
    - Languages, etc.

 

Dimensions (free version)

Read more on how to use Dimensions here

 

scite (free version)

Scite shows how a citation was used by displaying the surrounding textual context from the citing paper and a classification from their deep learning model that indicates whether the statement provides supporting or contrasting evidence for a referenced work, or simply mentions it. Scite has been developed by analysing over 25 million full-text scientific articles and currently has a database of more than 880 million classified citation statements. 

Install their browser extension to be able to read scite's citations in any webpage:

Download the extension here

Read more about their features here

Scite also recently launched and AI assistant: https://scite.ai/assistant

 

Google Scholar metrics

Available Metrics

The h-index of a publication is the largest number h such that at least h articles in that publication were cited at least h times each. For example, a publication with five articles cited by, respectively, 17, 9, 6, 3, and 2, has the h-index of 3.

The h-core of a publication is a set of top cited h articles from the publication. These are the articles that the h-index is based on. For example, the publication above has the h-core with three articles, those cited by 17, 9, and 6.

The h-median of a publication is the median of the citation counts in its h-core. For example, the h-median of the publication above is 9. The h-median is a measure of the distribution of citations to the articles in the h-core.

Finally, the h5-indexh5-core, and h5-median of a publication are, respectively, the h-index, h-core, and h-median of only those of its articles that were published in the last five complete calendar years.

We display the h5-index and the h5-median for each included publication. We also display an entire h5-core of its articles, along with their citation counts, so that you can see which articles contribute to the h5-index. And there's more! Click on the citation count for any article in the h5-core to see who cited it.

Source: https://scholar.google.com/intl/en/scholar/metrics.html#metrics

Google Scholar Profiles

Set up your Google Scholar profile in order to get your author metrics. Follow the steps set out in the link below:

https://scholar.google.com/intl/en/scholar/citations.html

Publish or Perish

https://harzing.com/resources/publish-or-perish

 

The most common method for evaluating journals is bibliometric citation analysis, where the frequency of citations related to the "average article" in the journal, reflects the popularity and influence of the journal.  Various citation indicators have been developed to reflect perceived quality.  These can be found in journal ranking lists.

Web of Science's Journal Citation Reports (JCR) ranks journals based on citation data. JCR gives an indication of influence and impact at a category level through its Impact Factor, showing citation relationships between journals. 

Scopus's Journal Metrics also ranks journals based on citation data. Like the Impact Factor, Scopus's CiteScore provides an indication of influence, ranking journals within their specific subject categories. 

Impact Factor (IF) is the most commonly used measurement to determine the reputation of a journal in relation to other journals in a specific field. The calculation of IF is based on the average number of times the articles of a journal is cited in a two/five year period.

CiteScore is used to measure a journals influence and impact and is Scopus' version of the Impact Factor. Like the IF, it is calculated over a period of time (3 years), and calculates the average number of citations received in a calendar year.

SCImago Journal Rank (SJR) ranks journals included in the Scopus database. It calculates not only the number of citations to articles in journals but also takes into account the ‘quality’ of the citing journal.

Source Normalised Impact per Paper (SNIP) also ranks journals included in the Scopus database.  The SNIP citation indicator is normalised according to the subject field.  Citation counts in the Life Sciences tend to be higher than in the Arts and Humanities.  SNIP “levels the playing field”.

Eigenfactor journal metrics computes two principal scores - the Eigenfactor Score and the Article Influence Score which take into account the quality of the citing journal, the number of articles in each issue and the different citation patterns in different disciplines. The Eigenfactor is included in the Web of Science database.

View a comparison between SNIP/SJR and Citescore below:
SNIP and SJR    Citescore and associated metrics                                            

ü Compensates for differences in field, type and age 

ü Large number   

ü Meaningful benchmark is “built-in” – 1 is average for a subject area

ü Simple, easy to validate

X People may not like small numbers

ü Communicates magnitude of activity

X Complicated; difficult to validate X Affected by differences in field, type and age
X No idea of magnitude: how many citations does it represent? X Meaningless without additional benchmarking

Source:

https://www.slideshare.net/GenevieveMusasa/scopus-as-a-bibliometrics-tool-citescore-metrics-more-metrics-the-importance-of-ranking/29

https://www-elsevier-com.ez.sun.ac.za/authors/tools-and-resources/measuring-a-journals-impact

Article-level metrics quantify the reach and impact of published research. It also seeks to incorporate data from new sources (such as social media mentions) along with traditional measures (citations) to present a rich picture of how an individual article is being discussed, shared, and used. Measuring and reporting societal impact is becoming increasingly important. You can use citation counts to see how many times a specific article has been cited by others.

Sources and tools

Scopus

Web of Science

Altmetrics

 

Altmetrics

  • This Altmetric Badge can be embedded on your researcher website.
  • Add “Altmetric it” bookmarklet to browser toolbar: https://www.altmetric.com/
  • Altmetrics Explorer for Librarians: Two librarians on our campus have access to Altmetric Explorer where you can find Altmetric information for your articles. Please contact Marié Roux if you would like to get your Altmetric scores.

 

Book Citation Index (Web of Science)

  • The Book Citation Index Science (BKCI-S) and Book Citation Index Social Sciences & Humanities (BKCI-SSH) are part of the of Web of Science Core Collection. To limit to book citation indexes, click Web of Science Index from the results screen and make your selection.
  • Selected books appear in the Master book list which may be an indicator of academic esteem.

Web of Science Cited Reference Search

  • To find out how many times a book has been cited by other books or articles, select the Cited References tab on the homepage

Scopus 

  • Scopus includes books and edited books. Click Document type from the results and select Book of Book Chapter. Available metrics include citations in Scopus and FWCI. See the Scopus content lists for more detail, including a list of books indexed.

Dimensions

  • Dimensions includes books, edited books and book chapters. Click Publication Type from the results screen and select Monograph, Edited Book, or Chapter.

Google Scholar

  • Google Scholar indexes books and book chapters from some publishers and is useful for tracking citations to books not indexed in Web of Science.

Source: https://guides.lib.monash.edu/research-metrics-publishing/book-metrics