Bibliometric data offers a quantitative method of analysing authors' or journals' output, but there are limitations to using bibliometrics:
- Comparisons between subject areas must be avoided. Some subject areas have a higher rate of publication and citation. For example molecular biology articles are produced rapidly and cited frequently compared to computer science or mathematics articles. This means that an average molecular biologist would probably have a larger h-index than a leading computer scientist.
- It is important not to make comparisons between authors of different ages or length of professional activity. Authors who have published for many years have had more time to accumulate citations and reputation. You can to some extent limit bibliometric results to a specific date range, for a fairer comparison.
- Papers often have multiple authors - but what proportion of the work can be attributed to each author? Citation metrics assume that each named author is equally accountable, when this might not always be the case.
- Citation counts could be misleading, for example if an author includes a large number of self-citations, or if a peer group agrees to cite each other to boost their citation rates. The peer review process for journals should spot and prevent this.
- Negative citations are counted as valid.
- Papers may be submitted under various forms of name although it is in fact the same author. There is also a lack of standard affiliation details. Databases have ways round this including grouping name variants and assigning each researcher an individual numerical ID.
- Some publication types tend to receive more citations than others. Review articles and methods papers, for example, are likely to be more highly cited than a paper based on a laboratory study.
- Citation metrics will differ depending on the data source, as different databases include different journals and years of coverage.
In addition, bibliometrics is a measure of the impact of research on further research, not necessarily of the quality of that research. Bibliometrics should therefore always be used with caution and not be considered a replacement for peer review, but best used to complement or verify qualitative evaluation.
Source: https://www.ucl.ac.uk/library/research-support-open-science/bibliometrics/bibliometrics-basics