Beyond the Numbers: An unconventional measure of success
In nowadays science, is the success of research all about the number of published papers and citations? I was pondering about one of my personal projects recently: the "OpenHVSR project", and about my choice of publishing it open source. The purpose of publishing open-source was to benefit the scientific community and promote collaborative research. Looking in retrospective, one may wonder:
Was It a good choice after all? Did my effort reach the impact I envisioned?
October 3, 2018: The date of this post and two years after the publication of OpenHVSR-Inversion (Bignardi et al, 2016):
- 12 Citations on Scopus (6 self-citations)
- 964 reads on ResearchGate
Since this research is within a niche of near-surface Applied Geophysics, I did not expect a large number of citations, and these figures were not bad after all, but... with a citations/reads ratio of about 0.6% these numbers did not look really encouraging. Then I realized that more and more people were contacting me every year, seeking my help and expert advice in using my software; some of these initial contacts even evolved into nice collaborations. One day, I was having a casual conversation on this subject with a senior colleague. He was impressed by the number of emails I received and their very enthusiastic tone. He told me that even for his most popular article he was contacted by no more than perhaps fifteen people. That made me think that I should have looked at the impact of this research from a different angle. Therefore, gathering the small information I could extract from this correspondence, mostly the names of the institutions from which I was being contacted, I decided to create a map; just to understand whether publishing my programs as open-source was actually fulfilling its purpose.
- 18 Countries on 5 continents
- more than 30 emails asking for information regarding one of my papers (see MAP)
These numbers might not be relevant to rating institutions, but they surely made me very happy!
MAP: Reasearchers who contacted me seeking my expert advice with OpenHVSR
October 15, 2021: Update
A few more years have passed and I felt the need to update this page. It is about five years since I published the first article about OpenHVSR, and I have also published a second article on the subject introducing another open-source software (OpenHVSR Processing Toolkit).
The number of citations may not be a strong indicator of success per se. What is important is the quality of those citations. When it comes to my research, I prefer to evaluate its impact considering those articles that actually made use of it rather than simply referring to it. As such, I went through all the papers citing OpenHVSR (as of October 15, 2021) and placed on a map all the locations (except in the case of Mars, Mahvelati S. and Coe J. T., 2021), investigated using my software. The following are the combined stats for the two articles about OpenHVST:
- 57 Citations on Scopus
- 80 Citations on Google Scholar
- ~2900 reads on ResearchGate
- 17 Research groups acknowledged using OpenHVSR, and investigated sites across 14 countries
- 4 Research groups applied their own customizations to OpenHVSR.
I am honored to know that my software has been used by at least 17 different research groups to study sites all around the world (even on Mars) and to carry out their own research. I was excited to find out that four of these groups actually applied customizations to my original code, which wouldn't have been possible had I not published it as open-source.
MAP: Sites investigated using OpenHVSR, (Not including Mars: Mahvelati S. and Coe J. T., 2021 )
Mahvelati and Coe (2021). Horizontal-to-Vertical Spectral Ratio (HVSR) Analysis of the Martian Passive Seismic Data from the InSight Mission. Earth and Space 2021: Space Exploration, Utilization, Engineering, and Construction in Extreme Environments - Selected Papers from the 17th Biennial International Conference on Engineering, Science, Construction, and Operations in Challenging Environments. pp. 108-115