Articles

Open science and evaluation


Abstract


The aim of this contribution is to investigate how open science can influence/support research evaluation and whether and how open science practice can be evaluated in its effort to avoid what is predicted by Goodhart's law (When a measure becomes a target, it ceases to be a good measure). The enterprise is not simple, as it focus on giving up points of reference that have been part of  common practice of hard sciences for years while we are now trying to implement them in humanities and social sciences. We must accept that the internet has changed the way science is produced, disseminated, validated and evaluated and has multiplied its channels of communication. For this reason the traditional bibliometric indicators, which refer to articles published in peer reviewed journals, preferably in English, as the only viable publication channel, become inapplicable. In an open environment, the role of peer review, in particular the idea of blind (single or double) peer review must also radically change.


Keywords


Open science; Bibliometrics; Research evaluation; Unintended effects; Open peer review

Full Text:

PDF


DOI: http://dx.doi.org/10.2423/i22394303v10Sp65

References


Baccini, A., De Nicolao., G., & Petrovich, E. (2019) Citation gaming induced by bibliometric evaluation: A country-level comparative analysis, PloS One. Retrieved from https://doi.org/10.1371/journal.pone.0221212

Banfi, A., Franzini, E., & Galimberti, P. (2014). Non sparate sull'umanista. La sfida della valutazione. Milano: Guerini e associati.

Biagioli, M., & Lippman, A. (Eds.) (2020). Gaming metrics: Misconduct and Manipulation in Academic Research. Cambridge: The MIT Press

Bianco, S., Caso, R., Destro Bisol, G., Di Donato, F., Galimberti, P., & Pievatolo, M.C. (2018). Scienza aperta. Come guardare (con convinzione) all’ignoranza degli esperti. In ADI-Associazione Dottorandi e Dottori di ricerca in Italia (Ed.), Ricerca in Vetrina 2018. Ricerca è democrazia. Il ruolo dell’attività scientifica nella costruzione di un futuro equo e sostenibile (pp 76-86). Milano: FrancoAngeli. Retrieved from http://ojs.francoangeli.it/_omp/index.php/oa/catalog/download/417/225/2005-1

Caso, R. (2016). La Scienza Aperta contro la mercificazione della ricerca accademica? (Open Science vs Commodification of Academic Research?). Trento Law and Technology Research Group, Research Paper Series, 28. Retrieved from

https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2768755_code1300904.pdf?abstractid=2768755&mirid=1

Caso, R. (2020). La rivoluzione incompiuta. La scienza aperta tra diritto dautore e proprietà intellettuale. Milano: Ledizioni

European Commission - Working Group on Rewards under Open Science (2017). Evaluation of Research Careers fully acknowledging Open Science Practices. Retrived from https://ec.europa.eu/research/openscience/pdf/os_rewards_wgreport_final.pdf

European University Association (2019). Research assessment in the transition to Open Science. Retrieved from https://eua.eu/downloads/publications/research%20assessment%20in%20the%20transition%20to%20open%20science.pdf

Galimberti, P. (2012). La valutazione della ricerca a livello istituzionale: problemi, sfide e possibili soluzioni. Il caso dell'Italia. Rassegna italiana di valutazione, XV(52), 66-80.

Galimberti, P. (2012). Qualità e quantità: stato dell’arte della valutazione della ricerca nelle scienze umane in Italia (Quality and quantity: HSS research evaluation in Italy. A state of the art). JLIS.it , 3(1), 1-25. DOI: http://dx.doi.org/10.4403/jlis.it-5617

Galimberti P. (2017). L’impatto dell’open access sul processo di valutazione. Quaderni di Articolo 33 – FLC. La valutazione del Sistema Universitario e della Ricerca, 3, 105-116. Retrieved from Zenodo http://doi.org/10.5281/zenodo.1210346

Galimberti, P. (2020). Cambiamento culturale e politiche di Open Science in Italia: il caso dell’Università di Milano. ROARS (Return On Academic Research). Retrieved from https://www.roars.it/online/cambiamento-culturale-e-politiche-di-open-science-in-italia-il-caso-delluniversita-di-milano/

Margoni, T., Caso, R., Ducato, R., Guarda, P., & Moscon, V. (2016). Open Access, Open Science, Open Society. Trento Law and Technology Research Group, Research Papers, 27. Retrieved from https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2751741_code1300904.pdf?abstractid=2751741&mirid=1

Mueller, J. Z. (2018). Against metrics. Princeton Unversity Press.

O’Neil, C. (2017). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Penguin.

Pascuzzi, G. (2017). Il fascino discreto degli indicatori: quale impatto sull’Università? Trento Law and Technology Research Group, Research Papers, 34. Retrieved from https://iris.unitn.it/retrieve/handle/11572/190765/164119/Law_Tech_Res_34_Pascuzzi.pdf

Pievatolo, M. C. (2017). La bilancia e la spada. Scienza di Stato e valutazione della ricerca. Retrieved from Zenodo http://doi.org/10.5281/zenodo.570042

Pievatolo, M. C. (2018). Integrità della ricerca: i numeri, gli uomini e la scienza. Bollettino telematico di filosofia politica. Retrieved from https://btfp.sp.unipi.it/it/2018/05/uominienumeri/

Pievatolo, M. C. (2019). Scienza aperta: il senso duplice di una rivoluzione. Retrieved from https://archiviomarini.sp.unipi.it/812/1/mcp_lecce_2019tex.pdf

Seeber, M., Cattaneo, M., Meoli, M., & Malighetti, P. (2018). Self citations as strategic response to the use of metrics for career decisions. Research Policy, 48(2), 479-491.

SPARC (2019). Landscape analysis. The changing academic publishing history- Implications for academic institutions. Retrieved from https://sparcopen.org/our-work/landscape-analysis/


Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.


Copyright (c) 2020 Paola Galimberti

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.