• ARRA a CoARA

• The Agreement on Reforming Research Assessment emerged from negotiations between nearly four hundred organizations from more than forty countries around the world and was published in 2022

• The text called for a transformation of methods of evaluating science towards qualitative measures, rejecting the uncritical application of metrics such as H-Index or Impact Factor

• Among others, the Ministry of Education, Youth and Sports, Charles University and the Academy of Sciences of the Czech Republic signed up for it

• The Coalition for Advancing Research Assessment is an international initiative that brings together universities, research institutions or grant providers

• The initiative promotes a reform of science evaluation towards a stronger emphasis on peer review practices while cautiously using quantitative indicators

• The signatories of the initiative include the Ministry of Education, Youth and Sports, The Research, Development and Innovation Council, Charles University, Masaryk University or the Academy of Sciences of the Czech Republic

• COOPERATIO

• program of basic institutional support of science and research at Charles University

• it consists of 43 scientific areas, which are further divided into more than one hundred and fifty individual scientific fields

• each faculty or unit has signed up for a certain number of consistently developed scientific fields

• the aim of the program is the coordination and cooperation of all faculties where the scientific field is developed, as well as better interconnection of scientific areas with study programs

• the structure of scientific areas and fields was approved by the Research Board of the university (25 February 2021), which also approved the registration of faculties for scientific fields (27 May 2021)

• the program is implemented in five-year cycles, the first one taking place in 2022–2026

• Creative Activities Evaluation Board and Expert Panels

• Creative Activities Evaluation Board is the highest authority of the evaluation process

• its members are eminent scientists working abroad, who are not in a conflict of interest with regard to the evaluated research units

• they participate in visits to the evaluated units (On-site visits) and subsequently process Summary Evaluation Report on a given scientific area

• Expert Panels are the second evaluation authority that assesses scientific areas

• there are four Panels: medical sciences (MED), natural sciences (SCI), social sciences (SOC), humanities (HUM)

• the number of Panel members is derived from the number of scientific areas that fall within its scope

An international evaluation of creative activity takes place at Charles University every five years. The evaluation is governed by Rector's Directive No. 17/2024, Strategy for the Evaluation of Creative Activities at Charles University for the period from 2019 to 2023

• the evaluated period is the calendar years 2019–2023 (the previous cycle evaluated the period 2014–2018)

• the evaluated unit is the scientific areas as defined for Cooperatio, the program of basic institutional support for science and research at the Charles University

• the aim is to obtain verifiable information about the quality of creative activity at the University through international comparison and, based on this, recommendations for its further development.

• the evaluation reflects the entry into CoARA, whereby the university has committed to meeting these conditions in 2022

• Creative Activities Evaluation Board supervises the process in terms of compliance with principles, rules and organization

• members of Expert Panels assess the scientific areas based on evaluation tools

• Research Area Boards are responsible for preparing Self-Evaluation Reports according to the specified form and submitting them to the Scientific Councils of the participating units

• The Coordinator of the Research Area Board, representing one unit that cultivates a given scientific are, ensures cooperation of its members from all units involved in the scientific area

The basis for the evaluation is Self-Evaluation Reports of the scientific areas, which are processed by representatives of university units involved in the scientific area.

Self-Evaluation Report is divided into five chapters and written in English. The chapters contain nominations of scientific outputs, excellent personalities, projects, examples of social relevance or popularization of science:

• Basic Information about the Research Area

• Personnel Situation

• Research Outputs

• Research Funding, Scientific Cooperation

• Impact of Research

Evaluators also have at their disposal Summary information and basic data on the University:

• of the persons participating in creative activities

• of the outputs of creative activities

• of the basic financial indicators

• of the doctoral programmes of study and numbers of students and graduates of these programmes

Written Assessments of the Research Areas:

• members of the Expert Panels prepare at least two assessments written in English, indicating partial grades for individual parts of the Self-Evaluation Report.

• in addition, total grades and at least two other assessments are applied in the form of partial and total grades without a written assessment.

• the assessments and grades are discussed in the Expert Panel, after which an overall grade for the scientific area is proposed

• the overall grade is based on the arithmetic average of all aggregate grades, excluding the one worst proposed grade, with a possible adjustment of one intermediate grade based on the decision and justification of the Expert Panel.

Summary Evaluation Report on a Research Area:

• based on written assessments and discussions during on-site visits

• contains the final grade for the scentific area

• The final grade may be adjusted by the Creative Activities Evaluation Board by one intermediate grade compared to the overall grade proposed by the Expert Panel, based on justification

Summary Final Report on the Evaluation of Creative Activities at the University:

• summarizes the evaluation process and its main outputs

• Creative Activities Evaluation Board submits it to the Rector, who then proposes that it be discussed by International Advisory Board of the Charles University, the Research Board of the University, and the Internal Evaluation Board of Charles University

• after those discussions, it is made available to members of the academic community

• 6. 3. 2024–30. 11. 2024: preparation of the Self-Evaluation Report of the scientific area

• do 30. 11. 2024: discussion of the Self-Evaluation Report by the Research Boards of all units involved in the given scientific area

• do 30. 11. 2024: the Dean of the unit approves the final version of the Self-Evaluation Report, which is subsequently submitted by the coordinator of the the Coordinator of the Research Area Board to the Research Support Office of the RUK

• 1. 2. 2025–28. 2. 2025: Research Area Boards resolve additional questions from the Expert Panel in the form of an annex to the Self-Evaluation Report

• 1. 5. 2025–30. 6. 2025: Research Area Boards and representatives of units participate in the on-site visit convened by the Creative Activities Evaluation Board

• IP DKRVO

• Institutional support for the long-term conceptual development of research organizations (Institucionální podpora na dlouhodobý koncepční rozvoj výzkumných organizací)

• DKRVO represents a fundamental strategic document that determines the direction of the research organization

• the provision of institutional support from the budget chapter of the Ministry of Education, Youth and Sports is tied to a five-year period

• MEP

• International Evaluation Panel (Mezinárodní evaluační panel)

• processes the evaluation based on the Self-Evaluation Reports of units, on-site visits and final Evaluation Reports

• it has at least seven members, of which an absolute majority must be non-contracted experts from abroad (one member is a representative of the provider)

• M25+

• Methodology for Evaluation of Research Organisations in the Higher Education Institutions Segment 2025+ (Metodika hodnocení výzkumných organizací (Metodika 2025+))

• a crucial tool for obtaining information about the R&D&I, which is used for strategic decision-making and funding settings

• the evaluation is intended to motivate and reinforce good practice through feedback at all levels (national, provider, research organization) that enter the evaluation

• Modules

• five basic areas that together should enable the fulfillment of the strategic objectives of the evaluation

• the evaluation modules are common without exceptions, but their significance and scope vary depending on the mission and status of a specific scientific field

• M1 (Quality of Selected Results), M2 (Research Performance), M3 (Social Relevance), M4 (Viability), M5 (Strategy and Policies)

• RVVI

• R&D&I Council (Rada pro výzkum, vývoj a inovace)

• advisory body of the government that ensures the preparation of the Methodology and its submission to the government, evaluation within the scope of the established principles and resolution of disputed cases

• it also prepares and controls the implementation of the national R&D&I policy and submits to the government a proposal for total expenditure for this budget chapter

• VaVaI

• R&D&I or research, development and innovation (výzkum, vývoj a inovace), a collective term for scientific research activities, their financing and state support

• IS VaVaI is a public administration information system that collects, processes, uses and provides information on R&D&I supported by public budgets

• IS VaVaI includes the CEP (Central Project Registry) or the RIV (Register of Information on Results)

An evaluation of public research organizations in the higher education segment is carried out once every five years. The evaluation is guided by Methodology for Evaluation of Research Organisations in the Higher Education Institutions Segment 2025+ and is provided by the R&D&I Council.

• the evaluated period is the calendar years 2019–2023 (M1–M3) and 2020–2024 (M4–M5)

• evaluation takes place in M1–M5

• the aim is to motivate research to improve its quality and efficiency, cultivate the research environment and strengthen the responsibility of individual actors in the R&D&I system.

• the overall grade is included in the distribution of the university's DKRVO finances for the next five years.

• the assessment reflects the trends in science evaluation formulated by the European Commission's ARRA initiative, as well as the CoARA coalition

• Ministry of Education, Youth and Sports as the provider of institutional support, is responsible for compliance with the principles, rules and organization of evaluation

• Committee for the Evaluation of Research Organizations and Concluded Programmes expresses its opinion on the proposed composition of the MEP

• the members of International Evaluation Panel conduct evaluation based on Self-Evaluation Reports of units, on-site visits and final Evaluation Report

• R&D&I Council presents the results of M1 and M2

• Expert Advisory Committee of the provider supervises the consistent application of evaluation criteria, and the provider's Methodologist also guarantees compliance with the evaluation principles.

• the results of the evaluation are discussed by the provider, R&D&I Council and the Czech Rectors Conference

The evaluation is carried out in five modules.: M1 + M2 (50 %), M3 (30 %), M4 (10 %), M5 (10 %).

• Module 1: Quality of Selected Results

• Module 2: Research Performance

• Module 3: Social relevance

• Module 4: Viability

• Module 5: Strategy and Policies

M1

• the results are nominated annually by the faculty based on a set quota

• the basic limit is filled primarily with bibliometric results (Jimp, Jsc), the non-biblio limit is filled exclusively with non-bibliometric results (B, C, Jost) in order to strengthen the evaluation of scientific results that can only be decided by the peer review method.

• nominated results must be in the RIV, they may not have been selected in previous years, it can only be selected from outputs reported during the last five years

• the research organization decides whether the result is nominated according to the criteria of contribution to knowledge or social relevance

• the nominated outputs are approved by the provider, the results are assessed by an Expert Panel through external evaluators, who assign a ranking on a scale of 1–5.

• the basis is a five-year overview of nominated and evaluated outputs

M2

• the basis for the statistics is a bibliometric analysis based on data from the Web of Science or Scopus databases and an overview of the results submitted by the research organization to the RIV.

• the analysis includes Jimp, Jsc and D type results, the basic statistical metric is AIS which is used for qualitative scaling of journals into so-called percentiles

• the overview includes the production of all types of basic and applied research results over the five-year period under review.

M3

• social relevance is understood as the measure of the positive impact of research activities on society, economy, public policy, culture and cultural heritage, justice, public health, environment and cultivation of the Czech Republic and the region in the broader sense of the word

• the evaluation is based on real impacts, including knowledge transfer, cross-sectoral cooperation, and popularization of research

• the basis is the Self-Evaluation Report, which the faculty prepares in seven chapters:

• Introductory Information

• Recognition by the Research Community

• Research Projects

• Research Results with Existing or Prospective Impact on Society

• Transfer of Results into Practice

• Popularization of R&D&I

• Implementation of Recommendations

Modul 4

• assesses the quality of the research organization's process settings, especially in the areas of personnel policy, human resources development and equal opportunities

• the evaluation is based on the management of science, including the administration and organization of infrastructure and the ability to ensure the conditions for obtaining grants

• the broader principle is to incorporate information from national evaluation into the research organization's internal evaluation processes

• the basis is the Self-Evaluation Report, which is prepared by the research organization in thirteen chapters:

• Organisation and Management of R&D&I

• System of Support for a Quality R&D&I Environment and Incentive Measures for Quality Science

• Quality Control System for R&D&I Environment

• Sustainability and Resilience of R&D&I

• Structure of Human Resources

• Academic and Research Careers

• Gender Equality Measures

• Mobility of Academic and Research Staff

• Research Infrastructure

• Budget and Structure of Financial Resources

• Rules for the Use of Institutional Support for the LCDRO

• Important Collaborations in R&D&I

• Doctoral Studies

• Implementation of Recommendations in Module 4

Modul 5

• formulation of a research organization's strategy for prospective development

• the evaluation assesses the organization's mission, the concept for fulfilling this mission and its actual fulfillment, including the vision for the next period

• the basis is the Self-Evaluation Report, which is prepared by the research organization in four chapters:

• Mission and Vision of the Evaluated Institution in R&D&I

• Research and Development Objectives

• Institutional Instruments and Measures for the Implementation of the Research and Development Strategy

• Implementation of Recommendations in Module 5

Self-Evaluation report

• output at the faculty level, which processes the basis for the M3 (centrally registered data on personnel capacities is provided by the research organization)

• MEP conducts an on-site visit to the research organization, or individual components, as well as performs a point and verbal evaluation of each indicator Self-Evaluation report

Evaluation report

• provider-level output that includes materials from M3–M5

• MEP conducts on-site visit to research organization for M4–M5

• results in M1–M2 are requested by the provider from the R&D&I Council, then forwarded to MEP

• the overall grade (A–D) is equal to the simple average of the grades awarded in the individual indicators

• by 30. 11. 2024: submission of proposals for the evaluation of the research organization by the provider and the composition of the MEP

• by 31. 01. 2025: establishment of the MEP and OPO

• by 31. 03. 2025: submission of the evaluation documents to the provider

• May–August 2025: evaluation of the MEP and on-site visits

• September–October 2025: finalization of the Evaluation Report

• November 2025: Rector's statement on the Evaluation Report

• December 2025: final review of the evaluation by the OPO

• January–February 2026: joint meeting of the provider, R&D&I Council and Czech Rectors Conference

• February–March 2026: preparation of the protocol and publication of the evaluation results