"The procedure of formation and allocation of state procurement for the supply of goods for state needs" - that was the title of document which regulated the distribution of state-funded places between universities in Ukraine till 2012. The names of documents were slightly changed in 2012-2014, but the principle of soviet-style state procurement system remained the same. There is no any planned economy or centralized distribution of graduates among enterprises for 25 years, but the state continues to operate under the old practices, which does not cover modern needs.
This policy paper describes and localizes performance-based funding of higher education institutions model that could potentially replace the mechanism of state procurement. The text contains basic statistical analyses of funding of higher education in Ukraine, a description of the regulatory framework of current state procurement system, analysis of its implementation with a focus on weaknesses, a localization of performance-based funding model and description of conditions that are necessary for its implementation.
- Statistics on higher education funding
- Ukrainian legislation on state procurement (public funding) of higher education
- Implementation of the legislative regulations
- Weaknesses of the state procurement system
- Localization of performance-based funding model for Ukraine
- Appendix: Brief description of the components of possible formula
Statistics on Higher Education Funding
This chapter lists the key statistical indicators that characterize the situation in higher education funding and some related phenomena. The data help to understand what were the funding priorities of the government and of households, as well as to predict future tendencies.
Amounts of State Procurement Funding
In 2014, most European countries increased the amounts of the government funding of education, after it had been cut as a consequence of the 2009-10 crisis. In contrast, Ukraine experienced the greatest decrease in the real level of spendings on education of all European countries (see Figure 1).
Figure 1. Dynamics of the amounts of government spending on education in 2013-14, in 2013 prices, %
Source of Data: National sheets on education budgets in Europe 2014 / Eurydice – p.4. Data about the EU countries for 2014 are presented in 2013 prices using the harmonized consumer price index. In order to translate the data about Ukraine, consumer price index was used. It must be noted that Ukrainian consumer price index is not harmonized with the EU, and the data are calculated as of the end of 2014, and not as of September 2014. In addition, the spendings in 2014 do not include the data on the temporarily occupied territories of Crimea, Sevastopol and the Anti-Terrorist Operation zone. Calculations by Oleksandra Slobodian.
In the last eight years, spendings on state procurement have varied between 3.2 and 4.1 percent of the consolidated state budget. Fluctuations towards increase or decrease have not always correlated with the dynamics of changes in the budget fraction spent on education in general. In 2010 and 2013, despite the general increase of the fraction of budget spent on education, the fraction spent on state procurement decreased; and in 2012, the budget fraction spent on education shrunk considerably, and the state procurement, in contrast, did not experience such cuts (see Figure 2).
Figure 2. State spendings on education as a fraction of consolidated budget, %
Source of data: Yearly decrees by the Cabinet of Ministers of Ukraine about the amounts spent on state procurement, and The Key Indicators of HEI Activities 2005-16 by the State Statistics Service. The data for 2014 do not take into account the temporarily occupied territories of Crimea, Sevastopol and the Anti-Terrorist Operation zone.
Nominally, spending on state procurement has always been increasing at all levels. However, in terms of real amounts (in 2007 prices), the funding of the state procurement of higher and aftergraduate education fell in 2015 by, respectively, 11 and 20 percent, compared to 2007. However, there was even a certain increase for professional-technical education (see Figure 3.1; 3.2).
Figure 3.1. Spending on state procurement, million UAH
Source of data: Yearly decrees by the Cabinet of Ministers of Ukraine about the amounts spent on state procurement.
Figure 3.2. Changes in state procurement in education in 2007 prices*
Source of data: Yearly decrees by the Cabinet of Ministers of Ukraine about the amounts spent on state procurement.
* Based on consumer price index.
The overwhelming majority of spendings (over 90 percent) on state procurement cover salaries and student stipends. The increase in costs of energy resources in the recent two years caused their fraction to rise by 1.3 percentage point: in 2013, they accounted for 4.7 percent of the spendings on state procurement, and in 2015, for 6 percent. Capital spendings are nearly nonexistent (see Table 1). In the situation when involving extra funds and increasing funding seems improbable, the rise in spendings on equipment upgrades and capital renovation is possible only via internal redistribution, i.e., by cutting spendings on labor compensations (employee dismissals) or taxes on salaries (e.g., from 2016 on, the single social contribution is reduced to 22 percent), or cutting stipend funds. However, current legislation demands to spend even more on stipends.
Table 1. Distribution of spendings on state procurement in the Ministry of Education system by type of spending, million UAH
|Labor compensation||4 419||44,7%||4 876||44,0%|
|Accruals for salaries||1 601||16,2%||1 768||16,0%|
|Social welfare (mainly stipends)||3 031||30,6%||3 515||31,7%|
|Utilities and energy resources||565||5,7%||665||6,0%|
|Total||9 893||11 071|
Data from the Ministry of Education, received upon official request.
Birth rate in Ukraine has been falling since the early 1990s, reaching its lowest point in 2001. That year, there were 43 percent less newborns than in 1990. In the 2000s, birth rate was gradually increasing until the recent years, when, after the economic crises of 2009 and 2014, birth rate somewhat decreased. Accordingly, in the nearest three admission campaigns, there will be less applicants for admission to universities. Soon afterwards, a certain increase will start, and it will intensify the competition between entrants for free higher education (see Table 2). Since Ukrainian universities are quite sensitive to the paying capacity of Ukrainians, the fraction of entrants who will choose to pay tuition will probably be higher.
Thus, if government funding will not increase, funding from household budgets will play an increasingly important role in university budgets. In addition, in the situation of greater competition for state funding, it will become especially important to improve the transparency and efficiency of funding distribution.
Table 2. Birth rate in Ukraine in 1990-2014
|Year||Number of newborns, thousand persons||Number of newborns, thousand persons in Crimea and Sevastopol||Year of school admission||Year of university admission*|
Source of data: Number of newborns according to the State Statistics Service of Ukraine. Data for 2014 do not include the temporarily occupied Ukrainian territories of Crimea, Sevastopol and the Anti-Terrorist Operation zone.
* Until 2001, the great majority of children in Ukraine started primary school at the age of 7 and studied for 10 years, skipping the 4th grade. Only a small minority of children started school at the age of 6 and studied for 11 years. I.e., those who started at six and at seven graduated from high school at the same time. In 2001, there was a shift to 12-year schooling that was supposed to start at 6; however, some parents exercised their right to bring their children to school at 7, because of their special development characteristics. However, Ukrainian schoolchildren never finished their 12 years of schooling: in 2010, the 11-year school was brought back. This time it was for all students, regardless of the year when they started school. The children who started their studies in the 12-year school, mechanically “jumped” into the 11-year one and studied in high school for two rather than three years.
Competition for State-Procured Places
Although the nominal amounts of funding for state procurement are increasing, the number of state-procured places constantly shrinks. The cuts in the number of full-time study places are the most significant for the Specialist degree. In the end, current law on higher education provides a plan to have the last round of admission for this degree in 2017. Generally, in the last nine years, the number of state-procured full-time study places on all educational levels was cut by 26 percent. The number of places at distance education programs decreased by 57 percent, mainly because of the radical drop in 2015, caused not only by the government’s decision, but also by the occupation of Ukrainian territories.
However, even in the situation of cuts, the competition between applicants did not increase. The reason is that the number of entrants was decreasing faster than the number of state-funded places at universities. That is why, even as the number of state-procured places was falling for the recent four years, the competition for those places at various education levels did not intensify (see Table 3).
Table 3. Competition for state-procured places on different levels, full-time and distance education combined
|Specialist based on full general secondary school education||4,5||3,5||2,9|
|Specialist based on Bachelor degree||3,5||3,0||3,8|
|Master and Specialist based on Bachelor degree combined||3,4||3,0||3,3|
Source of data: Unified State Electronic Database on Education, data obtained upon official request. Competition calculated by the author.
Field Priorities of the State Procurement System and of Entrants
In the last nine years, the total number of state-procured places has changed significantly, but their distribution between fields stayed practically stable. In full-time education, there was a minor decrease in the fraction of social sciences in favor of engineering and healthcare. However, the leaders stayed the same: engineering (34 percent), social sciences (19 percent) and humanities (12 percent).
In distance education, the changes are more significant. For example, in 2015, the weight of social sciences somewhat increased. The leaders in distance education are social sciences (35 percent), engineering (20 percent), humanities (12 percent), education (12 percent). Maintaining the long-term status quo in the situation of dynamic changes in the labor market leads to deepening discrepancies between universities and the economy. It is just another piece of evidence in support of the thesis that we need to step away from the state procurement model and look for other ways to distribute state funding.
The demand for various specializations among entrants is also characterized by a rather stable structure. The majority of applications for full-time education submitted in 2015 were for social sciences (41 percent), humanities (15 percent), and engineering (15 percent). At the same time, the popularity of engineering decreased a bit in the recent year, from 19 to 15 percent, due to unsatisfying results of the External Independent Testing in mathematics, which was failed by 22 percent of those who attempted to pass it.
The comparison between the fields preferred by the entrants and by the state procurement system points at substantial differences. For the state, the priority is engineering, and the entrants, on the other hand, primarily choose social sciences (see Table 4).
Table 4. State procurement and number of applications
|State procurement of training of Bachelors, according to the fields of International Standard Classification of Education (ISCED)||Number of applications for bachelor degrees, according to the fields of ISCED, full-time program|
|full-time program||distance program|
|Social sciences, business and law||23%||18%||36%||35%||41%||40%||39%||41%|
|Humanities and arts||13%||11%||12%||12%||14%||15%||15%||15%|
|Engineering, manufacturing and construction||33%||34%||26%||20%||19%||18%||19%||15%|
|Health and welfare||6%||8%||3%||4%||3%||3%||3%||3%|
|Source of data: Yearly decrees about state procurement by the Cabinet of Ministers of Ukraine. The fields of Ukrainian classification are combined according to ISCED 2011. Information technologies are singled out of the Sciences category. The Health category includes the number of places at the Specialist programs in Ukrainian fields of Medicine and Pharmaceutics, since there are no Bachelor studies in these fields, and training for them starts directly with the Specialist program based on full secondary education, and lasts for five years. A more detailed distribution according to Ukrainian classification of fields is available at the CEDOS website.||
Source of data: Unified State Electronic Database on Education, data obtained upon official request. Calculations by the author. The fields of Ukrainian classification are combined according to ISCED 2011. Information technologies are singled out of the Sciences category. Data for 2014-15 do not include the temporarily occupied Ukrainian territories of Crimea, Sevastopol and the Anti-Terrorist Operation zone.
State-procured places vs Tuition places
The lack of state funding and its inefficient distribution motivated universities to look for other sources of income in addition to the state budget. Tuition fees paid by students who are not covered by the state procurement system have become the most popular of these sources. The fraction of extra-budget income, the overwhelming majority of which comes from tuition fees paid by students, is the largest for the economical and medical HEIs, as well as for the institutions which are popular among foreign students. For these universities, the fraction of their income coming from tuition can reach up to 40-50 percent. In contrast, art and technical HEIs usually receive no more than 35 percent of their income from this source (CEDOS, 2015).
At the same time, the number of students paying tuition fees is falling more rapidly than the number of students in the state procurement system. The number of students accepted to the state-procured places between 2012 and 2015 fell by 38 percent, from 107,282 to 66,314 people. Meanwhile, the number of students accepted under condition that they pay tuition themselves fell by 59 percent, from 98,555 to 40,548 people. The total number of students accepted to full-time programs fell by 48 percent (see Table 5). This tendency had two main causes, the demographic crisis and the decline in the academic achievement of school leavers. In addition, there was the factor of a qualitatively different method of calculating the threshold number of points in the test, which was introduced in 2015, and which filtered out a higher percentage of potential applicants than in the previous years. Moreover, the steep decline in 2014-15 was caused by the occupation of Ukrainian territories .
If direct link between the number of faculty positions and the number of students will not be reduced, it will further increase the number of part-time teachers and lead to the prevalence of yearly (nine-monthly) contracts for teachers.
Among the students accepted during the last two admission campaigns, there were more of those who took the state-procured places (62 percent), while the number of tuition-paying students fell to 38 percent. At the same time, the number of students who studied for free could have been even higher, since, as of August 18, 2015 (when the admission campaign de facto ended), the number of vacant state-procured places was almost 5,000. Low academic achievement in mathematics and sciences among high school graduates, as well as the lack of proper equipment in most technical universities, resulted in low demand for these programs on the part of the entrants, leaving 3,200 state-procured vacancies in engineering, 530 vacancies in sciences, and 281 vacancies in computing unfilled.
Table 5. The number of accepted Bachelor students by form of payment, full-time programs
Data source: Unified State Electronic Database on Education, data obtained upon official request. Calculations by the author. The data for 2014-15 do not include the temporarily occupied Ukrainian territories of Crimea, Sevastopol and the Anti-Terrorist Operation zone.
It has become a tradition that more tuition-paying students are accepted to programs in the fields of social sciences, services, and health. In contrast, the state pays for the education of students mostly in the fields of engineering, IT, sciences and education. The number of students who pay and of those who do not pay tuition fees are nearly equal in humanities (see Table 6). These data demonstrate that, if not only the tuition fees paid by students, but also the government funds are distributed based on the entrants’ choice (voucher model), a radical reduction in funding for maths- or science-centered programs may start.
Table 6. The number of Bachelor students accepted in various fields, according to the type of payment, 2015
|Total||State procurement||Tuition fee||State procurement, %||Tuition fee, %||State procurement, %||Tuition fee, %|
|Education||5 418||3 533||1 885||65%||35%||52%||48%|
|Social sciences, business and law||32 899||13 937||18 962||42%||58%||33%||67%|
|Humanities and arts||18 445||10 330||8 115||56%||44%||53%||47%|
|Computing||8 322||7 051||1 271||85%||15%||77%||23%|
|Sciences||7 584||6 272||1 312||83%||17%||76%||24%|
|Engineering, manufacturing and construction||20 792||18 724||2 068||90%||10%||74%||26%|
|Agriculture||3 355||2 694||661||80%||20%||75%||25%|
|Health and welfare||5 733||2 318||3 415||40%||60%||42%||58%|
|Services||3 508||742||2 766||21%||79%||27%||73%|
Data source: Unified State Electronic Database on Education, data obtained upon official request. Calculations by the author. The fields of Ukrainian classification are combined according to ISCED 2011. Information technologies are singled out of the Sciences category. The data for 2014-15 do not include the temporarily occupied Ukrainian territories of Crimea, Sevastopol and the Anti-Terrorist Operation zone.
Ukrainian Legislation on the State Procurement (Public funding) in Higher Education
The procedure of forming and distributing state procurement is regulated by the Ukrainian Law “On forming and distributing state procurement of training professionals, scientific, scientific-educational, and working cadres, of advanced skills training, and of re-training cadres,” as well as by a number of regulatory acts issued by various government bodies. The Law was passed in 2012; however, it de facto reproduced the state procurement system that had existed for many years before that. In this chapter, the requirements of the legislature will be briefly reviewed; however, the reader must keep in mind that regulations are not always reflected in practice.
The responsibility for determining the general number of state-procured places, according to the Decree No. 363 by the Cabinet of Ministers, of May 20, 2013, “On the approval of the Procedure of distribution of state procurement of training professionals, scientific, scientific-educational, and working cadres, of advanced skills training, and of re-training cadres,” lies with the Ministry of Economy of Ukraine. It is the Ministry of Economy who determine the number of state-procured places based on forecast and suggestions from the Ministry of Education and Science, as well as all the other institutions that have the right to order state procurement. These institutions include other ministries and central bodies of executive power; the National Academy of Science; the Council of Ministers of the Autonomous Republic of Crimea; regional, Kyiv and Sevastopol city state administrations; or other government bodies, defined by the Cabinet of Ministers, which is the top manager of the state budget. In practice, about thirty state institutions order state procurement each year. The major state agents of procurement today are the Ministry of Education, the Ministry of Culture, the Ministry of Health, the Ministry of Defence and the Ministry of Internal Affairs.
According to the law, state procurement must be based on the mid-term forecast of the demand for professionals and workers in the labor market. The forecast is produced by the procedure provided by the Decree No. 35 by the Ministry of Economic Development and Trade of Ukraine, of March 26, 2013, “On approving the Methodology for making the mid-term forecast of the demand for professionals and workers in the labor market.” The forecast calculation is based on the need for replacement of the currently active workforce and the need for new workforce required for economic development, as well as on the trends of the recent years, and takes into account the statistical data and the suggestions from other central government bodies, institutions and organizations.
The amount of state procurement are planned in several stages:
- before September 15, the Ministry of Economy accepts the suggestions about the number of state-procured places;
- before November 1, the same Ministry informs the state agents of procurement about the mid-term forecast;
- before December 1, the state agents of procurement submit their proposals to the Ministry of Economy of Ukraine for the next three budgeting periods in terms of numbers of places and in terms of their cost. The cost of training is supposed to be calculated using the Methodology approved by the Decree No. 346 by the Cabinet of Ministers “On approving the Methodology of calculating the approximate average cost of training per one qualified worker, professional, graduate student, or doctoral student.” The Methodology involves the use of minimal spending indicators in nine categories, some of which are already outdated, such as the cost of utilities and energy resources. However, the Methodology has hardly been used in practice anyway.
Consolidated projected cost of state procurement is then included in the state budget project for the relevant year. After yearly Law “On the state budget” takes effect, the agents of procurement submit their final proposals to the Ministry of Economy about the state procurement for the planning year, which determine the cost of training, the number of state-procured places and the necessary spending from the state budget on the indicated purposes. Then the government issues a Decree that approves the total volume of state procurement for each educational level, field and specialization.
The distribution of state procurement is carried out on a competitive basis, except for the special cases indicated in legislation. The terms, the procedure of arranging the competition, and the members of the jury are determined by the Decree No. 363 by the Cabinet of Ministers, of May 20, 2013, “On the approval of the Procedure of distribution of state procurement of training professionals, scientific, scientific-educational, and working cadres, of advanced skills training, and of re-training cadres.”
According to the Decree, all applications for the competition must be submitted to the jury no later than 15 days after the competition is announced. The jury consists of the representatives of the state agents of procurement, the Ministry of Education and Science, the Ministry of Economy, the expert councils under the Accreditation Commission, the joint representative organization of employers, their unions, and student self-government bodies.
The Decree does not determine the exact number of jury members and allows to reorganize it within a year.
The results of the jury’s consideration are recorded in a protocol, and the competitors have the right to receive a copy of the protocol upon request. It is mandatory that every participant of the competition is informed about the results of the commission’s meeting within 10 days since the date of the meeting.
The Decree does not provide the exhaustive list of criteria for the selection of the HEIs that will become the executors of state procurement. The agents of state procurement have the right to define additional criteria of selection. At present, heads of HEIs are informed of the additional criteria via explanatory letters about the procedure of competitive selection of the executors of state procurement. Which means that the criteria keep changing every year, and it allows us not only to say that stable government policy is lacking, but also to question the transparency of selection and suggest that the process involves corruption.
New Model of State Procurement Distribution in 2016
The new Law “On higher education,” passed in July 2014, left the system of state procurement practically intact. Its only innovation concerned the procedure of distribution of state-procured places for Bachelor students, which, according to the Law, is supposed to change in the summer of 2016. The point of the innovation is essentially to introduce some elements of the so-called voucher approach.
Already at the stage of registering for the External Independent Evaluation test, school leavers must indicate the educational programs for which they wish to apply and the corresponding HEIs in order of preference . Ratings determine whether any particular applicant is eligible to receive state-funded education.
The entrants who earn their right to have their studies funded from the state budget are distributed, according to their places in the rating, among HEIs which offer studies in their chosen programs, according to the priorities indicated by the entrants and the admission rules of the higher education institutions they have chosen.
The admission candidates’ entering grade consists of several components. The admission rules define the proportion of every EIE certificate and of high school GPA in the grade — no less than 20 percent and no more than 10 percent, respectively. Creative contest, if applicable, can constitute no more than 50 percent of the grade. In addition, there is an incentive for the entrants to choose programs in science or engineering: if they take the relevant preparation courses at their chosen university and succeed in them, their course grade will count as up to 5 percent of their total grade while applying for these programs.
Thus, the rating which determines whether the entrant can be accepted to one of the state-procured places comprises several factors: the number of state-funded places for each specialization, defined by the Ministry of Economy; the HEIs and programs preferred by the entrant; and the admission rules of those HEIs, indicating the specific proportions of the components of the candidate’s grade; and the grade itself.
Therefore, the system proposed by the new Law essentially offers to change the procedure of distributing the state procurement of Bachelors. The model of distribution via decisions of jury will exist at the Master and PhD levels. In contrast, the funding for Bachelor programs will be distributed not by decisions of the jury organized by the agents of procurement, but by decisions of the entrants who will choose in favor of particular HEIs.
Implementation of the Legislative Regulations
The legislation specifically dedicated to determining and distributing state procurement, which was adopted in 2012, was intended to regulate the government funding of higher education. However, in practice, the law could not ensure the desired transparency and efficiency of budget spending. Openness of the process still completely depends on the will of the agents of procurement, as the competitions of 2013-15 demonstrated.
Determining the Total Amount of Spending
Although the law requires that the number of state procured-places is determined according to the forecast of the labor market’s needs, the actual numbers radically differ from the forecasts. The report by the Accounting Chamber of Ukraine demonstrates it:
“Thus, according to the conclusions by the Ministry of Economy and Development, provide in the letter No. 4803-17/38181-07 of November 1, 2013, the mid-term need for training professionals with higher education degrees and other workforce in 2015 was 151,241 people. However, according to the letter No. 4803-14/38025-03 of October 30, 2014, the same need in 2015 is 54,100 people. That is, within a year, according to the calculations by the Ministry of Economy, the need radically changed, more precisely, became three times smaller. At the same time, no explanation of the Ministry’s system of calculation was provided. Nevertheless, the Decree No. 462 (on the volume of state procurement) states that the number of state-procured places in the system of higher education for 2015 is 139,511, that is, almost three times more than the state needs (according to the most recent calculation by the Ministry of Economy and Development in 2014)” (Пилипенко, 2015, 21).
Such discrepancy arises because the process of determining the number of state-procured places is dominated by the needs of higher education institutions that submit their suggestions to state procurement agents, who, in turn, pass them on to the Ministry of Economy. Universities calculate their suggestions based purely on their own staffing needs, i.e., on the number of their employees and teachers whose labor must be compensated. This situation has developed because of the mandatory rate of students per one teaching position, established by the Decree No. 1134 by the Cabinet of Ministers on August 17, 2002; the rate is used to calculate the number of teaching positions for each university. Under this condition, the number of teachers at a particular university is strictly dependent on the number of students, which makes it impossible to expel students who do not fulfill their program’s academic requirements.
At the same time, the forecasts of the labor market’s needs, produced by the Ministry of Economy, have many defects that undermine the credibility of these calculations. The defects will be described in detail in the following chapters. Moreover, by now, the Ministry of Economy has already dissolved the department which had been responsible for calculating the mid-term forecast of the needs of the labor market, which supposedly should be used to determine the number of state-procured places.
Implementation of the mechanism for determining the guaranteed minimum amount of state funding allocated to higher education is worth mentioning separately. The current model is linked to demographic indicators: for every 10,000 citizens, there always must be no less than 180 students whose education is funded by the state. If you take the approximate population of Ukraine, according to the State Statistics Service’s data for November-December 2015 (42.6 million people), than, in 2016, there must be 770,000 students whose education is covered by the state budget. At the same time, in 2015/16, there were 732,000 such students (Кармазіна, 2015, 10). In addition to this mandatory minimum, the law states that the amount of funds spent on education from the central budget in any given year must not be smaller than the amount of the previous year, adjusted for inflation. However, in the recent two years, this requirement was not fulfilled, just as in other periods of financial crisis.
In this situation, the number of state-procured places in higher education, which defines how many professionals will be funded by the state in each of more than 130 specializations (according to the new official list of the fields of education), is, in practice, determined mostly by the staffing needs of HEIs, by inertia of the previous years, and by political lobbying by specific ministries or universities that interfere in the process of determining this number.
Determining the Cost of Training per Student
The methodology for calculating the cost of training per student, which was approved in 2013, was never fully implemented. According to the methodology, HEIs were supposed to determine the approximate cost of training per student using certain cost components and to provide these calculations to the agents of state procurement and the Ministry of Economy, so that the latter could use them as the basis for determining the total amount of state procurement. However, the overly specific components of the calculation (from office supplies to the kilowatts of energy consumed) made the use of the methodology impossible.
In this situation, the cost of state-funded training per student is still detached from real resource requirements for studies in various disciplines. The thing is that, due to continued lobbying by some universities and ministries, there is a considerable disproportion in the costs of state-funded training. For example, in 2014, educating one student at various programs in the HEIs under the Ministry of Culture mostly cost UAH 32,000-63,000 per year; at the same time, at medical HEIs, it cost UAH 22,000 per year. And, besides, the HEIs of the Ministry of Culture, using state budget funds, produce not only specialists in culture and arts, but also in economics, linguistics or management. The situation is the same in agrarian HEIs, which, using state’s funds, often train economists or lawyers. Actually, the annual cost of state-funded training of a BA in Law is considerably different for universities under different state bodies: for the Ministry of Justice, it is UAH 20,000; for the Fiscal Service, UAH 30,000; for the Ministry of Agriculture, UAH 31,000; for the Ministry of Culture, UAH 46,000; for the Border Service, UAH 103,000; for the Ministry of Defence, UAH 104,000; for the Security Service, UAH 115,000 . In turn, the average cost of state-funded training of one BA student in the system of the Ministry of Education is UAH 24,000, although it can vary depending on the university. For example, a BA student of Prykarpattya National University costs UAH 15,000, a BA student of Kyiv Polytechnic Institute costs UAH 34,000, while the average training cost for both BA and MA students at Sumy State University is UAH 22,000 a year .
Under conditions of powerful university lobbyism, the new system of determining the cost of training students must be simple and clear for most of the stakeholders. Only in this case it would be possible to achieve at least some approximate accordance between the monetary cost and the actual resources needed for the training.
Competitive Distribution of State Procurement among Universities
In 2013-15 there were many violations in the course of competition. First, there was a lack of transparency at the stage of announcing the competition. The official announcement of the beginning of the competition for the distribution of state-procured places was published only at the website of the Ministry of Education; other agents of state procurement ignored this requirement of the law. In addition, the fact that the agents of state procurement were allowed to independently establish the procedure of creating their juries and the rules of their functioning made it practically impossible for the public to control the process.
Second, the legislator has not provided the exhaustive list of criteria for selection among the participants of the competition. At the same time, the criteria listed in the Decree by the Cabinet of Ministers “On the approval of the Procedure of distribution of state procurement of training professionals, scientific, scientific-educational, and working cadres, of advanced skills training, and of re-training cadres” are too general. For example, the requirement of having a license for carrying out educational activities must become a condition of participation, and not a criterion of selection, since licenses cannot be better or worse. The same can be said of another criterion, providing the conclusion of an expert commission for accreditation. The conclusion does not ensure competitiveness between HEIs, since it was formulated to accredit a particular program, which does not entail distinguishing between programs of higher or lower quality, but rather simply ensures that all programs meet certain requirements. Another criterion is the fact of approval (or lack of approval) of the requested number of state-procured places by the regional Employment Center and the local government. However, surveys show that only 39 percent of young people register at employment centers, which means that the centers do not possess the relevant information about the needs of the labor market (Волосевич та інші, 2015, 39). Thus, two out of the three criteria which the law lists as mandatory are actually not criteria for selection, but rather conditions of participation in the competition. In this situation, the agents of state procurement distribute budget funds according to their own interests, just as before 2012.
Third, neither of the adopted legal regulations — the Law, the Decrees by the Cabinet of Ministers, or the Decrees by the Ministry of Education in 2013 — contained a requirement to publish the final results of the state procurement distribution with clearly specified HEIs and the corresponding fields and specializations taught there. It remains unclear why legislators did not oblige the agents of state procurement to publish copies of their jury’s protocols, since such an obligation would enable public control over the distribution of state procurement. Instead, the chain for publishing the results of the competition, from HEIs to the Unified State Electronic Database on Education to vstup.info , was faulty. HEIs often simply avoided providing information to the Database, and, accordingly, it did not appear in the vstup.info. After all, there is no legally specified time period, within which the executors of state procurement are obliged to provide information to the Unified Database; this allows some “special” HEIs to delay fulfilling the requirement of the Law. Thus, the Law of November 2012, which was supposedly aimed to make the process of distributing state procurement clearer and more transparent, does not have the intended effect.
In this situation, only the political will of the agent of state procurement to make the distribution more public enables public control. In 2014/15, some of the agents of state procurement tried to make the competition more transparent. For example, some of them did indeed publish the announcement of the competition and its conditions in advance. However, competition results and the relevant information about the number of state-procured places allocated to each HEI and each specialization were published only at the website of the Ministry of Education.
However, having ensured that the process is public, the Ministry of Education, as the largest agent of state procurement, confronted other problems. In the situation of the lack of mechanisms for monitoring and collecting data about the quality of higher education, any criteria, including the ones selected for funding distribution, do not guarantee its efficiency. In 2014/15, the Ministry of Education tried to take into account some criteria which could, at least nominally, be the evidence of education quality.
In 2014, Ministry of Education declared that they will take into account the non-governmental university rankings when they distribute state procurement. However, the use of such rankings entails a number of reservations.
None of the rankings cover all HEIs that participate in the competition. The rankings that are divided into categories by field, first of all, do not cover all fields in which state procurement is available, and second, are organized not according to the field classification provided by legal regulations. For example, the Humanities category in the ranking by Dengi magazine includes linguists, marketing specialists, copywriters and PR managers. That is why, if such rankings are used, the members of the jury must know which specializations were attributed to which field in any particular ranking.
The methodology of most rankings (Dengi, Correspondent, to some extent Compass) is based on employer surveys. The authors of the rankings do not publish the criteria for selecting particular employers. If such rankings are used, the members of the competition commission must be familiarized with these criteria.
The only ranking that uses methods other than subjective surveys of employers or experts, i.e., ranks HEIs according to objective criteria (indices), is TOP-200. However, the components of the indices still evoke caution. TOP-200 uses quantitative parameters that are no longer trustworthy, such as the number of teachers with science degrees and academic titles, or the rate of Bachelor students to Master students. Their international recognition index mostly consists of declarative components. The only non-declarative indicator has a methodological shortcoming, because measuring the absolute number of foreign students, as opposed to the proportion of foreign students, automatically favors large HEIs.
Thus, the use of rankings in the distribution of state-procured places is appropriate only if their methodology is explained in detail to the members of jury and if all the above-mentioned aspects are taken into account.
In 2015, bibliometric data and citation indexes were added to the criteria for distributing the state-procured places. Let us skip the discussion about the risks of using citation indexes and about the monopoly position of bibliographic databases that resemble giant business empires. I will only note that the discussion is not over, on the contrary, it is gaining momentum. In any case, this data is only one of the many means of evaluating the visibility of scientific work (and not necessarily its quality), so they can be used only in combination with other parameters.
At the same time, in some fields, scientometric indexes are not always an efficient tool. For example, the relations between these indexes and history, literature and some other fields are far from perfect. First of all, research in humanities usually produces a book rather than an article. An article is more of an attempt to formulate a question. Second, the topics of historians or historians of literature are often local or national, which limits their opportunities to publish in journals indexed by international bibliometric databases and slows down their accumulation of citations. Third, the citation practices themselves are different in these fields. For example, when historians look for new topics, they often do not have a large array of possible references, so they refer mostly to the original sources that are the objects of their research (Archambault, Gagné, 2004, 53-54). Fourth, humanities have their own specificity, which is usually manifested in the lack of collaborative research, particularly lack of international collaboration (Kamalski, Plume, 2013, 13-14). Laboratory research, made in the course of a group project, logically results in a collective publication. In contrast, researchers in humanities usually work on their own. All this makes citation indices less effective as a criterion for distributing state funding for programs in humanities.
Let us go back to the situation with the competitive distribution in 2012-15. For the lack of objective criteria of quality, which could have been used for the distribution, another problem emerged. In practice, any jury, even if its members are most qualified professionals, is unlikely to carry out an efficient distribution. The problem is that there are about 200 HEIs under the Ministry of Education, and they specialize in a wide range of fields. This means that the jury of the Ministry of Education is physically unable to engage a sufficient number of professionals. At the same time, the more members there are in a jury, the bigger the risk of lobbying certain universities’ interests. Thus, efficient distribution is practically impossible without automated and reliable systems of monitoring the quality of higher education.
Distribution of State-Procured Places for Bachelor Programs in 2016 – voucher model
Although the new system of distribution of state-procured places for Bachelor programs has not yet been implemented, and only some elements of it (the preference system) were tested in 2015, we can still make some assumptions about its possible advantages and disadvantages.
One of the greatest advantages of this approach is removing the agents of state procurement from the distribution procedure, which means that the flaws related to the functioning of competition commissions will be eliminated. It makes the procedure of distributing state-procured places for Bachelor programs among HEIs transparent and improves its trustworthiness.
At the same time, there is at least one substantial flaw in the voucher model. First, the entrant’s choice does not always guarantee that the funds will flow to powerful universities. Even if we assume that only those school leavers who are really motivated to study in the best universities will be eligible to state-funded education, there is a considerable danger of choosing an institution based on image, rather than on indicators of quality. The zone of increased risk is formed by popular programs in social sciences, humanities and economics, offered by many universities of varying quality.
In addition, the model, despite its undoubted transparency, increases the unpredictability of receiving (or being denied) state funding for the universities themselves, since entrants’ behavior is often less predictable than distribution by inertia via jury, or distribution according to articulated formulas. The unpredictability threatens primarily regional universities which are less popular among school leavers.
Weaknesses of the State Procurement System
This chapter provides an overview of the flaws in the state procurement system as a model of state funding of higher education. In order to better describe the shortcomings, they are categorized by the specific groups whose interests are harmed or, on the contrary, promoted by them.
First of all, the fact that a university has state-procured places does not guarantee the quality of a particular educational program, and can’t be used as a reference point by sensible applicants in their choice of HEIs and program.
The number of state-procured places in some educational fields (this applies primarily to professionally oriented programs) are not based on the needs of the labor market and employers. As a result, when students graduate, it is difficult for them to find a job in their professional field.
At the moment of admission, the link between academic performance and free education breaks. Academic stipend could be an incentive, but the requirements to receive it are not too high, and its size is rather small. In addition, it is unavailable to students who pay tuition fees themselves, even though it is supposed to equally motivate all students to achieve higher academic results.
At the same time, the current system of distribution of state procurement satisfies the needs of those students who are interested only in studying free of charge and do not care about the particular university or program. Such students usually choose to apply to those HEIs where it will be easier for them to study, the ones that have low academic integrity, tolerate plagiarism and bribery. In certain circumstances, the current system for distributing state procurement stimulates the demand for low-quality education among entrants.
At the same time, those who look for high-quality education do not have the relevant information about educational programs and opportunities provided by universities, since Ukrainian HEIs are not sufficiently open (CEDOS University transparency ranking, 2015).
One of the most substantial problems of the system of state procurement, from teachers’ point of view, is that teachers are unable to adequately evaluate the performance of students who fail to master the course material. Teachers are demotivated by the excessive dependence of funding on the number of students and the disregard for indicators of educational quality.
In fact, the resources to provide effective financial incentives based on the quality of teachers’ research or teaching activities have already been exhausted in the state procurement system. At the same time, legislation does not provide effective sanctions against those who do not adhere to the principles of academic integrity. These circumstances serve as a breeding ground for imitation of teaching, which is becoming more and more prevalent.
In addition, the unattractiveness of teaching positions for motivated and honest teachers often leaves universities without any incentive to organize actual competition rounds for vacant teaching positions.
The opaque process of planning and distributing the state-procured places among universities, as well as the privileges of some HEIs which receive higher funding according to special rates (Стадний, 2013), make the state procurement system barely predictable for universities themselves.
At the same time, the lack of efficient systems for monitoring the quality of higher education, as well as the disregard for indicators of quality in the distribution of state procurement, is an advantage rather than disadvantage for some HEIs. Many HEIs are not interested in demonstrating the real data about the state of their study programs and other information which could affect the distribution of budget funds.
The fact that differences in the cost of study in various fields (in various programs) are ignored in the process of distribution state-procured places motivates universities to open programs which do not fit in their profile, but which are popular among entrants. The tendency is particularly widespread among technical and agrarian universities. The quality of such programs is often considerably inferior to the quality of the programs which fit into the HEI’s specialization. However, the tuition fees paid by students at such uncharacteristic programs serve as a source of funds for filling up holes in the university budget.
The excessive dependence of funding on the number of students causes the quality of higher education to deteriorate, since universities are not interested in expelling failing students. Formalized control over the execution of state procurement does not provide incentives for HEIs to meet the society’s needs or take into account regional or national economic priorities. HEIs usually base their suggestions about the number of required state-procured places solely on their own staffing needs.
Stipend fund is received with the bulk of the total state procurement funding, and, depending on the university, it can comprise 20 to 45 percent of spendings from the general university budget. Unlike labor compensations or utilities fees, the size of the stipend fund is not so predictable, and this unpredictability impels universities to resort to informal artificial limitations of their students’ academic achievements.
Employers’ influence on the forming and distribution of state procurement is rather sporadic; legislation mentions the participation of employers in these processes only formally, so the level of their influence depends on their own situational activity and the willingness of the agents of state procurement to include them in decision making.
At the same time, various employer unions and professional associations are active in the state procurement system to various extents. Actually, employer unions cannot predict the need for specific professionals on a national scale, since they do not represent all the fields and areas of employment for graduates. At the same time, those employer organizations which are more involved and actually connected with business in their field, often cannot predict their staffing needs four to six years in advance because of the unstable political and economic situation, as well as the unstable state of foreign markets.
In many cases, there is mutual distrust between employers and universities. First, there is no trust in the system of evaluation used in HEIs. Second, active employer organizations criticize the overdependence of HEIs’ suggestions about the number of state-procured places on their own internal needs, rather than the needs of the labor market or of science and innovation. Third, in Ukraine, there has been very little experience of efficient, institutionally established cooperation between HEIs and employers in determining the content of study programs or final exams, or in providing adequate practical training. These things are organically linked, but each side tries to solve only some of them, ignoring the needs of the other side. For example, businesses try to influence the content of study programs and the final examination procedure, while HEIs try to obtain base for practice for their students. So the suggestions and needs of the sides often differ, and every side tries to introduce changes at the expense of the other.
At the same time, some employers have learned to use the flaws in the higher education system in general and the state procurement in particular. I mean the opportunity to obtain cheap workforce and to complete their training, which costs less than the hypothetical wages of graduates with proper knowledge and skills. In this situation, businesses often hire students who already have the necessary fundamental knowledge, but who are still studying, and provide them with the relevant professional skills; by so doing, they save money on labor compensation in the medium term. However, it reduces the opportunities for mobility on the labor market for the students in the long term.
Nominally, the numbers of state-procured places must be formed using calculations which are no longer relevant (they apply more to planned economy), using data that do not cover the whole economy. That is why, in practice, the numbers are based on the previous year’s numbers (i.e., formed by inertia), as well as on the outcomes of negotiations between each agent of state procurement and the Ministry of Finance. So the distribution of funds (of the number of state-procured places) between fields and programs does not take into account the needs of the labor market and the need for human capital development.
The current system of data collection does not allow to predict the needs of the labor market even in the medium term. The Ministry of Economy lists the following obstacles in particular:
- lack of data on the companies’ need for workforce across the labor market;
- state Statistics Service’s reports on employment in different professions are not representative enough;
- it is impossible to obtain reliable information from central and local governments without it being distorted;
- employer associations do not provide their real predicted need for professionals and workers.
In addition, because of the lack of established priorities in economic development, as well as of specific strategies of recruiting the human capital required to work on these priorities, prediction is made significantly more difficult even in wide terms, such as determining promising fields and specific clusters.
The current system of guaranteed number of state-procured places can be efficient only in the situation of financial prosperity and stable functioning of the system. However, it rules out the transition to other models, since it is difficult to keep up the necessary level of funding and to provide state funding to a proper number of students, if you need to review standard costs of training students in various fields and to eliminate the imbalances caused by institutional lobbyism.
The lack of unified and transparent system of collecting and analysing data about the functioning of HEIs reduces the opportunity to use relevant indicators that bear the smallest risk of Goodhart’s law in the process of distributing state-procured places. The absolute majority of indicators available at the moment are quantitative data about students and teachers, provided directly by higher education institutions. This means, first, that such data cannot be trusted with sufficient certainty; second, that they practically do not communicate anything about the quality of educational process. In fact, Ukraine has no automated data collection systems (in which information is retrieved not from the HEIs themselves) about the relations between the labor market and higher education, about the opinions of students about their programs and universities, about the international activities of Ukrainian universities, and about other parameters which are often used as indicators of the university’s efficiency, performance and output.
Stipend fund takes a third of the total state funding of higher education . Its major part is directed to paying academic stipends, which, despite their intended purpose, do not incentivize students to strive for outstanding academic achievement. At the same time, social stipends (as well as exemptions on the stage of admission) for certain population categories do not improve access to higher education, because the level of secondary education among these categories is insufficient for admission or for studying in a university. In addition, social stipends are available mostly to certain categories of benefit recipients and do not take into account the needs of the wider population of students (entrants) who have the necessary level of knowledge to be accepted to a HEI and receive state funding for their education, but do not have the sufficient material support to cover the associated costs (accommodation, food, travel, etc.).
Localization of performance-based funding model for Ukraine
During its almost forty years in use, performance-based funding (PBF) took various forms, but it has always been based on the invariable principle that institutions receive funding from the state budget for succeeding in certain predefined parameters. The parameters are included as components into a formula which serves to calculate the amounts of funding.
One of the first cases of implementing this model was in Tennessee in 1978 (Friedel et al., 2013). Since then, PBF has gradually been introduced into higher education funding in 32 American states. The model mostly calculates only a fraction of funding, from 5 to 50 percent. Another five states are currently gradually introducing PBF (NCSL, 2015).
Since the mid-1990s, some of the EU countries introduced PBF to fund a certain portion of their universities’ spendings. As of 2014, the calculation of the amount of state funding according to a formula with certain indicators is used in 21 EU countries and on some territories of Germany, Spain and the UK (Bennetot Pruvot, Estermann, 2015, 18).
Researchers divide the indicators in the formula into two main groups, the input indicators and the output indicators. The first group mostly includes the number of students studying for different degrees and at various programs, the number of teachers, or the student-to-teacher ratio. The second group includes a wider range of indicators: the number of graduates, the amounts of income from sources other than the state budget, the number of research contracts, the citation index, the rate of employment of graduates, the achievement of strategic goals (Bennetot Pruvot, Estermann, 2015, 24; De Boer et al., 2015, 9). The choice of indicators depends on the purpose of introducing PBF and the initial conditions of a particular higher education system, group of universities or activity (teaching, research). Therefore, there are no universal indicators that are equally efficient in different systems.
In order to describe the PBF model that could replace the state procurement system in Ukraine, we must answer at least a couple of questions. How do we calculate the total amount of state funding of higher education? What portion of the total funding can be determined using PBF? What indicators can be included in the formula? What are the external and internal transformations required to introduce PBF for Ukrainian universities?
The Total Amount of Funding and Priorities
The legally guaranteed ratio of state funding to total population (with the simultaneous prohibition to decrease spendings) is a more predictable form of ensuring stable funding than a portion of GDP or of total budget spendings. Moreover, because of the ageing of Ukrainian population, the approach will increase access to higher education. At the same time, it may hinder the redistribution of state funding in favor of the more costly programs, since such redistribution would reduce the number of those whose education is state-funded. That is why it is advisable to set a certain guaranteed amount of state funding in relation to demographic indicators only after the funding priorities are defined.
Prediction of the labor market’s needs is unusable for determining the amounts of funding for more than 130 fields. The obstacles to using it, in addition to the problems mentioned in the previous chapters, include differences in classification and the high changeability of the labor market’s needs at the macro level. Cooperation between employers and universities on the local level is more efficient. The cooperation must primarily center on particular skills and knowledge, rather than the numbers of graduates in any given profession. Therefore, it is about predicting the need for certain knowledge and skills.
On the national level, on the other hand, it is advisable to distribute the state funding of higher education in accordance with the state’s economic priorities, taking into account the analysis of the behavior of households in terms of choosing the direction of higher education. These two parameters can replace the current mechanism of determining the planned numbers of state-procured places. The state must set as its priorities those fields which are less popular among entrants, as well as the strategically important fields. This way, a balance between the state’s needs and the entrants’ demand can be achieved. And it seems that the economic priorities will mostly correlate with the needs of well-organized professional associations. In other words, the economic priorities will, most probably, include all or the majority of the fields which have solid employer associations which actively communicate with businesses in their sector.
However, the usual model of planned state procurement can remain in use for funding the workforce training in the systems of the Ministry of Defence, the Ministry of Internal Affairs and other military, security and law enforcement agencies, since, in their case, education is, in practice, a closed circle from training to directed employment. In addition, there is a belief that the traditional model is applicable to educating teachers and doctors, but additional research into this claim is needed.
Provided that economic priorities are defined, and the demand among households is analysed, the total amount of state spending on education can be distributed in two ways. According to the first method, it can be divided between wide fields (for example, according to an equivalent of ISCED); and then, within the fields, distributed among universities according to a formula (formulas). A variation of the same approach is to divide the total amount of spending into wide types of HEIs, rather than fields. For example, the academic and the professional type, or the types listed in current legislation (university, academy/institute, college). However, this approach is unstable, because it bears a high risk of lobbying by some ministries and powerful universities on the stage of planning of the state funding. Another method can be to take the economic priorities and the household demand into account already at the stage of distributing funds between universities using a formula. In this approach, the contract between the state and a HEI must indicate the fields in which the HEI can use the received funds. In addition, as long as the admission cap remains unrealistic (currently it does), it is necessary to set the maximum number of students universities are allowed to accept, because universities will naturally try to enroll as many students as possible. The second approach can serve as a transitional method for the period until the chances of lobbying are reduced.
A technical but important question is how to set the date of the distribution and transfer of funding. The divergence between the budget year and the academic year is usually overcome by distributing funding for a year and a half, two, or even three years. However, the unpredictability of Ukrainian situation does not allow that kind of long-term planning. So the only option left is to distribute funding for one year, as usual. It cannot be distributed earlier than the date of approving the state budget, and the formula must include only the indicators for which the data have already been collected and verified, and can be accessed freely. Thus, if the state budget is approved in December, the formula can calculate the distribution in December or January based on the data that passed all the stages before November.
Indicators to Be Included in the Formula
The formula must have the purpose of funding that will determine which portion of university funding it will calculate. Our aim here is to describe the model intended to introduce transparent and efficient distribution of funding the study process in particular. This means that the model is primarily designed to determine the spendings related to the educational component of university activities. That is why the model does not take into account research spendings, capital spendings, or funding of the social welfare of students, which, in our opinion, must be separated into a special flow of funding. In addition, student welfare must be excluded from the university’s responsibilities and included into the general welfare system, where, according to the recent legislative changes, a system of verification and improved targeting is to be introduced.
The abovementioned purpose can be divided into three goals: to distribute transparently (1), as close to the necessary costs as possible (2), and primarily to those HEIs that demonstrate the best performance results (3).
These goals lead to the principles which must serve as a basis for choosing what indicators to use: calculability, clarity and attainability. The parameters must be strictly calculable; their sources must be clear and trustworthy; and the data must be open. The parameters must also be compatible with the capabilities of Ukrainian universities.
The components of the formula for calculating the funding amounts can be divided into three types: input indicators, output indicators and the indicator of the previous year’s funding. All three groups have their own function and affect the behavior of universities. In choosing the indicators according to the declared goals and principles, some important conditions must be taken into account in order to predict the risks and be able to reach the goals. Is the indicator available and what is the source of data? What is the level of trust in the indicator among HEIs and employers, and what is the probability of triggering Goodhart’s law? What fields and types of HEIs the indicator is relevant for? Which activity is best represented by the indicator? What is the probability of Matthew effect ? What is the cost of achieving the indicator (operating costs, capital expenditures), to what extent it encourages the search for non-government funding?
In the situation where total state funding of higher education is being cut, the formula of performance-based distribution can only function properly if it determines not the absolute amount of funding for HEIs but the relative one. That is, if the formula will calculate the fraction of the total funding.
At the moment, several separate systems of data collection or publishing operate in Ukraine: the Unified State Electronic Database on Education, the State Statistics Service, the Institute of Education Analysis, Ukrainian State Center of International Education, State Treasury Service, the Official Portal of Finances of Ukraine. For a while, there were also projects dedicated to collecting scientometrics data from some databases, such as jsi.net.ua, a project by Ukrainian Research and Academic Network (URAN). To our best knowledge, Ukrainian officials (the government, the Ministry of Education) have no access to the statistical data in international bibliographic databases (Scopus, Web of Science) or patent databases (OECD, EPO, PCT, USPTO, Triadic Patent Families). In general, if you compare the data about higher education in Eurostat or Education at a Glance with the scope of data collected using various means in Ukraine, the latter will only correspond to a third of the former.
The available quantitative parameters can be divided into the following groups: the data about students and graduates (educational degrees, specializations, full-time or distance programs); the data about teachers (degrees, academic status); the data about income (fees for core and additional services, grants and gifts, foreign sources); the data about spendings. In addition, in December 2015, the Institute for Education Analyses (Ministry think tank) started to collect data about the international activities of Ukrainian HEIs (the number of foreign students, the number of students and teachers with the experience of academic mobility or degrees from foreign universities, programs in foreign languages).
However, there are no data about some categories of important indicators, particularly about the trajectories of graduates in the labor market; about the cooperation between universities and businesses; about citation indexes and patents; and about the evaluation of programs and universities by students and employers.
The next step should be to introduce a unified platform for the input and output of information collected directly from universities. It is crucial to ensure that there is a unified data and a unified input of periodic (yearly or more frequent) measurement. It is also mandatory that the collected data are published in maximum detail as open depersonified data, which will also allow to access the primary data without identifiers. Depending on the resources allocated to developing such a system, it can also offer some basic automatic analysis and data aggregation in a databox/datacube.
Most of the performance-based funding systems are based on formulas in which input indicators prevail, that is, the parameters that most adequately reflect the key spending points and the amounts required to provide the necessary equipment, resources and technical necessities, as well as to meet the staffing requirements. At the same time, the formula must include only a few key parameters; the more detailed the spendings, the less clear — and, therefore, the less realizable — the funding system becomes.
According to the research by European Universities Association, the most widespread and significant parameter is the number of students (Bennetot Pruvot, Estermann, 2015, 24). It is important to distinguish the partial relation between institutional funding and the number of students from the specifically Ukrainian practice of calculating the number of teaching positions based on the number of students. In our case, it is the former, which includes the opportunity for HEIs to configure the number of teaching positions depending on the faculty responsibilities and workload. The cost of training in various programs naturally differs, so the indicator must take into account at least three factors: the form of studying (full-time or distance program), the academic degree, and the cost index for various fields.
Under the condition that the formula will, in the end, calculate the relative (not the absolute size) of funding for HEIs, that is, determine the portion of total educational budget that goes to any particular university, the so-called basic absolute size of funding per student is not fundamentally relevant for the formula itself; instead, the abovementioned factors are crucial.
According to the data provided by the MInistry of Education, whose system comprises the majority of HEIs of various specializations, the average cost of training a Bachelor student in 2013 was 1.5 times higher than the cost of training a Junior Specialist. The rate is approximately the same for all the agents of state procurement.
The difference between training a Bachelor and a Master student was as little as 1.05 times, and it was the same not only for the Ministry of Education, but also for the rest of the major “civilian” agents of state procurement (the Ministry of Culture and the Ministry of Agriculture). In general, if we take into account all the agents of state procurement, the average cost of training a Bachelor student was 1.15 times lower than training a Master student. The ratio is skewed by the fact that in the system of the Ministry of Internal Affairs, which is one of the biggest agents of state procurement, the difference was 1.8.
One should keep in mind that the current differences in training costs at various fields are most likely to reflect not the actual differences, but rather the lobbying of some ministries and universities. Therefore, we can’t use it but we can use international practice as a reference point, taking into account that, in Ukrainian situation, wide fields of education will look more trustworthy than smaller divisions. Based on the practice in Poland, Czech Republic, Romania, Germany (BIP, 2013; Vlăsceanu, Miroiu, 2012; Pabian et al., 2006; Statistisches Bundesamt, 2014), we can form the following cost coefficients for Ukrainian list of fields:
|Humanities, Theology, Social and behavioral sciences, Journalism, Governance and administration, Law, Social work, Services||1|
|Information technology, Education, Mathematics and statistics||1-1,5|
|Mechanical engineering, Electrical engineering, Automation and device construction, Electronics and communications, Production and technology, Transportation, Architecture and construction||1,5-2|
|Chemical and biological engineering, Agricultural science and food supply, Biology, Natural sciences, Culture and arts||2-2,5|
|Veterinary medicine, Healthcare||2,5-3|
In order to determine the cost differences in relation to the form of studying (full-time or distance), a special research must be conducted, since the existing financial reports and the reports on state procurement do not reflect the actual state of affairs. And the differences in tuition fees for full-time and distance programs, which usually is about 1.5 to 2 times, reflects the students’ purchasing power and the popularity of any particular profession, rather than the actual cost.
The information collected from universities is traditionally considered to be the least trustworthy. Some HEIs will always be interested in manipulating indicators if they are included in the formula for funding distribution. The information they submit can be false, or their quantitative indicators can be impossible to transform into the qualitative ones. That is, Goodhart’s law can apply.
In order to reduce these risks, it is necessary that the formula includes the information which, first, characterizes the qualitative side of the functioning of HEIs; second, is not collected by HEIs, or at least involves them only indirectly. This kind of information includes a couple of groups of data, in particular, the level of knowledge among entrants, the connection between the labor market and HEIs, evaluation by students and employers, the visibility/influence of research, and international activities.
The system of External Independent Evaluation places the levels of knowledge among entrants on a relative scale. To distribute funding based solely and directly on the EIE results is to absolutize this aspect, turning the level of secondary education and the family capital of school leavers into indicators which determine whether HEIs receive or do not receive funding, although universities barely play any role in it. However, it is justified to partially reflect the parameter in the formula. In particular, in the cases when the university’s policy of setting the minimum grade which makes an applicant eligible for admission is a part of the indicator. Thus, the average grade in EIE test of those entrants who were accepted to receive state-funded education the year before the funding distribution takes place can be used as one of the output indicators. At the same time, at the the first year stage of implementing the formula, it is advisable to adjust the indicator for the number of state-procured places available at the moment of admission, because the less places the university had, the more intense was the competition, and therefore, the higher the average grade of the accepted freshmen.
The rate of employment among graduates is used for distributing funding for universities in seven European countries (Bennetot Pruvot, Estermann, 2015, 26). One of the many possible ways to evaluate the connection between HEIs and the labor market is to track a graduate’s trajectory by his or her appearances in the social security and tax systems. Some preconditions for this already exist. Every student is listed in the Unified State Electronic Database on Education by his or her taxpayer’s number, and the same number is later used to tax their income and salaries. So it is possible to develop software that would synchronize the Electronic Database on Education and the corresponding taxpayer databases in order to obtain data about the trajectories of graduates in the labor market. Obviously, the system requires proper personal data protection, so only the data that have been aggregated to a certain level can be publicly accessible. Indeed, the public nature of the data will require special attention. On the one hand, the trust in this indicator, if it is used for funding distribution, will build up only if the data will be accessible in a user-friendly form; on the other hand, they can also serve as a point of reference for entrants, their parents, and businesses — that is, as a source of extra-budget income for universities.
At the same time, the data about the trajectories of graduates in the labor market can be used for distributing the government funding between HEIs only if some important risks are taken into account. First, the quality of higher education is only one of the many factors that affect the success of graduates. A lot depends on the situation in the economy of the country, region, or the particular industry, as well as the condition of international markets. In addition, the success of graduates depends on their social (family) capital, their motivation during their studies. Second, the culture of declaring your income is still insufficiently developed in Ukraine; “shadow” employment is widespread, and it will hinder the tracking. Third, it takes different periods of time to find official jobs for students in different fields. In fact, it is not possible to establish a fixed connection between the completed educational program and particular positions in the labor market. There are programs in humanities, social sciences and mathematics, which are not strongly or not at all connected to certain professions and fields, in contrast to programs in IT, natural sciences, engineering, education or health.
These and other circumstances require certain adjustments in the implementation of the system of labor trajectory tracking. Evidently, it is necessary to dedicate a certain period of the system’s functioning solely to informing universities. It will allow HEIs to choose their own strategies. The weight of this indicator in the distribution of funding cannot be significant, and its increase (if planned) can happen only after the system’s functioning has been studied and numerous simulations have been run. At the same time, during the trial period, it is necessary to ensure that there is simultaneous qualitative research of the behavior of graduates and employers in the labor market, which would reduce the risk of burdening the HEIs with unnatural responsibilities.
At the initial stage, it is justified to take into account the plain fact of the graduates’ employment, under condition that the measurement includes only those graduates who completed their studies no sooner than 35 months before the calculation. Research shows that 34,4 months is the average time it takes for a graduate to find stable or satisfactory employment (Лібанова et al., 2014, 45). In addition, it is necessary to determine, by special research, the upper limit of the period for which the graduates must be taken into account. Moreover, the system must count someone who has a BA and an MA from two different universities as two graduates, but count someone who has both a BA and an MA from the same university as one graduate.
The results of national surveys of students are rarely used to distribute state funding. Today, they are used for this purpose only in Finland and in Tennessee (De Boer et al., 2015, 9). They are much more often used to analyse certain phenomena related to ensuring the quality of higher education. In addition, the survey results are usually publicly available. So entrants partially base their choice of a university on student surveys; therefore, the surveys indirectly affect the university’s income to the extent that depends on the system of funding distribution.
As with other indicators, the trust in this one (primarily among students themselves) will increase if universities staff will be excluded from the surveying process. That is, direct connection with the surveyed student is a necessary condition. Probably the most suitable for this purpose is the system of personal electronic cabinets which only the students will be able to open with their personal keys. At present, only entrants have such cabinets when they apply to universities, but if they kept the cabinets after they were accepted, it would be possible to establish a direct connection with them. However, it makes sense to arrange the surveys in the senior years, since by that time the students will already have enough experience with their program and their university to evaluate them. At the same time, this fact entails the risk that a major part of the survey questionnaires will be left unfilled. Firstly, by their last year, students can simply forget about their cabinets. Secondly, the interest in making a difference for their university will be naturally lower among seniors, because they will understand that their answers can have any impact only after they graduate. In this situation, filling out the questionnaire can be set as a requirement for receiving a degree document; however, such an approach will lead to lower-quality answers. That is why broad offline and online campaigns are needed to popularise the survey and explain its importance. A certain engagement threshold must be established to determine whether the answers about a particular university should be taken into account. Another way to increase the survey engagement could be the informational support by the universities themselves. In order to encourage them, the engagement indicator itself can be included in the formula for funding distribution, which means that high students’ engagement in survey could increase funding; however, the evaluation (the survey result) must have greater influence in the formula. At the same time, in order to increase the trust in the survey on the part of universities, it must provide an additional benefit to them. For example, in addition to the mandatory questions (which will count for the formula), HEIs can have an opportunity to add their own questions, albeit only a limited number of them. The answers to these questions could be publicly accessible only if universities want them to be. In contrast, the general survey results, included in the formula, have to be published online mandatorily, as in the case of British National Student Survey (Стадний, 2015, 19-22).
One of the most important elements of surveying is questionnaire design. It must not have too many questions; on average, the questionnaire must take no more than 20 minutes to fill in. The questions must be easy to understand and to evaluate on the Likert scale. Preliminary focus groups will allow to determine the best way to formulate questions so that the answers cover the key issues in the system, particularly the evaluation of teaching quality, teacher availability, practical skills training, the condition of HEI’s infrastructure and its information policy, opportunities to shape one’s own educational trajectory, and the prevalence of typical academic misconducts.
Another possible performance parameter is the size of the university’s non-governmental income. However, in Ukrainian situation, one can rely mostly on the cases when a HEI has won the trust of (qualified for) international grant programs. So the indicator of the fraction of income received from international grants or donors is completely relevant. It is used in eleven european countries (Bennetot Pruvot, Estermann, 2015, 26).
In addition, it is important to motivate HEIs to spend their funding on research. From this perspective, not only the indicator of income, but also of the indicator of spendings becomes important. For example, the formula could include the portion of income from tuition fees spent on research activities. However, in this case, it is necessary to ensure that the spendings are transparent, and to reduce the risk of fabricating research.
The main advantage of research visibility indicators is that the data for them are collected independently from HEIs. That is why even those who oppose their use agree that they have one of the highest levels of trustworthiness among the methods of evaluating the visibility of scientific activities of universities. At the same time, in Ukraine, research does not have the adequate infrastructure and support, so the majority of Ukrainian HEIs can find any more or less visible scientific results unfeasible. It will motivate the universities to try to imitate the indicators. Thus, it is necessary to differentiate requirements for various HEI types. At the same time, universities will be able to independently choose a certain type in which they will be able to achieve better results. Obviously, proper quality of teaching requires higher quality of research in a more academically-oriented university, while the same disciplines in professionally-oriented HEI do not necessarily depend so much on the teachers’ research activities. Therefore, in order to use the indicators of scientific activity for the distribution of government funding of higher education, it is necessary to assess the importance of scientific activity for educational service provision in any particular program, institution or region. It is also necessary to take into account the special characteristics of some fields, primarily some disciplines in the humanities, where research can not always be evaluated using scientometrics data, as discussed above.
The indicator of past funding
The indicator of past funding is used to achieve two goals. First, to reduce the unexpected risks of the transition to a new funding model: for this reason, for example, Poland and Italy started to implement PBF under condition that 70 to 77 percent of current funding was formed based on the previous year’s funding. Second, the approach is used to increase or decrease the so-called Matthew effect, where the stronger competitors win in the situation of free competition because of their better initial conditions.
Mostly, although not always, the higher weight of the indicator of the previous period’s funding in the formula, especially if the period is not long (for example, the previous year), is more advantageous for bigger universities that had significant income from state budget under the previous system of distribution. In contrast, the lower weight of the indicator, while using new indicators in the formula (especially the output indicators) will rather benefit the medium-sized, but powerful universities.
In Ukrainian situation, it is necessary to take into account the conservative character of Ukrainian HEIs and low trust in any innovation in the field, so the initial weight of the indicator of past funding must be at least 70 percent. At the same time, in order to reduce the Matthew effect, it is necessary to establish a plan for gradual reduction of the weight of this parameter down to 30 percent in no more than first four to five years of the system’s functioning.
The Necessary Conditions
In addition to the abovementioned trial period, needed to test and calibrate the data collection systems, there are a couple of important conditions which must be met in order to ensure the stable functioning of performance-based state funding of HEIs.
In order to build up the trust in the system, it is necessary to provide everyone with an opportunity to check the calculations on their own using the formula. This entails open access to all data included in the formula. In addition, it would be useful to launch an online calculator which could be used to model the amounts of funding for any particular university.
Performance-based funding of universities (of each of the defined HEI types, or of each of the fields at later stages) must be concentrated in a single fund or budget item, since only under that condition the formula will be able to calculate the fraction of funding for any particular HEI. The state procurement model can remain valid for funding workforce training in the systems of the Ministry of Defence, the Ministry of Internal Affairs and other military, security and law enforcement systems.
The use of funds which universities earn on their own (the so-called special fund) must be relieved from the limitations imposed by the Budget Code . In the present situation, the source of income must correspond to the further use of the funds. In particular, the income received by HEIs from tuition-paying students can be used only to provide educational services. The financial control institutions do not allow to spend these funds on research, despite the fact that, according to the general logic, such spendings not only can but sometimes even must be a part of educational services. Relieving the limitation will allow to mitigate the damage from the transition to the new model of budget funding distribution, since HEIs will be able to cover the gaps in their funding more flexibly. However, the income must be distributed in strict accordance with the declared goals of any particular HEI.
At the same time, it is necessary to make sure that the spending of income obtained from sources other than the state budget is transparent. First of all, the relevant changes in legislation must be introduced, which would extend the validity of the Official Portal for Public Finance of Ukraine to cover extra-budget income of HEIs, which is not included in the State Treasury’s accounts.
It is also necessary to create a legal framework for introducing efficient internal control by supervisor institutions. University administration must be excluded from the process of forming Supervisory Boards. Possibly, the founders of HEIs must be responsible for forming the Boards. The Boards must not include people who are actively connected to the university, or relatives of those who are actively connected to it. In particular, the employees and the students of the HEI whose Board is being formed must not become the Board members, in order to draw a clear line between the functions of supervising certain processes and of being responsible for those processes. The workers and the students of other HEIs cannot become Board members, since they, in effect, represent rival institutions. In this situation, employers who are interested in graduates of a particular HEI become the key potential members of the supervisory board. It is totally possible that they will turn out to be its graduates, so they will have certain obligations to their alma mater. In addition, the Supervisory Board must be given the real power of control over forming the university’s budget, distributing its wage fund, and implementing its development strategy. In particular, its powers must include the possibility to initiate early resignation of the university administration (not necessarily of the rector) precisely for failing to implement the development strategy and for financial violations.
First of all, however, strategic planning must become a necessary element of developing a successful higher education institution. At present, only a third of national universities have up-to-date strategies of development (CEDOS, 2015). However, experience shows that strategic planning provides an opportunity to achieve certain material and immaterial advantages: to improve financial indicators, to expand the opportunities for saving and efficient use of resources, to define priorities. It becomes possible to predict future problems and minimize the impact of unfavorable factors, as well as to predict new opportunities and use them to the fullest. As a result, it leads to faster and more efficient decision making and to more efficient functioning; therefore, motivation and satisfaction of the staff improves, and the prestige of the HEI among students increases. In order to make HEIs finally start to take strategies more seriously, the rector’s mandate of trust must be linked to the university’s strategy. If the strategy becomes an integral part of the contract between the head of any particular university and the Ministry of Education, it will be the legal grounds for demanding to implement the strategy and base the rector’s policies on it. Another option might be to include certain strategic goals and indicators in the appendix to the agreement about providing additional state funding, which will be described later in the chapter.
The system of external quality assurance must aim for gradual reduction of the importance of state standards of higher education ; increase of the role of non-governmental organizations for quality assurance, primarily powerful professional associations; and constant development of internal quality assurance in the HEIs. One of the key elements of internal quality assurance must be the constant self-analysis of educational programs and business processes at universities.
At the same time, the success of the proposed model will depend on how consciously entrants choose particular universities and programs. So it is necessary to launch a network of advisors who would provide professional advice on what to take into account while choosing a university, how to verify the information about it, and which economic clusters are promising. In addition, the consulting centers could popularise the attitude that learning is a constant process, inevitable in the current circumstances, and spread the awareness of the fact that geographical and interdisciplinary mobility of professionals in the labor market is becoming a more and more widespread attribute of a successful career.
The proposed model will, most probably, lead to a reduction in the number of students whose education costs are covered by the state budget, because it will entail an increase in budget spendings per student. Therefore, the resources of the stipend fund must be dedicated to improving the access to higher education for those who are capable of being accepted in a competitive admission campaign, but are unable to cover the costs associated with studying: accommodation, transportation, food, learning materials, etc (need-based approach). It means that social stipend must become central to the stipend provision system; its distribution must be as targeted as possible, and not based on broad population categories eligible for welfare payments. The income of the persons who support a student and the distance from his or her place of residence to the HEI must be two key criteria of stipend eligibility. The social stipend recipients should be grouped into a number of categories that will receive stipends of different sizes. If a student belongs to the group with the lowest income, his or her social stipend must be higher. It is also necessary to adjust the stipend by the index of living costs in different cities, since accommodation and food costs often significantly differ even among the major student cities of Ukraine, not to mention the rest of the country. In addition, a priority principle should be adopted. If the funds in the state budget are insufficient to cover stipends for everyone who needs them, the social stipend must be given first to the students with the lowest income, who are the most vulnerable.
Also, as it was already noted, it is advisable to exclude the social security system (stipends) for students from the responsibilities of universities, including it into the general welfare system, in which, according to the recent legislative changes, a system of verification will be introduced, and which will become more targeted.
In contrast, the academic stipend must be turned into a real incentive to strive for extraordinary academic achievements, and it must be available to all students, regardless of who pays for their education (tuition fee students can’t get them now). In the situation when the basic stipend form will be the social one (need-based), the number of academic stipends will be significantly smaller than now. Moreover, their number will be reduced because their size will increase, which is a necessary condition for turning the stipend into an academic incentive. Eligibility for the academic stipend should be based not only on the GPA (which must be not lower than A), but also taking into account additional criteria, such as achievements in sciences, arts or sports.
Another policy that could improve access to higher education in the proposed model are soft student loans provided by the government. However, they must be given only to those who will study at the programs which have the highest chance of leading to employment and providing stable income after graduation. That way, the loans will not lead to debt accumulation. At the moment, such programs probably lie in the field of information technologies. However, the mechanisms for determining such fields and the link between them and particular field can be established only after solving the same problems which arise when we try to introduce the employment indicator.
Unfortunately, the policies outlined here have no influence on one of the most important factors that limit the access to higher education, namely the low quality of secondary education. However, the problem is much wider than the issue of higher education funding, and it requires separate broad research.
Another condition necessary for the success of the model described here is the academic community’s awareness and understanding of the planned changes and the instruments that will be used to introduce them. High level of awareness among managers, teachers and students increases their chances of being prepared and avoid negative consequences. For this reason, a wide information campaign must be organized before and during the process of implementation of the model. We must acknowledge that, if the key agents in the process are well-informed, they are likely not only to support the changes, but also to resist them. However, their ignorance can only increase resistance.
The provision of funding calculated using the formula must be based on an agreement between the founder and the university. The agreement can have at least two purposes. First, it establishes the sides’ responsibilities and the amounts of funding, and it can, to a certain extent, regulate the use of the provided funds. Second, it can describe the conditions for receiving certain additional funding.
The founder must assume the responsibility to provide the whole amount of funding, and the university, on its part, must use it according to the agreement. In particular, the agreement must determine the fields in which the university can provide state-funded education, and the cap on the number of students who can be accepted to the programs in those fields. Another way to regulate the use of the funds could be to determine the number of graduates who must complete these programs; however, in that case, the university will be discouraged to expel the students who do not complete the program requirements, and it will decrease the quality of education.
Moreover, the agreement can be an instrument of reducing the Matthew effect. If additional funding is available and a certain level of trust between the sides has been established, the agreement can have some motivating functions. Given that the HEI network is quite extensive, the additional funding should be distributed only among a small number of universities that need help and have the proper preconditions to implement the necessary changes and demonstrate results. Depending on the situation in a university and on its programs, the founder and the university can agree in the course of negotiations on certain indicators to be achieved. These can include the issues of internal quality assurance, dedicating funds to research, implementing the strategy, etc.
Appendix: Brief Description of the Components of Possible Formula
The table below presents the components of possible formula (and their weight) for performance-based funding three different stages. The minimum time period between different stages should be at least two years. One year is needed for the legislative process and establishing necessary systems and one more year will be required to test and simulate their operation. Starting from transitional stage it is proposed to distinguish two different types of HEIs which have slightly different components of the formula.
|Indicators||Initial stage||Transitional stage||Long-term perspective|
|Academic version||Professional version||Academic version||Professional version|
|The amount of state funding of workforce training in the last time period. The higher the past amount, the higher the current funding.||70%||60%||60%||35%||35%|
|The number of students whose studies are state-funded, categorized by full-time/distance programs, degrees, and groups of fields, as of November of the year preceding the year of funding. The higher the number, the higher the funding.||20%||25%||25%||35%||35%|
|The average EIE grade of the entrants accepted to study at the state-funded places, in the year preceding the year of funding. The higher the grade, the higher the funding. On the initial stage the indicator must be adjusted to the level of competition for state-funded places.||10%||10%||10%||10%||10%|
|The fraction of extra-budget income from the students, directed to covering research costs. The higher the fraction, the higher the funding.||1%||2%|
|The fraction of extra-budget income from international grant programs, directed to research activities. The higher the fraction, the higher the funding.||2%||2%|
|The fraction of students who participated in the programs of international credit mobility. The higher the fraction, the higher the funding.||1%||1%||2%||2%|
|The number of Master programs where classes that constitute 36 ECTS credits in total are taught in a foreign (not Russian) language, provided that no less than 10 percent of students study in such programs. The higher the fraction, the higher the funding.||1%||1%||2%||2%|
|The fraction of extra-budget income from business. The higher the fraction, the higher the funding.||3%||4%|
|H-index for HEIs in Scopus or Web of Science (or other indicator of research visibility). The higher the index, the higher the funding.||2%|
|The fraction of employed among those who graduated no less than 3 years ago. The higher the fraction, the higher the funding.||5%||7%|
|The evaluation of the HEI by its senior students (based on a national student survey, provided the necessary engagement). The survey results should be translated to a scale that would allow to determine the final grade. The higher the grade, the higher the funding.||5%||5%|
The conditions for the indicator “The number of students categorized by full-time/distance programs, degrees, and groups of fields”
Bachelor program is 1.5 times more expensive than Junior Specialist program.
Master/Specialist program is 1.2 times more expensive than Bachelor program.
PhD program is 2 times more expensive than Master/Specialist program.
Full-time program is 2 times more expensive than distance program.
Coefficients by Groups of Fields
|International practice||Average value|
|Humanities, Theology, Social and behavioral sciences, Journalism, Governance and administration, Law, Social work, Services||1||1|
|Information technology, Education, Mathematics and statistics||1-1,5||1,25|
|Mechanical engineering, Electrical engineering, Automation and device construction, Electronics and communications, Production and technology, Transportation, Architecture and construction||1,5-2||1,75|
|Chemical and biological engineering, Agricultural science and food supply, Biology, Natural sciences, Culture and arts||2 - 2,5||2,25|
|Veterinary medicine, Healthcare||2,5-3||2,75|
The formula calculates the size of block funding as a fraction of the current year’s State Budget spendings for the corresponding expenditure item.
Волосевич І., Герасимчук С., Костюченко Т. 2015. Молодь України — 2015. Київ: GFK.
Кармазіна О. 2015. Основні показники діяльності вищих навчальних закладів України на початок 2015/16 навчального року. Київ: Державна служба статистики.
Лібанова Е., Цимбал О., Лісогор Л., Марченко І., Ярош О. 2014. Перехід на ринок праці молоді України. Женева: Міжнародна організація праці.
Пилипенко В. 2015. Звіт про результати аналізу формування, розміщення і виконання державного замовлення на підготовку фахівців з вищою освітою. Київ: Рахункова палата.
Світящук І., Солодько А., Стадний Є. 2014. Державне фінансування підготовки кадрів у вищій освіті: досвід та виклики. Київ: Центр дослідження суспільства.
Стадний Є. 2013. “Статус національного ВНЗ: політичний вимір,” Osvita.ua, 26 січня. Відвідано 4 лютого 2016.
Стадний Є., Гладченко М., Когут І., Солодько А., Шевченко І. 2015. Моніторинг прозорості національних ВНЗ. Київ: Аналітичний центр CEDOS.
Archambault É., Gagné É. 2004. The Use of Bibliometrics in the Social Sciences and Humanities. Montreal: Social Sciences and Humanities Research Council of Canada.
Bennetot Pruvot E., Estermann T. 2015. DEFINE Thematic Report: Performance-Based Funding of Universities in Europe. Brussels: European University Association.
Biuletyn Informacji Publicznej (BIP). 2013. Last modified in May. Accessed February 4, 2016.
De Boer H., Jongbloed B., Benneworth P., Cremonini L., Kolster R., Kottmann A., Lemmens-Krug K., Vossensteyn H. 2015. Performance-based funding and performance agreements in fourteen higher education systems. Twente: Center for Higher Education Policy Studies (University of Twente).
CEDOS. 2015. “Фінанси національних університетів.” Last modified May 15. Accessed February 4, 2016.
CEDOS. 2015. “Рейтинг прозорості національних ВНЗ.” Last modified May 20. Accessed February 4, 2016.
Friedel J., Thornton Z., D’Amico M., Katsinas St. 2013. “Performance-Based Funding: The National Landscape,” Education Policy Center. September 19. Accessed February 4, 2016. http://uaedpolicy.ua.edu/uploads/2/1/3/2/21326282/pbf_9-17_web.pdf.
Kamalski J., Plume A. 2013. Comparative Benchmarking of European and US Research Collaboration and Researcher Mobility. Elsevier.
National Conference of State Legislatures. 2015. “Performance-Based Funding For Higher Education”. Last modified July 31. Accessed February 4, 2016.
Pabian P., Melichar M., Šebková H. 2006. Funding Systems and Their Effects on Higher Education Systems. Country study – Czech Republic. Paris: OECD IMHE.
Statistisches Bundesamt. 2014. “Current funding per student by areas of study - 1 000 Euro”. Accessed February 4, 2016.
Vlăsceanu L., Miroiu A. 2012. “Relating quality and funding: The Romanian case.” In European Higher Education at the Crossroads: Between the Bologna Process and National Reforms. Edited by Adrian Curaj, Peter Scott, Lazăr Vlasceanu, Lesley Wilson, 791-811. Dordrecht: Springer.
Translation supported by TTF