Designing/developing a measurement agenda for the post-2015 education goals

Albert Motivans

UNESCO Institute for Statistics

Chief, Education, Indicators and Data Analysis, UNESCO Institute for Statistics, Montreal, Canada 

 

This is an Author’s Original Manuscript of an article submitted for consideration in Compare [the exclusive right to publish residing with Taylor &Francis, copyright BAICE];The final version is available here.

Drawing on the initial results of UN-led expert groups and global consultations among the education and broader development communities – the directions for the post-2015 global priorities have begun to emerge and gain critical momentum.  These efforts have included the UN High Level Panel, the World We Want consultation and a number of global, regional and national consultation exercises. As noted by observers[1] – attention thus far has focused on three key areas for the post-2015 education agenda:

1. Equitable access to education for all children, especially the poorest and most marginalized;
2. Emphasis on learning outcomes and education quality;
3. Learning across a continuum – from early childhood to adolescents and the transition to work.

These are not new areas for the education community, and in fact, have been highlighted by previous efforts, such as the Education For All movement from 1990 and even earlier.  While they may not necessarily imply a new agenda per se, this still represents an opportunity for refining approaches and renewing efforts for political and social mobilization around global progress in these areas.

In addition, these efforts have highlighted the importance of data and measurement as essential for monitoring progess, transparency and accountability. Indeed monitoring indicators help planners make better-informed decisions by presenting the best available evidence. However, in many countries, issues with data availability and quality limit their use and impact. The High Level Panel paper calls for a ‘data revolution’ to take advantage of the advances in technologies for data collection, processing and analysis, which would allow developing countries to “leapfrog” ahead in terms of measuring progress.

But, how do we get from these broad policy areas to meaningful and monitorable indicators? In theory, measurement should respond to a clearly articulated policy framework – it should aim to capture the key concepts and priorities in such a framework. However, in practice, the policy to data process is often a complicated one with many different levels, pathways and players.

Some proposals which list specific indicators have appeared (e.g., High Level Panel, Save the Children, EFA GMR et al). The risks in not considering technical aspects at an early stage of the discussions are evident from the MDG process itself – where some targets/indicators were poorly defined and later dropped or supplemented with other indicators. While it is useful to consider the feasibility of measures when considering goals and monitoring, it can also risk distracting attention away from the formulation of solid rationales for what constitute the most important development and education priorities.

In developing the global measurement agenda it is important to keep in mind two fundamental principles.  First while most of the attention is drawn to articulating a small and influential set of high level goals and the indicators used to monitor them, there is a pressing need for the renewal of a broader and more holistic framework (e.g., Education for all) which encompasses the education sector and its constituencies. It would seem reasonable to build from this renewed vision in order to derive the key priorities for the smaller set of measures required for monitoring development goals. Second, efforts should be driven by country needs – the measures should take into account their relevance for for monitoring national plans and priorities.

Thus, in addition to set of key global indicators, a broader roadmap which goes beyond collecting the most readily accessible data. It should set out an agenda to work with national partners to build demand for data use, improve data systems, strengthen technical expertise and to invest in the longer term methodological development for both national and cross-national standards and best practices. This would require some type of multi-stakeholder effort which would help to align and guide the activities and ambitions of a wide range of global, regional and national actors, which doesn’t currently exist.

Measuring equitable access to education

The coverage of data on education inequalities has improved, most notably through the use of standardized household surveys funded by UN agencies, multi-lateral and bi-lateral agencies, which capture data on school participation and attainment. Since the 1990s, some 300 DHS surveys were conducted in 90+ countries and 215 MICS surveys in 140+ countries. Using these survey data, it has been possible to identify children and youth who are out of school and their individual and family characteristics.  Efforts like the EFA Global Monitoring Report’s WIDE database report data widely and the UNICEF/UNESCO initiative on Out of School Children (OOSCI) which works with 25 countries some which have the highest numbers of out of school children to improve data quality and promote the use of these data to better understand the barriers to school access.  While basic measures of access to education from household surveys are increasingly reported, they are still relatively limited in scope and often perceived as outside of national processes and thus underutilized by policymakers.

Recognising the lack of a common language for discussing the issue of equity in education, there is the need to bring together some of the diverse approaches and to take a more systematic approach to conceptualising and measuring the equity of countries’ education systems. For example, if the goal is that all children have equal learning opportunities, there could be a number of indicators which could inform progress. But if the goal is that children facing disadvantage receive unequal (e.g, compensatory) learning opportunities, this would be difficult to monitor. Likewise is it the absolute gap in opportunity – all children should complete primary education and acquire minimum competencies of basic skills – or the relative gaps in opportunity – the differences between children should be minimized. Reaching consensus on an approach will place allow for the development of indicators to assess the equity of  education systems and to develop and implement policies that address the most critical equity issues.

In addition to this, there are still a number of technical issues which need to be better addressed, including sample designs that better capture disadvantaged populations, better measures of the ages of children, measures which go beyond participation and attainment and the need to focus not only on individual attributes in relation to access, but also maintain the focus on learning providers.

Measuring learning outcomes and education quality

Much progress has been evident in the greater recognition of the importance of learning and its measurement throughout the 1990s and 2000s leading increasing numbers of countries taking part in global assessments such as PIRLS, TIMSS and PISA, to new regional initiatives that monitoring reading and maths (MLA, SACMEQ, PASEC, TERCE), the doubling of the proportion of countries, globally, with national large-scale assessment systems.

Building on this progress, a wide range of education stakeholders are working collectively to define and measure progress toward learning goals at the global level and to inform the post-2015 global development policy discourse – and as importantly create a platform for further advancing this agenda. The Learning Metrics task force represents UN agencies, regional organizations, national governments, bilateral donors, and civil society organizations and accesses a global network of technical expertise in the evaluation of learning, and capacity to move learning forward on the global development agenda. UNESCO, through its Institute for Statistics, and the Center for Universal Education at the Brookings Institute forms the Secretariat.

Based on recommendations from technical working groups and input from broad global consultations, the task force has mobilized education stakeholders to raise the profile of learning and measuring outcomes. It has recommended a framework for seven key learning domains with numerous competencies, identified six areas for measuring learning opportunities and outcomes and considered the implementation of learning assessment.  These recommendations are designed for use globally, building and complementing efforts to measure learning that are already underway at national and regional levels. The next steps involve the articulation of a multi-stakeholder group to carry forward the recommendations and provide support to countries engaging in large-scale assessment.

Future directions

The data revolution is partly about the opportunities provided by new tools, but it’s not just about the technology. It’s about the use of the technology, the valuing of its results, and the use in policy formulation and assessment. This isn’t just about quick solutions but about long-term institutional change. The scope and ambition of such an effort to mobilise national governments and international stakeholders in the areas of equity and quality would need to be unique and unprecedented and will require a long-term plan and strategy which should go beyond an initial selection of global indicators. Addressing the information gap in these areas has been an on-going concern that has made gains, and it is clear that it will take considerable time and resources to further build on these gains. Thus the human and financial resource gaps in achieving these ambitions should not be underestimated.