Detailed Program of the Conference

Please select a date or location to show only sessions at that day or location. Please select a single session for a detailed view (with speakers and abstracts).

 
 
Overall view of the program
Session
Parallel session - F.4 Reinventing Decision-Making in Schools: Self-Evaluation & Data Literacy
Time:
Wednesday, 02/June/2021:
12:00pm - 2:15pm

Session Chair: Michela Freddano
Session Chair: Serafina Pastore
Location: Room 7

Session Panels:
F.4. Reinventing Decision-Making in Schools: Self-Evaluation & Data Literacy

External Resource:
Presentations

TEACHER TRAINING ON DATA FROM THE INVALSI LARGE-SCALE STUDY ON STUDENT ASSESSMENT: THE LIGURIAN CASE

Michela Freddano1, Valeria Pandolfini2

1INVALSI, Italy; 2University of Genoa

Large-scale studies on student assessment offer to decision-makers data that are useful to take decisions and to guide public policies at macro and micro level. The evaluation of the instructional system is the other face of school autonomy. As a fact, schools organize and take decisions on didactical strategies with the aim to rise the learning objectives specified into the national indication for the curriculum from the first to the eighth grade of schooling and into the guidelines for the secondary school (from the nine to the thirteen grade). At the same time, each school year, the legislator provides national testing at census level for assessing student learning at different grade of schooling. At the national level, data are useful to account the state and the conditions of the educational system in term of students’ learning quality, social inequalities and underline points of attention. The production of large-scale studies on student assessment by time offers theoretical material and data for schools that allow principals and teachers to take decisions for school improvement.

The paper shows the case of the teacher training on data from the standardized tests done by the Italian National Institute for the Evaluation of the Education System (INVALSI) promoted by a school network of the Liguria Region, in the North of Italy, with the support of the Liguria School Regional Office. That promotion replies to the needs expressed by schools and regional public educational authorities of better understanding and using INVALSI data to improve school educational and organizational strategies.

A formal school network had been constituted by the 10 Ligurian polo schools for teacher training with the objective to organizationally and economically sustain the development of teachers’ data literacy for improving the quality of teaching and learning processes.

Two research plans had been defined: the first one about training teachers on the use of INVALSI data for the improvement of the educational strategies, and the second one on the promotion of action-research for school evaluation and improvement.

The paper presents the development of the first plan on teacher training experience involving 674 Italian teachers working in primary and secondary schools in Liguria Region. The training course aim is to train teachers in analysis, interpretation and use of data emerging from the assessment of students’ learning on Reading, Mathematics and English language at different school levels.

The training course started in November 2019 with 5 in presence plenary sessions and gathering 521 teachers working in the different provinces of the Liguria Region. Due to the pandemic COVID-19, the specific training workshops addressed to teachers that had been foreseen in presence, took place at distance, thought online platforms, from November 2020 to January 2021. Two online surveys have been addressed to teachers attending the training course.

Results show the experience and focus on the importance to train teachers on data literacy and to encourage teacher professional development initiatives promoted by agents and public authorities at local levels in order to better meet the specific local needs.



ARE ITALIAN SCHOOL-PRINCIPALS AND TEACHERS DATA LITERATE? AN ANALYSIS OF DATA LITERACY BY INVALSI SCHOOL QUESTIONNAIRE AND TEACHER QUESTIONNAIRE

Paolo Barabanti

Università Cattolica del Sacro Cuore, Italy

For several years, Invalsi has being giving back to each school data about previous-year’s Invalsi test, in order to help schools to find out strong points and improvements, so that the school staff could be able to build organizational and didactic practices aimed to improve school efficacy and quality learning. Actually, large-scale assessments of student achievement provide a window into the broadly defined concepts of literacy and generate information about levels and types of student achievement in relation to some of the correlates of learning, such as student background, attitudes, perceptions, and school and home characteristics, thanks also to the addition of data collected with some integrative questionnaires (such as the ones for students, teachers, principals and parents).

Even if some researches have proved that, when teachers use data literacy to assess their students and adjust their lessons, they see an increase in student learning and understanding, several studies, however, showed that there is still too little attention to this data and one reason of this lack is due to the tough technical approach of these surveys which are considered, by headmasters and teachers, something only for statisticians: they indeed don’t have the confidence and/or training needed to successfully engage with and interpret data results.

That’s why in recent years policy makers and scholars are emphasizing the importance of data in education and promoting the goal of using data for continuous improvement as, since the new millennium, new and diverse sources and forms of data have proliferated within the education sector, with the help of new technologies, by which to manage, access, and analyze such data.

In light of these considerations, this research aims to explore school-principals’ and teachers’ data literacy, in particular the one about the ability to understand and read, analyze, share and communicate data and the ability to use it for both self-evaluation and decision making. Are there aspects, such as gender, seniority, specific teaching school subjects, that affect the attitude in the use of such data? Do teachers feel more inclined to accept and use the data returned by Invalsi to reflect on teaching and learning if their school manager considers this evidence useful? Are teachers and head-masters more aware of the potentialities of this data if they had a specific training about Invalsi tests? How could Invalsi Institute help to develop and improve these skills? To investigate these questions, data derived from the Integrative Surveys on School and Teacher that accompany Invalsi tests in school-year 2018/19 have been used.

The preliminary results highlight different attitudes and use of this data in relation to school grade, geographic area, specific training about Invalsi tests, seniority, and headmaster’s opinion about this survey. Moreover, there are also evidences about how this data could be instrumentalized – if there’s a poor knowledge of its strengths and weakness – and be used, for instance, in a high-stake way within the school dynamics and process.



ASSESSMENT LITERACY, DATA USE, AND TEACHER DECISION-MAKING. A PROFESSIONAL DEVELOPMENT ATTEMPT

Serafina Pastore

University of Bari A. Moro, Italy

Together with the spread of standardized assessment, there is a relevant growth of systems and technologies aimed to store, manage, analyse, and report data on student performance.

Despite the broad recognition of the advisability of using data on student learning for decision-making (at educational policy and practice levels) data use of instructional decision making is not widely recognized Few research studies, indeed, have tried to understand what schools and teachers do with data on student learning.

Given these assumptions, the present paper reports on an attempt realized in the Italian school system to implement an interim assessment. A light professional development path has been designed and implemented with the aim to improve teacher assessment literacy, data use, and instructional decision-making.

The teacher inquiry path, through the implementation of the interim assessment system, involved 37 primary school teachers (and 404 students).

The present paper reports the main aspects of this path and critically reflects on how to support teacher assessment literacy and data use.



PARENTS AS DECISION-MAKERS. FINANCIAL-ORGANIZATIONAL INVOLVEMENT OF FAMILIES IN THE SCHOOL OF AUTONOMY

Anna Granata, Valerio Ferrero

University of Turin, Italy

The involvement of families in school governance is a crucial aspect for the Italian school, especially after the reform of school autonomy. This research investigates the role of representatives of families in the governance of Italian schools in (1) managing the donations of families for the educational offer and (2) organizing school time.

Analyzing the legislation, parents are part of almost all of the collegiate bodies through their representatives and a parent presides over the Institute Council, a body with a deliberative function (D.Lgs. 97/1994). The presence of different temporalities is important especially in primary school (D.P.R. 89/2009). The donations are voluntary contributions that families can pay to schools for the expansion of the educational offer (MIUR Notes 312/2012 and 593/2013), informed on their destination, according to PTOF.

A mixed method was used. The compilation of a questionnaire involved 300 school principals from Piedmont and Lombardy; in-depth interviews were then carried out with seven principals and six representatives of families (non-probabilistic sample for reasoned quotas) from the same regions.

The data collected show schools in which heterogeneity is a norm and represents a challenge in terms of responding to the needs of pupils and families. As regards school time, it emerges that the strong presence of 40-hour classes depends on the needs of families to support their family organization. On the other hand, parents’ economic contribution is a necessity for schools, since without this sum a large part of the educational offer would not be realized and there would not be the necessary funds even for ordinary school expenses. This contribution is voluntary, but there are schools or parents’ representatives who present it almost as mandatory: it is an issue that generates a clash between school and family regarding the destination of funds and the development of strategies capable of guarantee equity.

The research reveals a marked role of parents in school governance. Economic intervention of families and activation of long school times represent a compensation mechanism for the lack of educational and social policies. The close link between the possibility of families to contribute economically and the quality of the educational offer is configured as a non-traditional factor of school inequality. The role of principals remains crucial in the school of autonomy: they can develop alternative fundraising strategies to overcome starting inequalities.

References

Ferrera, M. (1996), «The “Southern model” of Welfare in Social Europe», Journal of European Social Policy, 6, (1), 17-37.

Gavosto, A. (2013), «Più autonomia ma senza segregazione», Scuola democratica, 2, 521-526.

González, R. L. & Jackson, C. L. (2013), «Engaging with parents: the relationship between school engagement efforts, social class, and learning», School Effectiveness and School Improvement, 24, (3), 316-335.

Mincu, M. E. (2015), «The Italian middle school in a deregulation era: Italian modernity through path-dependency and global models», Comparative Education, 51, (3), 446-462.

Rowe, E. e Perry, L. B. (2020) Inequalities in the private funding of public schools: parent financial contributions and school socioeconomic status, Journal of Educational Administration and History, 52, (1), 42-59.



THE USE OF DATA TO SUPPORT STRATEGIC PLANNING AND SCHOOL IMPROVEMENT AND INNOVATION

Serena Greco, Elettra Morini

Istituto Nazionale di Documentazione, Innovazione e Ricerca Educativa-INDIRE), Italia

The theme of school improvement and innovation is central in national and international debate and regards crucial issues for the evolution of educational systems: the ability to respond to social change, to meet the challenges of innovation, to stay abreast in a globalized and interconnected context. In this scenario, schools must characterize themselves as learning organizations, i.e. as organizations that learn and are capable of profound changes in roles and functions, but above all at a cultural level (Kolls&Stoll, 2016).

Schools as learning organizations develop processes, strategies, and structures that enable them to learn and respond effectively in uncertain and dynamic environments. We can consider, for example, how important it was to govern the unexpected (Weick, 2010) during the Covid-19 pandemic. To use and review existing knowledge making effective use of data by the school community has become central to improvement processes.

The framework for the evaluation of innovation processes is an evaluation model elaborated by INDIRE, aimed at investigating the experiences of schools that have activated innovation processes, to understand how these processes change the organization of the school and what impact they have on the main actors of the school. The model has been the framework for the research “Evaluation of innovation processes” conducted by INDIRE and involving 52 secondary school participating in “Avanguardie Educative”. This model considers the school as a learning environment and works on the fundamental areas of innovation also identified by the theoretical framework Innovative Learning Environments (OECD, 2013). The framework is organized into 4 macro-dimensions and related sub-dimensions: 1) Propensity for change; 2) Transformation of teaching and learning practices; 3) Leadership and organizational development; 4) Openness and interrelationship with the outside world.

We intend here to explore the first dimension, aimed at investigating whether and how the sample schools implement strategic planning and use of data for improvement. In particular, the first sub-dimension of “Propensity for change” regards the use of data for improvement and strategic planning of the educational organization (PTOF, RAV, PdM and RS). This sub-dimension aims to investigate if and how the monitoring of improvement actions is carried out by the school; who manages the process and who are the actors involved; how the school identifies improvement goals and from what information (e.g. data provided by international surveys, INVALSI, internal evaluations); the overall level of strategic planning of the school.

Indeed, data do not represent knowledge in itself, it requires a sharing process for processing in the school context. Only through sharing, exchange, and collaboration can be developed relevant knowledge and readiness for change. School staff must be trained to analyse and use data effectively, to improve and transform existing practices.

References:

Kools, M. and L. Stoll (2016), "What Makes a School a Learning Organisation?", OECD Education Working Papers, No. 137, OECD Publishing, Paris, https://doi.org/10.1787/5jlwm62b3bvh-en

OECD (2013). Innovative Learning Environments, Educational Research and Innovation. OECD Publishing. http://dx.doi.org/10.1787/9789264203488-en

Weick, K., Sutcliffe, K. (2010). Governare l’inatteso. Organizzazioni capaci di affrontare le crisi con successo. Milano: Raffaello Cortina Editore.



USEFUL SELF-ASSESSMENT

Daniela Robasto

Università degli Studi di Torino, Italia

When the School is called upon to reinvent itself, it is also necessary to reinvent the decision-making processes within the institution. How are improvement decisions made? Using what data? Self-assessment processes risk still having a marginal role in the improvement process and being experienced by many with a compliance value. The goal is to overcome the gap between the narrated school and the lived school and this goal is achievable if we focus on some critical issues that characterized the first three years of application of DPR 80/2013 (2015-2018). This contribution illustrates some critical issues that have emerged within an empirical research, conducted by the authoress, on RAVs and PDMs. We will question what are the essential elements of methodological accompaniment so that the teaching staff can effectively participate in the drafting of a quality PDM, which can be monitored over time, pursued and consistent with the intervention priorities identified with the Self-Assessment Report. An implicit will be addressed that too frequently remains latent: the improvement of an institution necessarily passes through change and (quality) change should necessarily pass through learning. The contribution therefore presents some proposals relating to training plans and learning elements that should be favored in the evaluation team (NVI but not only) in order to be able to tackle the challenge of useful self-evaluation with mastery.