Vol. 22, n. 3, settembre 2023
RICERCHE, PROPOSTE E METODI
Valutare la qualità dell’inclusione scolastica: sviluppo e validazione dell’Ecological Assessment Scale for Inclusion (EASI)
Annalisa Morganti,1 Elisa Delvecchio,1 Francesco Marsili,1 Maria Mirandit,1 Alessia Signorelli1
Sommario
Il contributo presenta il percorso di costruzione e validazione italiana di una scala europea di misurazione della qualità dell’inclusione scolastica denominata Ecological Assessment Scale for Inclusion (EASI). Tale strumento nasce all’interno del progetto finanziato dal programma Europeo Erasmus+ KA3 — Supporto for policy reforms — chiamato Algorithm for New Ecological Approaches to Inclusion «ECO-IN» (N. Id 612163-EPP-1-2019-1-IT-EPPKA3-IPI-SOC-IN). La scala EASI è uno strumento europeo, in primis perché è stato costruito e sperimentato in quattro nazioni europee: Italia, Romania, Lituania e Spagna. Poi perché mira ad avere un utilizzo e una diffusione in tutto il continente, consentendo a ciascun contesto educativo di valutare e poi introdurre dei miglioramenti all’interno del proprio sistema scolastico inclusivo. Il percorso di costruzione dello strumento è stato svolto attraverso una survey in cui si è riproposto uno strumento elaborato in un precedente progetto europeo, già pubblicato e denominato Inclusive Processes Assessment Scale (Cottini et al., 2016). Successivamente si sono svolti dei focus group con un’ampia pletora di attori della comunità educante: dirigenti, policy-maker, insegnanti, famiglie, studenti delle scuole secondarie, psicologi e educatori. Dai risultati della survey e dei focus group sono stati elaborati gli indicatori che costituiscono la scala EASI. In questo studio vengono riportati solo i dati della validazione provenienti dal contesto italiano.
Parole chiave
Educazione inclusiva, Validazione, Qualità, Progetto europeo, Modello ecologico.
Research, proposals and methods
Evaluating the Quality of School Inclusion: Development and Validation of the Ecological Assessment Scale for Inclusion (EASI)2
Annalisa Morganti,3 Elisa Delvecchio,2 Francesco Marsili,2 Maria Mirandi2 e Alessia Signorelli2
Abstract
The paper presents the process of developing and validating a European scale for measuring the quality of school inclusion for the Italian context, called the Ecological Assessment Scale for Inclusion (EASI). This tool was created as part of a project funded by the European Erasmus+ programme KA3 - Support for policy reforms entitled Algorithm for New Ecological Approaches to Inclusion «ECO-IN» (N. Id 612163-EPP-1-2019-1-IT-EPPKA3-IPI-SOC-IN). The EASI scale is a European tool, firstly because it was constructed and tested in four European nations: Italy, Romania, Lithuania and Spain. Secondly, because it aims to be used and disseminated throughout the continent, allowing each educational context to evaluate and then introduce improvements within its inclusive school system. The tool development process was carried out through a survey in which a tool developed in a previous European project, already published and called the Inclusive Processes Assessment Scale (Cottini et al., 2016), was re-proposed. Subsequently, focus groups were held with a wide range of educational community players: managers, policymakers, teachers, families, secondary school students, psychologists and educators. The indicators that make up the EASI scale were developed from the results of the survey and focus groups. This study reports only the validation data from the Italian context.
Keywords
Inclusive education, Validation, Quality, European Project, Ecological model.
Introduction
The implementation of school inclusion is supported, at all levels, by the orientations expressed in European regulations, which include extensive documentation aimed at conveying cultural and ethical perspectives. Over the last decade, UNESCO and the European Agency for Inclusion have issued documents (EASNIE, 2017, 2018, 2019) aimed at directing national policies about inclusive education towards common horizons and proposing theoretical models, models of governance and financing, system objectives and practices. Moreover, numerous European countries have played an active role in formulating documents that aimed at realizing the goals of the inclusive education movement at an institutional level. Examples of such documents include the Salamanca Statement (UNESCO, 1994) and the Convention on the Rights of Persons with Disabilities (UN, 2006, 2016). The international regulatory and documentary landscape reaffirms the principles expressed by the Education For All (EFA) movement for equal and inclusive education for all students, which are also among the objectives of the UNDP’s 2030 Agenda for Sustainable Development (UNDP, 2015) adopted by the United Nations General Assembly (Morganti, 2018).
Inclusive education is a multifaceted and complex process, built on several levels: the level of principles, the organizational level, the methodological-didactic level, and the level of empirical evidence (Cottini, 2017). Despite being advocates and supporters of ambitious cultural transformations, international organizations and agencies sometimes underestimate the uncertainties and difficulties that arise at all of these aforementioned levels. The situation of school inclusion presents controversial elements, vulnerabilities, and inefficiencies. Firstly, the general theoretical framework is characterized, despite efforts to reach a unanimous consensus (Ainscow, 2016), by a dual conceptualization of inclusion: on the one hand, a longer-standing «narrow» conception of school inclusion, inclined towards caring for and managing vulnerable categories of the student population in addition to general education, thus dedicated to finding solutions for disabilities and special educational needs; on the other hand, a «broad» conception of inclusion, committed to promoting systemic and structural change, transcending special categories and embracing all inter-individual differences (Dell’Anna et al., 2023).
Furthermore, many systems that support school inclusion have inherited structures and competencies from the special education and pedagogy sector (Dell’Anna, 2021), adopting a medical-individual categorization model that places the pedagogical sphere in the background, in favour of an approach focused on psycho-physical dysfunctions and rehabilitation, which, as widely demonstrated by the cultural field of Disability Studies, poorly incorporate multidimensional and ecological approaches. To find a mediation context between special and inclusive education, the Commission on the Rights of Persons with Disabilities has proposed a holistic model consisting of four key indicators (UN, 2016):
- Availability: free and quality systems;
- Accessibility: systems accessible in terms of curriculum, teaching, evaluation and environmental factors;
- Acceptability: systems capable of supporting the needs, culture, and aspirations of people with disabilities;
- Adaptability: systems that respect the principles of Universal Design for Learning for the educational benefit of all students.
However, the issue remains open as such principles cannot be limited to individual interventions, but rather should be characterized by systematic and joint actions that involve a wide range of players and are guided by shared theoretical principles. In this perspective, several proposals have been put forward. Mitchell (2015), for instance, has proposed an ecological and multidimensional implementation model on inclusive education using ten indicators:
- Vision: institutions at all levels (national, regional, and local) promote inclusive education;
- Acceptance: the educational community recognizes and accepts special educational needs and diversity;
- Leadership: school leaders and governing bodies promote and coordinate inclusive actions;
- Resources: schools have materials and tools for inclusive education;
- Support: teachers and external experts (psychologists, educators, social workers, etc.) collaborate on systematic intervention proposals;
- Placement: all students have the right to attend school free of charge, in traditional classrooms;
- Access: buildings, materials, and tools are designed to be accessible to all students;
- Adapted curriculum: contents are common to all students, and adaptation follows the principles of Universal Design for Learning;
- Adapted teaching: strategies employed by teachers are evidence-based;
- Adapted assessment: evaluation is consistent with adaptations and any individualized goals.
Ainscow (2016), on the other hand, identifies possible paths that are coherent both in theory and application:
- Inclusive education, disability and special educational needs: inclusion is driven by a categorical perspective that pays particular attention to learning environments and accessibility;
- Inclusive education and «disciplinary exclusion»: in addition to special needs, it involves the reintegration of individuals who have been expelled from other educational contexts due to problematic behaviours;
- Inclusive education and vulnerable groups: the school becomes a space for social inclusion as efforts are made to create the most suitable environment to overcome social, ethnic and religious discrimination;
- Inclusive education and promotion of a school for all: it involves a system that does not differentiate students based on academic performance but accepts all differences;
- Inclusive education and education for all: it is the broadest and most ambitious concept of inclusive education that not only accepts differences but values them.
Ainscow (2016) observes that despite being the most desirable approach, towards which even international institutions have directed their compasses, the inclusive education that values all differences has not been fully realized yet, as the current system is still based on special categories.
As we have noted in previous works (Marsili, Sisti & Morganti, 2021a), differences in approaches are reflected in the literature at both terminological and research levels, polarizing the debate and influencing normative approaches and practical directions. At the national level, if we consider the Italian situation, this heterogeneity is found in the system and manifests itself both within the scientific community, with opposing but dialoguing views (see the entire 2021 issue 1 of the «Italian Journal of Special Education for Inclusion»), and in the comparison between implementation and regulations, as well as between the latter and the theoretical approaches of Pedagogy for Inclusion (Ianes, Demo & Dell’Anna, 2020; Marsili, Morganti & Signorelli, 2021b). Dell’Anna and colleagues (2023) stated that the Italian inclusion system has been in a phase of multi-level opposition for some time, where action clashes with the declared (inefficiencies) and the latter clashes with the necessary perspectives for achieving a genuine inclusive education system, in the ecological logic of the bio-psycho-social paradigm (paradigm immutability). In particular, the author highlights that the system presents distortions: a constant increase in certifications, the cultural dominance of the medical-individual model, quantity rather than quality of resources, the «different» status of the support teacher, poorly designed and poorly utilized structures, and standardized and transmissive teaching practices. Consequently, some scholars’ prevailing attitude, despite such evidence, is one of scepticism towards inclusive education (Ianes & Augello, 2019; Imray & Colley, 2017), which is vulnerable, especially because it lacks the necessary monitoring and evaluation tools to bring about systemic improvements (Ianes & Dell’Anna, 2020; Dell’Anna, 2021; Marsili, Sisti & Morganti, 2021a; Dell’Anna, Bellacicco & Ianes, 2023).
The issue of evaluating the quality of school inclusion is a key to silencing any scepticism regarding inclusion and to introducing mechanisms for improvement that can provide evidence of the efficiency and effectiveness of a fully inclusive model (Ianes, 2023). Despite the promotion of the question of evaluating school inclusion in recent Italian legislation (D.lgs. 66/2017 and 96/2019) and the Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione (The National Institute for the Evaluation of the Education and Training System) or INVALSI being tasked with managing such evaluation, schools are currently only required to provide an Annual Plan for Inclusivity (nota 2563, MIUR, 2013), a document in which they self-declare projects and practices related to inclusion processes. Therefore, schools are not equipped with tools that allow for a multidimensional evaluation of the inclusion process, involving other players in the educational community, which can assess the outcomes of inclusion and the performance of teaching and administrative staff as a contribution to the inclusive process.
This issue appears to be present at an international level, resulting in a substantial incomparability among implementation models of inclusion (Wolff et al., 2021; EASNIE, 2020). Despite conceptual, structural, valuative, and political barriers hindering comparability (D’Alessio & Cowan, 2013), the lack of reliable and dynamic tools also weighs in, preventing each system from analysing the quality of the inclusion process in their national context and submitting it to a genuine and systematic comparison with other countries. Currently, the literature mostly presents theoretical analysis models (for a review, see Dell’Anna, 2021), each considering different dimensions and variables of school inclusion. Dell’Anna (Dell’Anna, Bellacicco & Ianes, 2023) has summarized these dimensions and variables into a single model:
- Procedural or horizontal dimension, which inherits the context, resources, process, and result/effect factor analysis approach based on input-process-output logic;
- Contextual or vertical dimension, following the multilevel logic of the Brofenbrenner model (1979);
- Temporal dimension, which examines short, medium, and long-term effects over time.
However, despite the proliferation of such analysis models in the literature, there are very few attempts to build tools capable of supporting a systematic ecological evaluation, able to provide a composite interpretation of an inclusive school system (Marsili, Sisti & Morganti, 2021). One attempt that has been made comes from the German context with the Quality Scale of Inclusive School Development-short form (Schurig et al., 2020), which, again from an ecological perspective, allows us to investigate inclusive education on five distinct levels: the level of individual student needs, Inclusive Instruction, Multiprofessional Cooperation, Inclusive School Concept and School Life, and External Support and Communal Networking.
In light of the above, this contribution presents the process of developing and validating the Ecological Assessment Scale for Inclusion (EASI) for the Italian educational context. This tool was created as the main objective and product of the ECO-IN Algorithm for New Ecological Approaches to Inclusion project (N. Id 612163-EPP-1-2019-1-IT-EPPKA3-IPI-SOC-IN), funded through the Erasmus+ KA3 programme on educational policies and coordinated by the University of Perugia. In this context of structural funding, the experimental and validated evaluation tool is based on the theoretical principles of Bronfenbrenner’s ecological model (1979), Mitchell’s ecological-systemic inclusion model (2018), and the input-process-output model which analysed inclusion at three stages and was proposed by the European Agency for Inclusion (Kyriazopoulou & Weber, 2009) and other authors (Loreman, Forlin & Sharma, 2014; Kinsella, 2018). The EASI tool is therefore based on a dual approach to school inclusion: vertically, inclusion is understood as a complex system of relationships that occur between the child/young person and a network of individuals and contexts, from the closest to the farthest, such as family, school, community, and society in general (see Figure 1); horizontally, considering the relationships of the input-process-output model, where structural aspects intersect with interventions and resulting changes. The heart of the ECO-IN project and the main objective of the EASI tool is to investigate the value, practical and design profile of the main players of the educational community at institutional, managerial, educational, family, and professional levels. The players involved in this process of developing and validating the tool include policymakers, teachers, family members, school heads, educators, students and psychologists. Only from such a complex set of participants is it believed that an authentic profile of the real situation of an inclusive educational system can emerge, to highlight its potentials, capabilities and weaknesses, with the invariable aim of improving progress.
Method
Participants and procedures
The present study aimed to investigate the presence and importance regarding the quality of school inclusion in a sample of 434 Italian family members and teachers (50% family members; 50% teachers). Specifically, the participants were administered a general scale consisting of 10 ad hoc items regarding presence and 10 regarding importance.
Measures
In order to create the assessment tool, according to the project’s design, two measures were implemented in the project’s early life cycle: an online survey and a focus group. Considering the complexity of the framework investigated, both qualitative and quantitative studies were used (Cottini & Morganti, 2015) for the creation and validation of the tools, and are described in this contribution.
The crossing and matching of the qualitative data analysis of both provided the basis for the identification of the 10 common items of the ECO IN project assessment scale.
Creation and administration of the online survey
The online survey was the joint work of the Italian universities taking part in the project: the University of Perugia and the University of Urbino. The creation of the online survey was grounded on the results obtained through the administering of the Inclusive Processes Assessment Scale (Cottini et al., 2016), which highlighted a series of weaknesses concerning poor inclusive policies; poor accessibility to school info; poor involvement of the families in school’ initiatives; poor interaction between the different stakeholders involved in inclusive education; poor collaboration between the schools and different key players, such as external experts and counsellors (Zorc-Maver, Morganti & Vogrinc, 2019). Further analysis of these results leads to the outlining of three main dimensions that were also implemented in the analysis of the focus group interviews:
Relations between school and local area: this dimension focused in particular on two aspects: the collaborations already existing between the schools and local agencies and how to forge new ones;
School organization and their local area: this dimension explored both the strengths and weaknesses of aspects concerning the sharing of documentation and knowledge on inclusive practices between the schools and their local communities and how to establish and develop a common vision for improving inclusion;
Training, updating, and inclusive cultures: this last dimension dealt with several crucial aspects such as the promotion and implementation of extensive training on the topics concerning inclusive education aimed not only at teachers and school heads but also at families, caregivers, parents, and other key figures such as educators, policymakers and other educational experts.
Plus, a number of items which were narrowed down to 8 for each of these three dimensions.
The respondents were asked to rank the items from 1 (the statement(s) they believed to be least relevant) to 8 (the statement(s) they believed to be the most relevant of all). Respondents were therefore asked to identify the elements that, above others, can contribute to inclusive education, namely in the creation of educational contexts (classes and schools) capable of seeing individual differences as an enrichment and removing barriers to learning, participation and achievement of all (figure 1).
Figure 1
|
Relationship between school and local area: general aspects |
||||||||
|
To assess the quality of school inclusion, how relevant are the following to you: |
Please rank this statement from 1 (is the least relevant) to 8 (is the most relevant) |
|||||||
|
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
|
|
That the school would organize moments of public debate and public training on the topic of inclusion that would see the involvement of families also |
||||||||
|
That the school would be aware of external projects aimed at promoting inclusion and would use the resources available in the local area (families, associations, social private institutions, not-for-profit sector) to improve the teaching practices and the inclusive perspective. |
||||||||
|
That activities in horizontal continuity (families, associations, local area) and vertical continuity between the different school years and between the classes of the same school) would take into consideration the special needs of some learners |
||||||||
|
That the school would promote inclusive projects concerned with the local area (sustainable mobility for learners, youth centres, accessibility, inclusive basketball (baskin) tournaments, reading aloud projects...) |
||||||||
|
That the whole documentation related to the inclusive process is available to all subjects (families, associations, etc.) and that the info provided would be made accessible to all, even using different modalities braille, texts translated into different languages, technological devices. etc.) |
||||||||
|
That local communities are involved in the school activities and vice-versa, also through the opening of the school spaces to initiatives aimed at the local area. |
||||||||
|
To involve not only the schools, but the whole community in the debate about the improvement of inclusive education |
||||||||
|
That the counselling activities would take into consideration the special needs of some learners |
||||||||
Online Survey Common indicators for the first dimension: Relationship between school and local area
The survey was circulated between the months of February and March 2021. All primary and secondary schools from Umbria and Le Marche were sent an invitation to answer, on a voluntary basis, the online survey. Schools were instructed to send out the link to the online survey to the highest number possible of respondents from the categories listed in Table 1.
Table 1
|
04/02/21 |
11/02/21 |
18/02/21 |
26/02/21 |
05/03/21 |
12/03/21 |
|
|
teachers |
69 |
155 |
239 |
310 |
321 |
322 |
|
school heads |
8 |
19 |
21 |
21 |
21 |
21 |
|
policy makers |
3 |
3 |
3 |
3 |
3 |
3 |
|
families |
1 |
3 |
3 |
3 |
3 |
3 |
|
students |
2 |
2 |
2 |
2 |
5 |
5 |
|
educators |
3 |
12 |
13 |
14 |
16 |
16 |
|
psychologists/others |
5 |
6 |
6 |
11 |
14 |
15 |
|
92 |
200 |
287 |
364 |
383 |
385 |
Number of respondents — Survey, Italian version.
Focus Group
Parallel to the circulation of the online survey in the month of January 2021, two focus groups, one in Umbria and the other one in the Marche Region were set up.
The qualitative approach of the focus group is widely used in social research. Focus groups are considered quite useful when it is necessary to identify or understand deeper and more complex underlying issues (Powell & Single, 1996; Ryan et al., 2014; Cottini & Morganti, 2015; Sullivan & Forrester, 2019) that cannot be investigated through more quantitative methods.
The use of focus groups allows researchers to explore phenomena or specific issues (Tong, Sainsbury & Craig, 2007) in what Powell & Single (1996) call a more «naturalistic setting» (p. 499), using semi-structured interviews carried out with a sample consisting of 4 to 12 people (Tong, Sainsbury & Craig, 2007). However, sampling in focus groups is not set out to be fully representative of a population but aims at recruiting a group of people who can relate to the conceptual framework of the study and can generate a discussion that will extend the way an issue is thought about (Macnaghten & Myers, 2004). This is what Glaser and Strauss call a «theoretical sample» (Macnaghten & Myers, 2004). The interview techniques implemented in the focus group are grounded on the idea that it is the interaction between the participants that fosters a deeper exploration of the issues at hand together with a sharing and clarification of individual perspectives (Tong et al., 2007).
Participants in the focus groups were selected upon invitation and were representative of schools with high rates of migrant, refugee, special needs (disabilities included) and lower socio — economic status students. The final sample consisted of 11 participants in both focus groups representing school heads, teachers, families/parents, educational experts, policymakers and students.
The focus group interviews were conducted with the aim of further investigating the three main areas to better understand the underlying dynamics of inclusive ecological processes and approaches that were impossible to gather from the quantitative data collected through the online survey.
The interviews were analysed according to the following three criteria:
- Analysis of the participants’ interventions based on frequency, extension, and specificity.
- Analysis of the main contents gathered from the diverse experiences.
- Relational analysis of the behavioural dynamics, of the internal relationships between the groups, and moderating and leadership styles.
Following this first part, the abstract concepts highlighted in the focus groups were turned into indicators. Firstly, qualitative analysis allowed for the identification of two key features: the main abstract concept and its relative dimension(s). Then, we proceeded to operationalize both the abstract concepts and their dimension(s) and, therefore, we created indicators, which are their «practical» representations in terms of actions, planning and intentions.
Importance and Presence
The last action was crossing and matching the quantitative and qualitative results obtained from the survey and the focus groups to identify and select the 20 items that would compose the general part of the Ecological Assessment Scale for Inclusion (EASI), divided into 10 items for importance and 10 for presence.
Initial work was done on the survey. For each of the three dimensions (relations between the schools and their local area; school organization and their local area; training, updating, and inclusive cultures) the 5 items that received the most significant occurrence of high scores (between 7 and 8) were pulled out from each respondents’ category (teachers, managers, policymakers, families, students, educators, psychologists and other experts) for a total of 40 items.
Items that explored similar concepts were grouped to reach a maximum number of 15 items, then the research team performed a further narrowing down to remove those items which were deemed too «technical» and those that required a specific knowledge that would not be common amongst the different types of respondents, and to make every item clear and understandable and «readable» to all prospective respondents.
Data analysis of the survey and focus group interviews also highlighted two main factors through which the discourse of inclusive education was expressed, mainly in the vertical and horizontal dimensions (Dell’Anna, 2021), which have been interpreted in the EASI as the factors called presence and importance.
These two factors provide a crucial key to the interpretation of what is needed to work on to improve inclusive educational processes and, within the EASI scale, to represent the conceptual «weights» that would ideally steer future decisions on how to enhance the quality of school inclusion.
The 20 general items that represent the core of the EASI scale were therefore worded in terms of importance and presence (Figure 2); in this way the scale analysis would not only show how important an item is deemed, on a scale from 1 to 5, thus collecting information on the visions of the various respondents, but also perform important tracking to understand if the factor of importance corresponds to that of presence.
Figure 2
|
ID |
importance |
presence |
|
|
gen |
1 |
It is important to establish relations with public bodies and agencies in the local area involved in the inclusion process |
Relations with public bodies and agencies in the local area involved in the inclusion process are established |
|
gen |
2 |
It is important to establish relations with public and private bodies in the local area involved in the promotion of socialization in their community |
Relations with public and private bodies in the local area involved in the promotion of socialization in ther community are estabished |
|
gen |
3 |
It is important to plan continuity actions between the different players in order to support the transition from one school year to another |
Continuity actions between the different players in order to support the transition from one school year to another |
|
gen |
4 |
It is important to establish and define orientation pathways (academic, social, work related, etc.) in order to support «aspiration and self-determination abiities |
Orientation pathways (academic, social, work related, etc.) in order to support «aspiration» and self-determination ablities are established and defined |
|
gen |
5 |
It is important to carry out meetings in spaces within the school, to share and discuss topics relating to inclusion between all the stakeholders involved |
Meetings in spaces within the school, to share and discuss topics relating to inclusion between all the stakeholders involved are carred out |
|
gen |
6 |
It is important to realize a plan of the educational offer that sees the organization of the curriculum/programme and assessment practices from an |
The plan of the educational offer that sees the organization of the curriculum/programme and assessment practices from an inclusive perspective is carried out |
|
gen |
7 |
ilt is important to monitor the level of inclusion of the school through assessment and self assessment tools, sharing the outcomes with all |
The level of inclusion of the school through assessment and self-assessment tools, sharing the outcomes with all stakeholders involved is monitored |
|
gen |
8 |
It is important to promote training on inclusion, engaging all inolved players (teachers, educators, school heads, families, etc.) |
Training on inclusion, engaging all inolved players (teachers, educators, school heads, famiies, etc.) is promoted |
|
gen |
9 |
It is important to promote continuous and systematic exchanges between schools, universities and research centres, in order to carry out active research pathways |
Continuous and systematic exchanges between schools, universities and research centres, in order to carry out active research pathways based on scientific results, are promoted |
|
gen |
10 |
It is important to feel actively involved in the educational inclusion process |
You feel actively involved in the educational inclusion process |
Items for pilot version of EASI - General part.
Analysis
Statistical analyses were performed using the Statistical Package for Social Science (IBM SPSS Version 22) and R (Version 4.1.2; lavaan - Rosseel, 2012 and semTools packages — Jorgensen et al., 2018). All analyses were performed in the absence of missing data. To assess the construct validity of the proposed questionnaire (GEN), an exploratory factor analysis (EFA) and a confirmatory factor analysis (CFA) were used. The twenty items of the GEN scale were subjected to Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test for data adequacy. According to the academic requirement, a KMO value of over 0.8 indicates the adequacy of sample data for factor analysis. The p-value of the Bartlett X² test was <0.001, indicating that the sample data were correlated and concentrated, and suitable for factor analysis. GEN confirmatory factor analysis (CFA) was performed to verify the factorial validity using the robust diagonally weighted least squares (DWLS), as the observed variables were ordinal. Values of the root-mean-square error of approximation (RMSEA; Marsh et al., 2004) and standardized root-mean-square residual (SRMR; Hu & Bentler, 1999) ≤ 0.08, comparative fit index (CFI) and Tucker-Lewis index (TLI) ≥ 0.90 (Bentler, 1990) indicated the model fit was good. Cronbach’s alpha values ≥ .70 indicated acceptable reliability (Taber, 2018). Multi-group CFAs (MCFAs) were performed to assess measurement invariance between roles (family members and teachers). At first, configural invariance (equivalence of factorial structure) was performed, then the metric invariance and scalar invariance (equivalence of factor loadings) were assessed. Measurement invariance was established when the associated ΔCFI values were less than 0.010, the value of ΔRMSEA was less than 0.015 and the value of ΔSRMR was less than 0.030 for the metric invariance or less than 0.001 for the scalar invariance (Chen, 2007). Meade et al. (2008) suggest different cut-offs for ΔCFI < 0.018 for the scalar invariance. The independent-sample t-test was used to examine differences in scores between teachers and family members on the GEN scale.
Results
Exploratory and Confirmatory Factor Analysis
EFA conducted on two-thirds of the sample identified a two-factor structure (Table 2) explaining 50% of the variance. The KMO test demonstrated sample adequacy (Table 2), while Bartlett’s test (Χ²= 3522.931; p < .001) and the moderate correlations (r= .39) between factors indicate the absence of multicollinearity and singularity.
CFA shows adequate results for the two-factor model was confirmed (χ2(190) =9377.864, p < .001, CFI = 1.00, TLI = 1.00, RMSEA = 0.00 (90% CI = [.00, .01]), SRMR = 0.05; see Figure 3). All items were loaded in the expected directions and were significant at p < .05. They displayed loadings ranging from .64 to .81.
Table 2
|
Factor Loadings |
|||
|
Factor 1 |
Factor 2 |
KMO |
|
|
importance_1 |
0.64 |
0.85 |
|
|
importance_2 |
0.72 |
0.87 |
|
|
importance_3 |
0.61 |
0.91 |
|
|
importance_4 |
0.62 |
0.90 |
|
|
importance_5 |
0.61 |
0.92 |
|
|
importance_6 |
0.78 |
0.89 |
|
|
importance_7 |
0.69 |
0.91 |
|
|
importance_8 |
0.70 |
0.90 |
|
|
importance_9 |
0.71 |
0.91 |
|
|
importance_10 |
0.63 |
0.89 |
|
|
presence_1 |
0.72 |
0.92 |
|
|
presence_2 |
0.76 |
0.90 |
|
|
presence_3 |
0.74 |
0.93 |
|
|
presence_4 |
0.76 |
0.93 |
|
|
presence_5 |
0.79 |
0.92 |
|
|
presence_6 |
0.75 |
0.93 |
|
|
presence_7 |
0.76 |
.92 |
|
|
presence_8 |
0.74 |
.93 |
|
|
presence_9 |
0.67 |
.91 |
|
|
presence_10 |
0.62 |
.93 |
|
Questionnaire items and corresponding factor loadings from the EFA rotated pattern matrix and KMO-values.
Figure 3
Standardized Factor Loadings and Factor Correlations in GEN scale.
Reliability and measurement invariance of family members and teachers
Cronbach’s alpha for both dimensions were excellent (α = .90, 92). The MCFA results confirmed the configural, metric, and scalar invariance across roles, indicating the same factorial structure and factor loadings for both teachers and family members. Analysis of the t-test for independent samples revealed no statistically significant differences in the scale of importance and presence between teachers and family members (table 3).
Table 3
|
χ2 |
df |
Δχ2 |
CFI |
RMSEA |
ΔRMSEA |
SRMR |
ΔSRMR |
TLI |
||
|
GEN |
||||||||||
|
Configural |
1.003.252 |
338 |
.873 |
.095 |
.055 |
|||||
|
Metric |
1.039.157 |
356 |
35.905 |
.870 |
-0.003 |
.094 |
-0.001 |
.065 |
0.011 |
0.004 |
|
Scalar |
1.139.064 |
374 |
99.907 |
.854 |
-0.016 |
.097 |
0.003 |
.069 |
0.004 |
-0.009 |
Measurement Invariance of GEN across role.
Discussion
Inclusive education is profoundly impacted by the extent to which systems enable their primary stakeholders to act inclusively, through cross-ministerial and cross-sectoral collaboration, and by providing diverse forms of improvement and support (Soriano, 2017). Some studies (Peeters et al., 2018) highlight inclusion as a process in which successful outcomes hinge on the teamwork of all essential players (schools, communities, and policymakers), who must receive adequate training to ensure inclusion both within and outside the school and to prevent any type of segregation, discrimination or radicalization. Boosting and supporting an ecological model of inclusive education is essential due to the difficulties in ensuring the same level of expected training and competencies across systems. Without an adequate control system, there is a risk of wasting economic and human resources. Teacher training, particularly initial teacher training, plays a crucial role in supporting these processes. School heads also have a pivotal role in developing sound competencies on inclusive topics and investing in the creation of inclusive policies, spaces, maintaining contacts with communities, and ensuring families’ active engagement. The more inclusive teachers and educational leaders’ competencies, knowledge, and attitudes are, the more positive their impact will be on students and on building an inclusive context. However, only 15% of families are involved in inclusive processes, and different cultural contexts can affect their opinions and attitudes, emphasizing the importance of training within and outside school in non-formal learning environments.
Based on the ecological model, the success or failure of an inclusive education and social system relies on the involvement and responsibility of each different system. The ECO-IN project partners chose this model as the theoretical foundation to evaluate the compliance of different players involved in bettering whole school inclusion across four partner countries. In this study we validate the Ecological Assessment Scale for Inclusion in the Italian educational context. In regard to validation, it pertains solely to the sample of teachers and family members who were identified as the two largest groups through analysis. As for the results of the exploratory analysis, it explains precisely the two structural factors of the scale: presence and importance. This, along with the correlation between factors (r= .39), leads us to believe that the scale also has a strong structural correlation. Therefore, the general scale (GEN) should always be proposed in combination and never separately. By using these two categories, researchers can achieve an interesting insight not only into what is already working and aligned in terms of importance and presence, but from an improvement perspective it is possible to infer, in case of a discrepancy between what is deemed as important and its effective presence, where actions can be implemented to make the importance a reality, thus a presence. While we are aware that the category of importance is strictly connected to and influenced by the person’s own set of beliefs, experiences and vision of the world and of the idea of what «inclusion» means, the category of presence is, on the other hand, a more concrete, factual one, which can provide a general picture of what is already existing or lacking in terms of resources, spaces and opportunities connected to the concept of quality of inclusion. However, the matching (or mismatching) of these factors can be essential to assess inclusion through an ecological lens. These two categories are, in other words, the trajectories through which the project aims to assess, from an ecological standpoint, the quality of inclusion, and to plan and propose, where necessary, remedial actions.
As for the items, the results (α=.91) indicate that the scale is highly reliable and capable of measuring the same factors regardless of whether family members or teachers respond. Although the items are limited in number, they have the potential to shed light on crucial elements of school inclusion through two different perspectives: the actual presence in the context and the importance this aspect holds from the subject’s formative and value-based profile.
The EASI scale, developed within the ECO-IN project, is a European scale not only because it was created in educational contexts in Lithuania, Romania, Spain, and Italy, but also because we aspire to its dissemination at a continental level. To ensure its highest circulation, the EASI scale is designed to be a free digital tool that will allow schools to access in a user-friendly way both the form to fill in and visualization of the final results. Our aim is twofold: to influence European policies and to instil assessment and improvement methods for the effectiveness and efficiency of national educational systems. Therefore, we hope that the EASI scale will be validated in other European countries. Moreover, it is crucial to validate both the general scale for important subjects in the educational community such as school heads, psychologists, students, and policymakers, and to verify the validity of specific scales, which have been created to deepen the perspective of each member of the educational community and shape a multi-faceted and deep assessment. The EASI scale also provides sound and up-to-date evidence of the results and impact produced to inform policy-makers about strengths and weaknesses, in order to formulate appropriate policies for the improvement of different school systems. The dynamic and innovative evaluation system will allow us to map educational and training needs in Europe, through a new ecologic perspective of educational and social inclusion.
References
Ainscow M. (2016), Diversity and equity: A global education challenge, «New Zealand Journal of Educational Studies», vol. 51, pp. 143-155, https://doi.org/10.1007/s40841-016-0056-x.
Bentler P.M. (1990), Comparative fit indexes in structural models, «Psychological Bulletin», vol. 107, n. 2, pp. 238-246, https://doi.org/10.1037/0033-2909.107.2.238.
Bronfennbrenner U. (1979), The ecology of human development: experiments by nature and design, Cambridge, Harvard University Press.
Chen F.F. (2007), Sensitivity of goodness of fit indexes to lack of measurement invariance, «Structural Equation Modeling: A Multidisciplinary Journal», vol. 14, n. 3, pp. 464-504, https://doi.org/10.1080/10705510701301834.
Cottini L. (2017), Didattica speciale e inclusione scolastica, Rome, Carocci.
Cottini L. & Morganti A. (2015), Evidence based education e pedagogia speciale, Rome, Carocci.
Cottini L., Fedeli D., Morganti A., Pascoletti S., Signorelli A., Zanon F. & Zoletto D. (2016). A scale for assessing Italian schools and classes inclusiveness, «Form@re - Open Journal Per La Formazione in Rete», 16(2), pp. 65–87. https://doi.org/10.13128/formare-18512.
D’Alessio S., & Cowan S. (2013), Cross-cultural approaches to the study of «inclusive» and «special needs» education, «Annual review of comparative and international education», Emerald Group Publishing Limited, Bingley, 20, pp. 227-261. https://doi.org/10.1108/S1479-3679(2013)0000020021
Dell’Anna S. (2021), Modelli di valutazione di un sistema scolastico inclusivo, Milan, FrancoAngeli.
Dell’Anna, S., Bellacicco R., & Ianes D. (2023), Cosa sappiamo dell’inclusione scolastica in Italia? I contributi della ricerca empirica, Trento, Erickson.
European Agency for Special Needs and Inclusive Education (2017), Raising the achievement of All Learners in Inclusive Education: Lessons from European Policy and Practice (A. Kefallinou and V.J. Donnelly, eds.), Odense, Denmark.
European Agency for Special Needs and Inclusive Education (2018), Analysis Framework for Mapping Inclusive Education Policies (V. Soriano, A Watkins, M. Kyriazopoulou, V. Donnelly, A Kefallionu, S. Ebersold and G. Squires, eds.), Odense, Denmark.
European Agency for Special Needs and Inclusive Education (2019), Inclusive School Leadership: Exploring Policies Across Europe (E. Oskarsdottir, V. Donnelly and Turner-Cmuchal, eds.), Odense, Denmark.
European Agency for Special Needs and Inclusive Education (2020), European Agency Statistics on Inclusive Education: 2018 Dataset Cross-Country Report. (J. Ramberg, A. Lénart, and A. Watkins, eds.) Odense, Denmark.
Hu L.T. & Bentler P. M. (1999), Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives, «Structural Equation Modeling: A Multidisciplinary Journal», vol. 6, n. 1, pp. 1-55, https://doi.org/10.1080/10705519909540118.
Ianes D. (2023), Cosa sappiamo dell’inclusione scolastica in Italia? I contributi della ricerca empirica. In S. Dell’Anna, R. Bellacicco e D. Ianes (eds.), L’inclusione scolastica in Italia, Trento, Erickson, pp. 13-44.
Ianes D. & Augello G. (2019), Gli inclusioscettici. Gli argomenti di chi non crede nella scuola inclusiva e le proposte di chi si sbatte tutti i giorni per realizzarla, Trento, Erickson.
Ianes D. & Dell’Anna S. (2020), Valutare la qualità dell’inclusione scolastica. Un framework ecologico, «L’Integrazione Scolastica e Sociale», vol. 19, n. 1, pp. 109-128, https://doi.org/10.14605/ISS1912011.
Ianes D., Demo H. & Dell’Anna S. (2020), Inclusive education in Italy: Historical steps, positive developments, and challenges, «Prospects», vol. 49, pp. 249-263, https://doi.org/10.1007/s11125-020-09509-7.
Imray P. & Colley A. (2017), Inclusion is dead: long live inclusion, Abingdon, Routledge.
Kinsella W. (2018), Organising Inclusive Schools, «International Journal of Inclusive Education», vol. 24, n. 12, pp. 1340-1356, DOI: 10.1080/13603116.2018.1516820.
Kyriazopoulou M. & Weber H. (2009), Development of a set of indicators — for inclusive education in Europe, European Agency for Development in Special Needs Education, Odense, Denmark.
Loreman T., Forlin C. & Sharma U. (2014), Measuring indicators of Inclusive Education: A systematic review of the literature. In C. Forlin & T. Loreman (Eds.), Measuring Inclusive Education, Bingley, UK, Emerald Publishing Limited, pp. 165-187.
Macnaghten P. & Myers G. (2004), Focus Groups: The Moderator’s View and the Analyst’s View. In C. Seale, G. Gobo, J.F. Gubrium & D. Silverman (Eds.), Qualitative Research Practice, London, Sage, pp. 65-80.
Marsili F., Sisti F. & Morganti A. (2021a), L’approccio schoolwide alla valutazione dell’inclusione scolastica in Europa: Una revisione della letteratura, «Giornale Italiano dei Disturbi del Neurosviluppo», vol. 6, n. 2, pp. 121-135.
Marsili F., Morganti A. & Signorelli A. (2021b), The Italian leadership on inclusive education: Myth or reality?, «Science Insights Education Frontiers», vol. 9, n. 1, pp. 1241-1263, https://doi.org/10.15354/sief.21.or048.
Marsh H.W., Hau K.T. & Wen Z. (2004), In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings, «Structural Equation Modeling: A Multidisciplinary Journal», vol. 11, n. 3, pp. 320-341, https://doi.org/10.1207/s15328007sem1103_2.
Meade A.W., Johnson E.C. & Braddy P.W. (2008), Power and sensitivity of alternative fit indices in tests of measurement invariance, «Journal of Applied Psychology», vol. 93, n. 3, pp. 568-592, https://doi.org/10.1037/0021-9010.93.3.568.
Mitchell D. (2015), Inclusive Education is a multi-faceted concept, «CEPS Journal», vol. 5, n. 1, pp. 9-30, https://doi.org/10.26529/cepsj.151.
Mitchell D. (2016), Diversities in education: Effective ways to reach all learners, Abington, Routledge.
Mitchell D. (2018), The Ecology of Inclusive Education. Strategies to tackle the crisis in educating diverse learners, Abington, Routledge.
Morganti A. (2018), L’insegnante efficace. Promuovere le competenze socioemotive per l’inclusione, Rome, Carocci.
Peeters J., Sharmahd N. & Budginaite I. (2018), Early childhood education and care (ECEC) assistants in Europe: Pathways towards continuous professional development (CPD) and qualification, «European Journal of Education», vol. 53, n. 1, pp. 46-57, https://doi.org/10.1111/ejed.12254.
Powell R.A. & Single M.H. (1996), Focus Groups, «International Journal for Quality in Healthcare», vol. 8, n. 5, pp. 499-504, https://doi.org/10.1093/intqhc/8.5.499.
Ryan K.E., Gandha T., Culbertson M.J. & Carlson C. (2014), Focus group evidence: Implications for design and analysis, «American Journal of Evaluation», vol. 35, n. 3, pp. 328-345, https://doi.org/10.1177/1098214013508300.
Schurig M., Weiß S., Kiel E., Heimlich U. & Gebhardt M. (2020), Assessment of the quality of inclusive schools A short form of the quality scale of inclusive school development (QU! SS) – reliability, factorial structure and measurement invariance, «International Journal of Inclusive Education», pp. 1-16, https://doi.org/10.1080/13603116.2020.1862405.
Soriano V. (2017), Inclusive education for learners with disabilities, European Asylum Support Office, Malta, Retrieved from https://policycommons.net/artifacts/2176306/inclusive-education-for-learners-with-disabilities/2931990/
Sullivan C. & Forrester M.A. (2019), Doing Qualitative Research in Psychology. A practical guide, https://uk.sagepub.com/en-gb/eur/doing-qualitative-research-in-psychology/book254461 (consulted the 28th of April 2023).
Taber K.S. (2018), The use of Cronbach’s alpha when developing and reporting research instruments in science education, «Research in Science Education», vol. 48, n. 6, pp. 1273-1296, https://doi.org/10.1007/s11165-016-9602-2.
Tong A., Sainsbury P. & Craig J. (2007), Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups, «International Journal for Quality in Health Care», vol. 19, n. 6, pp. 349-357, https://doi.org/10.1093/intqhc/mzm042.
UN (2006), Convention on the Rights of Persons with Disabilities, United Nations.
UN (2016), CRPD and Optional Protocol Signatures and Ratification, https://treaties.un.org/Pages/ViewDetails.aspx?src=IND&mtdsg_no=IV-15-a&chapter=4&clang=_en,
UNDP (2015), Human Development Report, United Nation Development Programme.
UNESCO (1994), The Salamanca Statement and Framework for Action on Special Needs Education, Salamanca, Spain.
Wolff C.E., Huila H., Tzaninis Y., Ros Magnusdottir B., Lappalainen S., Paulle B., Seppanen P. & Kosunen S. (2021), Inclusive education in the diversifying environments of Finland, Iceland and the Netherlands: A multilingual systematic review, «Research in Comparative & International Education», vol. 16, n. 1, pp. 3-21, https://doi.org/10.1177/1745499921991958.
Zorc-Maver D., Morganti M. & Vogrinc J. (2019), A comparative analysis of inclusion in Slovenia and Italy. In P. Zgaga (Ed.), Inclusion in education: Reconsidering limits, identifying possibilities, Berlin, Peter Lang GmbH, pp. 97-111.
Appendix
Appendix 1
Ecological Assessment Scale for Inclusion (EASI)
The General section of the EASI, which is the tool that proposes common questions to all stakeholders of the educational community, is a five-point Likert scale (ranging from 0 to 4), consisting of two symmetrical sections. One section comprises 10 questions related to the factor defined as «importance» while the other section comprises 10 questions related to the factor defined as «presence». The possible responses for the «importance» section range from 0, indicating «not at all important» to 4, indicating «very important». For the «presence» section, the responses range from 0, indicating «not present at all» to 4, indicating «very present».
|
IMPORTANCE |
|||||
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
PRESENCE |
|||||
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
|
0 |
1 |
2 |
3 |
4 |
1 Università degli studi di Perugia.
2 The contribution is the result of the joint work of the authors. Solely for attribution purposes, the introduction should be attributed to F. Marsili. The sections in the method defined as «Participants and procedures» and «Analysis», as well as the results section should be attributed to E. Delvecchio and M. Mirandi. The section in the method defined as «Measures» should be attributed to A. Signorelli. Finally, the discussion should be attributed to A. Morganti.
3 University of Perugia.
Vol. 22, Issue 3, September 2023
