Investigating the Facilitators and Barriers of Mathematics Teachers’ Assessment Literacy in the Digital Era: Instrument Development and Validation

The advancement of digital technology offers teachers many opportunities to improve their educational assessment. However, many teachers lack the competence to measure and support students’ learning through digital or non-digital assessment tools. This shortcoming became apparent when the COVID-19 pandemic closed school buildings and drove most teaching online. Hence, investigations to understand teachers’ assessment literacy and determine the facilitators and barriers (antecedents) of assessment practices in the digital era are critical. However, systematic reviews shows little validity or reliability evidence to support the current assessment literacy measures. Also, most studies overlooked the digital component in assessment literacy, whereas the others focused on digital assessment but neglected the values of traditional assessment methods (e.g., observation and question-answer). Moreover, no published instrument measures a teacher's ability to measure and support students’ learning integrating digital and non-digital assessment tool (Assessment Literacy in the Digital Era or ALDE).

Focusing on mathematics, a required subject in most education systems, this proposed study helps address this gap by developing and testing an instrument to investigate teachers’ Mathematics Assessment Literacy in the Digital Era (M-ALDE), investigating the profiles of local teachers’ M-ALDE, and exploring and examining the facilitators and barriers of M-ALDE. Based on our definition of M-ALDE in our theoretical framework, we develop potential test questions based on qualitative analysis of focus group interviews with experts and experienced teachers, have mathematics experts check them for substantive and content validity, and polish the questions based on qualitative analysis of focus group interviews with in-service teachers. Then we identify these questions’ underlying factor(s) for each M-ALDE dimension, test the unidimensionality of identified factor(s), determine each question’s quality, distinguish M-ALDE dimensions (knowledge, practice, and affect), and determine the relation between experts’ evaluations and teacher self-reports of the M-ALDE (external validity) on a random subsample. These validate the new instrument.

Based on 600 teachers’ survey responses to this instrument, we will investigate their M- ALDE profiles and their facilitators and barriers during focus group interview. Using our quantitative data, we will then examine the identified facilitators’ and barriers’ relations with M- ALDE.

In summary, the proposed project will present an instrument for M-ALDE. Researchers can use it for related research, such as how M-ALDE can facilitate student learning. Findings about the profile of local teachers’ M-ALDE and its facilitators and barriers will provide insights into local teachers’ M-ALDE, help stakeholders to improve in-service teachers’ M-ALDE, and eventually improve student learning.