Student evaluation of teaching (SET) ratings are used to evaluate faculty's teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., ; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty's teaching effectiveness.
The idea that assessment is intrinsic to effective instruction is traced from early experiments in the individualization of learning through the work of Benjamin Bloom to reviews of the impact of feedback on learners in classrooms. While many of these reviews detailed the adverse impact of assessment on learning, they also indicated that under certain conditions assessment had considerable potential to enhance learning. It is shown that understanding the impact that assessment has on learning requires a broader focus than the feedback intervention itself, particularly the learner's responses to the feedback, and the learning milieu in which the feedback operates. Different definitions of the terms “formative assessment” and “assessment for learning” are discussed, and subsumed within a broad definition that focuses on the extent to which instructional decisions are supported by evidence. The paper concludes by exploring some of the consequences of this definition for classroom practice.
Construct validity of peer assessment (PA) is important for PA application, yet difficult to achieve. The present study investigated the impact of an assessment rubric and friendship between the assessor and assessee on construct validity of PA. Two-hundred nine bachelor students participated: half of them assessed a peer's concept map with a rubric whereas the other half did not use a rubric. The results revealed a substantial reliability and construct validity for PA. All students over-score their peers’ performance, but students using a rubric were more valid. Moreover, when using a rubric a high level of friendship between assessor and assessee resulted in more over-scoring. Use of a rubric resulted in higher quality concept maps for peer and expert ratings.
The main aim of the study was to identify school characteristics that can reduce the relation between socio-economic status (SES) and achievement, so that equity of educational outcomes can be improved. Data from 50 countries participating in the Trends in International Mathematics and Science Study (TIMSS) conducted in 2011, focusing on Grade 8 mathematics, was analysed. Two-level random slopes models fitted at school- and student-levels were used to investigate the influence of quality and quantity of instruction, school climate, and school SES on the within-school regression slope for achievement on SES. The results showed school SES to be the strongest determinant of slope differences across schools and educational systems. Whether school SES relates negatively or positively to the within-school regression of achievement on student SES is an indicator of whether the educational system is compensatory or anti-compensatory with respect to student SES.
Environmental education imparts knowledge and creates experience to change beliefs, attitudes – and most importantly – behavior. What are the deep motivators of human behavior? Theory and research suggest that feeling connected to someone or something motivates protective and self-sacrificing behavior. This paper reviews the large body of research demonstrating that connectedness to nature is an important predictor of environmentally responsible behavior. We review past research on self-reported behavior, then summarize new research from our lab that demonstrates a link between connectedness to nature and actual conservation behavior (electricity use). We conclude that promoting connectedness to nature should be a goal for environmental education programs, and should therefore be an important part of any assessment.
Scholars often focus on the presence of teacher stress and anxiety as emotional side effects of a school inspection, though this limited focus has resulted in an incomplete view of teachers’ experiencing of other emotions. Additionally, lack of evidence on positive emotional inspection outcomes raises questions about the presence of such emotions during a school inspection visit. In this study, we examined the presence of teachers’ emotions with regard to different moments during the actual school visit. Additionally, we explored teachers’ cognitive responses associated with the experience of these emotions. Survey data from 316 teachers in 42 primary schools is analyzed using multilevel analyses. Findings show that emotions of joy were most frequently reported with regard to the three moments. Emotions of anger and sadness were reported to a lesser extent. Findings showed that friendly, constructive and transparent communication between both parties is important for teachers’ experience of positive emotions.
This study attempts to guide the development of pharmacy mentors by investigating postgraduates’ and mentors’ perceptions regarding the importance of mentor competence. We interviewed 20 experts from the Army Medical University of China to identify items representing mentor competence and used these items to prepare a questionnaire. Then, we conducted an exploratory factor analysis of the responses of 216 pharmacy mentors and postgraduates from 8 universities in Western China concerning the importance of many aspects of mentor competence. Six dimensions (research competence, educational competence, supportive competence, communication competence, academic achievement, and professional knowledge) were identified by the factor analysis, which showed an internal consistency reliability ranging from 0.83 to 0.92 and explained 67.12% of the total variance. The results of this study are useful for evaluating pharmacy mentor competences at universities.
The Accelerated Schools Program (ASP) mobilizes a cohesive set of informal control mechanisms as a part of its approach to school-wide improvement. However, despite their frequent use in educational settings, the empirical investigation of the nature and function of organizational controls in building trust in schools has been largely overlooked in the educational literature, particularly from within a causal research framework. This study employed a propensity score matching approach to data from the Study of Instructional Improvement, a longitudinal study of the design, implementation, and instructional effectiveness of three widely-adopted comprehensive school reform models including Accelerated Schools. This quasi-experimental approach permitted the isolation of ASP’s effect on growth in teacher-teacher relational trust over time while controlling for implementation depth across the 54 schools and 1561 teachers in the sample. Analysis revealed a robust treatment effect of the ASP model on growth in teacher-teacher relational trust over time. Several components of the informal control process also produced positive change in teacher-teacher trust growth over time, including critical dialogue, innovation and risk-taking, and collective responsibility.
This study aims to provide more insight into how students’ perceptions of assessment quality are related to their learning approaches and learning outcomes. Six variables associated with the construct of students’ perceptions of assessment quality are distinguished: 1) effects of assessment on learning, 2) fairness of assessment, 3) conditions of assessment, 4) interpretation of test scores, 5) authenticity of assessment, and 6) credibility of assessment. 204 higher education students completed the Students’ Perceptions of Assessment Quality Questionnaire (SPAQQ), and the Approaches to Learning and Studying Inventory (ALSI), and the students’ learning outcomes (grades) were obtained. Firstly, results indicate that the students’ perceptions of the effects of assessment on learning are positively related to the students’ deep learning approach and the strategic learning approach and negatively related to the surface learning approach. Secondly, the students’ perceptions of the conditions of assessment are positively related to their learning outcomes of the assessments.
Reading comprehension is an important skill in secondary education, yet many history teachers find it difficult to provide adequate reading strategy instruction. In this study, we designed a digital learning environment to support teachers’ instruction of reading strategies based on student data. We provided history teachers in the experimental conditions with a visualisation of student performance data. Additionally, these teachers received professional development (PD) training and a guiding manual on how to translate these data into structured, explicit reading strategy instruction. Teachers in the control condition were only provided with basic data. We investigated teachers’ personal experiences through micro-level analysis of qualitative interview data. Our results show that teachers in the experimental condition improved the variation of their strategy instruction and used modelling behaviour more often after the PD training. However, we also identified multiple contextual implementation barriers that provided us with important suggestions for future practice-oriented educational research.
This study compared teacher’s feedback and students’ self-regulated learning (SRL) in mathematics and the relationship between them in a high- and a low-achieving secondary school. A total of 1282 secondary students participated in this study, 652 and 630 from the high- and the low-achieving school, respectively. The results of -tests indicated that teachers at the high-achieving school compared to teachers at the low-achieving school provided more praise and more verification, directive, and scaffolding feedback but less criticism. Students at the high-achieving school reported using more SRL strategies and having greater motivation in mathematics than those at the low-achieving school. Two-group structural equation modelling indicated that merely verification feedback and teacher’s praise had positive relationships with high-achieving school students’ SRL, whereas scaffolding feedback, teacher’s praise, and criticism had positive relationships with low-achieving school students’ SRL. Particularly, directive feedback had more negative correlations with students’ SRL at the low-achieving school than at the high-achieving school.
Quality assurance in higher education has developed and has been totally reformed in the last decades. In particular, in Italy the AVA system (Autovalutazione, Valutazione e Accreditamento, in English self-evaluation, evaluation and accreditation) was used in 2016 to verify the performance of 118 randomly selected university study programmes (SPs). This study investigated the relationship between quality performance (AVA evaluation) and discipline areas with the aim of understanding the ‘quality mechanism’ thus determining whether some disciplines perform better in quality assurance processes. A strong correlation between technical/engineering SPs and good quality assurance results was found, probably because quality expertise is particularly developed in these disciplines. The creation, for each university of an expert’s unit constituted by professionals with a technical/engineering mentality to monitor and improve quality, is therefore recommended.
The Perceived Stress Scale (PSS-10) is widely used. Factor-analytic studies have reported a two-dimensional structure (Perceived Stress and Perceived Lack of Control subscales), however, few studies have employed Item Response Theory (IRT) or focused on the issue of differential item functioning (DIF). The Rasch Model (RM) for ordinal data and graphical loglinear Rasch models (GLLRM) were used with data from 1,552 technical and psychology university students. After elimination of item 6 the two subscales were found each to fit a GLLRM. However, lack of conditional independence of the items (LD) and equal item functioning (DIF) were found. The findings point to the need for further construct validity studies of the PSS-10 subscales focusing on the issues of DIF and LD.
Previous research point to difficulties for teachers to interpret reading assessment data with regard to instructional decisions. This study explored Swedish primary teachers' use of assessments and recommendations, in order to be able to target individual needs. Eight teachers participated in a reading program and were interviewed in focus-group meetings. The analysis of teacher narratives stemming from assessment use resulted in three themes: , and . The themes indicated that the teachers had become aware of their students’ learning, had employed teaching based on informed decisions, and showed initial professional growth. However, the assessment details and the recommendations allowed for more adjustments than was evident in the teachers’ narratives. The results point to the relative difficulty of targeting individual needs in the general classroom education, and to the challenges of changing teaching practices.
A consensus is emerging for meta-analyses regarding a range of selected topics in pharmacology education. This review provides a critical overview of problem-based learning (PBL) in pharmacology education. Based on several inclusion criteria, databases were searched, and 37 controlled trial studies were identified and extracted. The meta-analysis found that PBL had a positive effect on gaining higher theoretical scores (SMD = 6.77, 95% CI [5.23, 8.31], < 0.00,001) assessed through examinations. The results of questionnaires for students’ feedback showed that PBL was superior to conventional teaching methods in improving students’ outcomes of self-study, learning interest, team spirit, problem solving, analyzing, scope of knowledge, communication, and expression. These results suggest that PBL in pharmacology education is considered superior to traditional lecture-based learning.
The present paper analyzes the impact of blended learning (BL) on the academic achievement of higher education students. A meta-analysis ( = 51 effect sizes) was conducted to perform a statistical synthesis of studies contrasting student performance in BL conditions with traditional classroom instruction. We include disciplines and instructors’ end-of-course evaluation method as moderating variables. The results show that BL demonstrates a small summary effect ( = 0.385, 0.001) compared to traditional teaching methods A significantly higher mean effect size was found in STEM disciplines ( = 0.496) compared to that of non-STEM disciplines ( = 0.210). Nevertheless, the weighted mean effect sizes reveal no significant differences regarding of end-of-course assessment methods, namely one-moment and multiple-component assessment. The finding confirms that BL is significantly associated with greater learning performance of STEM-disciplined students than with traditional classroom practice. Accordingly, discussion concerning the findings and implications for future research are elaborated.
This study investigated the effect of online summative and formative assessments on 130 Iranian English as foreign language (EFL) junior university students' writing ability. Three assessment interventions in writing performances of participants were investigated in 27 sessions using pretest/posttest time series design. The interventions included online summative assessment and online portfolio writing assessment conducted individually and online collaborative formative assessment. Data were collected from students' individual writing in both online summative and portfolio formative assessments as well as collaborative writing in online collaborative formative assessment in e-writing forum. The writing performances were assessed using International English Language Testing System (IELTS) rating scale. Paired sample -test and analysis of covariance results indicated improved writing ability in all interventions and highest significant enhanced writing in online collaborative writing assessment intervention. The results imply that using engaging technology and techniques along with appropriate assessment strategies is a powerful way of making learning efficient.
English as a second/foreign language education (ESL/EFL) in the supplementary setting is under-researched despite its prevalence worldwide. This quantitative study investigated the effects of supplementary and mainstream ESL/EFL education on eleven facets from learners’ secondary-level and tertiary-level perspectives. In addition, we examined learners’ perception of the usefulness of supplementary and mainstream classes in equipping them for the use of English for different purposes in the tertiary setting. 203 participants studying at two tertiary institutions in Hong Kong completed a 72-item questionnaire. The MANOVAs conducted revealed significant main effects of education settings (i.e. supplementary/mainstream) and education levels (i.e. secondary-level/tertiary-level) as well as interaction effects of these variables on learners’ perception. There were also significant differences in learners’ views of the usefulness of the two education settings in preparing them for the use of English for specific purposes at tertiary level. This article culminates with a discussion of these findings and implications.
Assessment as learning (AaL) puts students at the centre of learning and is considered a powerful alternative assessment approach that can maximize student learning. How it can be applied as a pedagogical and assessment tool in writing classrooms, however, is not clearly understood. Research on the implementation of AaL as a unitary concept in L2 writing classrooms is scant, particularly at the primary level. To fill these gaps, the study investigates the implementation of AaL in two Hong Kong primary writing classrooms from teachers’ and students’ perspectives, focusing specifically on the perceived benefits and problems. Using data gathered from interviews with two participating teachers and six selected students, as well as classroom observations, the findings show that the teachers’ very first attempt to put AaL into practice reaped benefits for themselves and the students, but a few problems were noted too, which provide useful implications for practice and further research.
Given recent evidence of rising levels of social segregation in European countries, this study uses standardized data from the Program for International Student Assessment ( = 171,159; 50.5% male) to examine the extent to which education systems in Europe are socially segregated and whether social segregation in the school system affects achievement gaps between students of different social origin. Results suggest that the degree of social segregation within education systems varied substantially across countries. Furthermore, multilevel regression models indicate that the effect of socioeconomic status on student achievement was moderately but significantly stronger in more segregated education systems, even after controlling for alternative system-level determinants of social inequality in student achievement. These findings provide original evidence that social segregation in education systems may contribute to the intergenerational transmission of educational (dis)advantage and thus serve to exacerbate wider problems of socioeconomic inequality in Europe.