Ronald A Berk

Thirteen Strategies to Measure College Teaching

A Consumer's Guide to Rating Scale Construction, Assessment, and Decision-Making for Faculty, Administrators, and Clinicians

• Broschiertes Buch

Produktbeschreibung

* Student evaluations of college teachers: perhaps the most contentious issue on campus

Produktdetails

• Verlag: Taylor & Francis
• Seitenzahl: 288
• Erscheinungstermin: 5. Mai 2006
• Englisch
• Abmessung: 228mm x 154mm x 17mm
• Gewicht: 426g
• ISBN-13: 9781579221935
• ISBN-10: 1579221939
• Artikelnr.: 22251497

Autorenporträt

Ronald A. Berk is Professor Emeritus of Biostatistics and Measurement and former Assistant Dean for Teaching, The Johns Hopkins University. He received the University's Alumni Association Excellence in Teaching Award in 1993 and Caroline Pennington Award for Teaching Excellence in 1997 and was inducted as a Fellow in the Oxford Society of Scholars in 1998. He has published 11 books and 130 journal articles / chapters. These publications reflect his unwavering commitment to mediocrity and his motto: "Go for the Bronze!" He is a popular speaker on teaching and assessment throughout the U.S. and Europe. Michael Theall is Director, CATALYST & Associate Professor of Education, Youngstown State University

Inhaltsangabe

ACKNOWLEDGMENTS
A FOREWORD (IN BERKIAN STYLE. BY MIKE THEALL
INTRODUCTION
1 TOP 13 SOURCES OF EVIDENCE OF TEACHING EFFECTIVENESS
A Few Ground Rules
Teaching Effectiveness. Defining the Construct
National Standards
Beyond Student Ratings
A Unified Conceptualization
Thirteen Sources of Evidence
Student Ratings
Peer Ratings
External Expert Ratings
Self-Ratings
Videos
Student Interviews
Exit and Alumni Ratings
Employer Ratings
Administrator Ratings
Teaching Scholarship
Teaching Awards
Learning Outcome Measures
Teaching Portfolio
BONUS. 360° Multisource Assessment
Berk's Top Picks
Formative Decisions
Summative Decisions
Program Decisions
Decision Time
2 CREATING THE RATING SCALE STRUCTURE
Overview of the Scale Construction Process
Specifying the Purpose of the Scale
Delimiting What Is to Be Measured
Focus Groups
Interviews
Research Evidence
Determining How to Measure the "What"
Existing Scales
Item Banks
Commercially Published Student Rating Scales
Universe of Items
Structure of Rating Scale Items
Structured Items
Unstructured Items
3 GENERATING THE STATEMENTS
Preliminary Decisions
Domain Specifications
Number of Statements
Rules for Writing Statements
1. The statement should be clear and direct
2. The statement should be brief and concise
3. The statement should contain only one complete behavior, thought, concept
4. The statement should be a simple sentence
5. The statement should be at the appropriate reading level
6.The statement should be grammatically correct
7. The statement should be worded strongly
8. The statement should be congruent with the behavior it is intended to measure
9. The statement should accurately measure a positive or negative behavior
10. The statement should be applicable to all respondents
11. The respondents should be in the best position to respond to the statement
12. The statement should be interpretable in only one way
13. The statement should NOT contain a double negative
14. The statement should NOT contain universal or absolute terms
15. The statement should NOT contain nonabsolute, warm-and-fuzzy terms
16. The statement should NOT contain value-laden or inflammatory words
17. The statement should NOT contain words, phrases, or abbreviations that would be unfamiliar to all respondents
18. The statement should NOT tap a behavior appearing in any other statement
19. The statement should NOT be factual or capable of being interpreted as factual
20. The statement should NOT be endorsed or given one answer by almost all respondents or by almost none
4 SELECTING THE ANCHORS
Types of Anchors
Intensity Anchors
Evaluation Anchors
Frequency Anchors
Quantity Anchors
Comparison Anchors
Rules for Selecting Anchors
1. The anchors should be consistent with the purpose of the rating scale
2. The anchors should match the statements, phrases, or word topics
3. The anchors should be logically appropriate with each statement
4. The anchors should be grammatically consistent with each question
5. The anchors should provide the most accurate and concrete responses possible
6.The anchors should elicit a range of responses
7. The anchors on bipolar scales should be balanced, not biased
8. The anchors on unipolar scales should be graduated appropriately
5 REFINING THE ITEM STRUCTURE
Preparing for Structural Changes
Issues in Scale Construction
1. What rating scale format is best?
2. How many anchor points should be on the scale?
3. Should there be a designated midpoint position, such as
"Neutral," "Uncertain," or "Undecided," on the scale?
4. How many anchors should be specified on the scale?
5. Should numbers be placed on the anchor scale?
6. Should a "Not Applicable" (NA. or "Not Observed"
(NO. option be provided?
7. How can response set biases be minimized?
6 ASSEMBLING THE SCALE FOR ADMINISTRATION
Assembling the Scale
Identification Information
Purpose
Directions
Structured Items
Unstructured Items
Scale Administration
Paper-Based Administration
Online Administration
Comparability of Paper-Based and Online Ratings
Conclusions
7 FIELD TESTING AND ITEM ANALYSES
Preparing the Draft Scale for a Test Spin
Field Test Procedures
Mini-Field Test
Monster-Field Test
Item Analyses
Stage 1. Item Descriptive Statistics
Stage 2. Interitem and Item-Scale Correlations
Stage 3. Factor Analysis
8 COLLECTING EVIDENCE OF VALIDITY
AND RELIABILITY
Validity Evidence
Evidence Based on Job Content Domain
Evidence Based on Response Processes
Evidence Based on Internal Scale Structure
Evidence Related to Other Measures of Teaching Effectiveness
Evidence Based on the Consequences of Ratings
Reliability Evidence
Classical Reliability Theory
Summated Rating Scale Theory
Methods for Estimating Reliability
9 REPORTING AND INTERPRETING SCALE RESULTS
Generic Levels of Score Reporting
Item Anchor
Item
Subscale
Total Scale
Department/Program Norms
Subject Matter/Program-Level State, Regional, and
National Norms
Criterion-Referenced versus Norm-Referenced Score Interpretations
Score Range
Criterion-Referenced Interpretations
Norm-Referenced Interpretations
Formative, Summative, and Program Decisions
Formative Decisions
Summative Decisions
Program Decisions
Conclusions
References
Appendices
A. Sample "Home-Grown" Rating Scales
B. Sample 360° Assessment Rating Scales
C. Sample Reporting Formats
D. Commercially Published Student Rating Scale Systems
Index.