Linking Teacher Learning to Student Success:

TAG:  walden university 

Published Time: -

Filetype: pdf

Filesize: 124381

Linking Teacher Learning to Student Success: The Impact of Walden University’s Master’s Program in Elementary Reading and Literacy A Distance-Learning Program Executive Summary 27 © 2005 Walden University Executive Summary Measuring Teacher Classroom Performance Through Student Achievement “Recent studies offer compelling evidence that teacher quality is one of the most critical
components of how well students achieve. In recognition of the importance of having
effective teachers, No Child Left Behind includes provisions stating that all teachers in
core academic areas must be highly qualified by the end of the 2005–2006 school year.” — No Child Left Behind: A Toolkit for Teachers In its continuing efforts to measure the impact of its master’s degree programs on teacher
effectiveness and student achievement, Walden University, in association with Canter, has
commissioned a study comparing teachers enrolled in its MS in Education with a focus on
Elementary Reading and Literacy program with teaching colleagues who held master’s degrees from
other universities. The participants and teachers were located in the Peninsula and Tacoma region of
the state of Washington. Although this study was launched prior to the creation of the No Child Left Behind Act of 2001, its
design shares a common objective to measure teacher quality through scientifically based research.
Within this context, the study employed rigorous methods to measure teacher performance in the
classroom and its relation to student achievement in reading. This is the second in a series of studies
conducted by Canter in an effort to quantify the impact of its learning model and course content
on teacher effectiveness. Initial findings from the Washington state study revealed students of the teachers enrolled in the
Walden University master’s degree program experienced greater gains and clearer improvement based
on test results than students taught by teachers who had not enrolled in the Walden degree
program. Study Framework
The purpose of this study was to determine the impact of the Walden master’s degree program by
exploring: • the extent to which those completing the degree used the practices associated with the effective teaching of reading • the gains made by the students of those completing the degree over the course of one school year • whether the achievement of the students of Walden University graduates would equal or exceed the achievement of students of traditional on-campus master’s degree graduates Forty-six teachers and 805 students participated in the study, which examined changes in teacher
practices and resulting student reading scores on standardized tests. Participants were full-time
teachers in grades 1–5 representing 14 elementary schools. Student profiles indicated that both
groups were similar in terms of socioeconomic background, with the majority of students classified
as low socioeconomic. In fact, 75 percent of student participants were considered low
socioeconomic based on the percentage receiving free and reduced lunch. 28 © 2005 Walden University The study was conducted by an independent evaluator with data collected through pre- and post-
testing of students, responses to questionnaires, sample lesson plans, and reports of classroom
activities. The Degrees of Reading Power (DRP) test for primary and standard grades was chosen for
student testing due to its reputation for reliability and low measurement error. Study Findings Teacher Behavior
Walden teachers (the treatment group) participating in this study had taught an average of 7.5
years, compared to the non-Walden teachers (the control group) who had an average of 17 years of
classroom experience, although experience had ultimately no significant impact on study results. When asked if they had made changes in how they taught reading during the current school year,
90 percent of Walden teachers agreed they had, as compared to 50 percent of the non-Walden
teachers. More than 87 percent of Walden teachers planned to make additional changes during the
next school year, compared to 57 percent of non-Walden teachers. Teachers were also asked to rate the actual impact they believed their master’s degree program had
on their teaching in the classroom on a scale of 1–10. Walden teachers rated their program 8.3 on
average, compared to 6.2 by their study counterparts. Student Achievement
According to the 2002 National Assessment of Educational Progress, only 31 percent of students in
the fourth grade performed at or above the proficient achievement level. Within this study, students
from grades 1–5 were administered the Degrees of Reading Power (DRP) test to measure progress
throughout the academic year. Student achievement results were viewed from three perspectives: statistical significance; effect size;
and educational significance. Analysis of overall gains made by students in grades 1–5 revealed that
the students of teachers completing the Walden University program (the treatment group) scored
significantly higher than the students of non-Walden teachers (the control group), based on
standardized tests. Analysis by grade level found that first-grade students of Walden teachers exhibited very significant
gains in reading proficiency. According to the Department of Education, an effect size of .33
standard deviations (SD) is considered educationally significant; an effect size of .50 SD is classified
as educationally exemplary. The test score improvement of the first graders in the treatment group
represented an effect size of 1.6 SD. The effect size for grades two and three were .81 and .33 SD
respectively. The effect size for grades four and five were not educationally significant. In addition, the Walden group performed proportionately better than the national norm reference
group. The standardized test used in this study controlled for pre/post improvement based upon a
large national sample of students who had previously taken the test. The Walden group, however,
clearly outperformed these assumed gains by moving from the 40th percentile in the pre-test (Fall
2002) to the 59th percentile in the post-test (Spring 2003). The control group remained below the
50th percentile in both tests. About Walden University and Canter Walden University has provided graduate distance learning education for more than 30 years and
has students from over 40 countries and in all 50 states. Accredited by the Higher Learning
Commission, Walden is a member of the North Central Association of Colleges and Schools
(NCA). Since 1975, Canter has built a reputation for developing research-based, practical curriculum. That
tradition continues today with partnerships with leading accredited universities including Walden
University. The Walden University MS in Education with a focus on Elementary Reading and
Literacy is based on the International Reading Association standards for the classroom professional.
The program includes five reading-specific courses that are aligned with the research-based
recommendations found in the Report of the National Reading Panel (2000). Course strategies and curriculum materials were collaboratively developed by Walden University
and Canter to link theory and practice in three ways: 1) educational experts present the latest
research; 2) educational theory is modeled by master teachers in classroom settings; and 3) texts,
readings, and study guides encourage peer study groups of working teachers to reflect on their
teaching and to practice strategies in the classroom. About the Study Author Evelyn Ogden received her Ed.D. from Rutgers University in the fields of educational and
psychological evaluation and statistics. She is the author of five books as well as major studies and
reports on effective and ineffective schools, programs that work, strategies to increase student
achievement, programs for at-risk students, and dissemination of successful practices. Dr. Ogden
served for many years on the U.S. Department of Education Program Evaluation Panel and served
as the Deputy Assistant Commissioner for Research, Planning and Evaluation. Currently she serves
as Deputy Superintendent of East Brunswick Public Schools in New Jersey. For More Information Terrance Peet-Lukes
Vice President, Research & Analysis
Canter, a subsidiary of Laureate Education, Inc.
(800) 733-1711, ext. 4830
terry.peet-lukes@laureate-inc.com 29 © 2005 Walden University Linking Teacher Learning to Student Success: The Impact of Walden University’s Master’s Program in Elementary Reading and Literacy A Distance-Learning Program 30 © 2005 Walden University White Paper 31 © 2005 Walden University Table of Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Design of the Research Study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Research Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Introduction The ultimate demonstration of effective teaching is student achievement. However, many teachers
who want to improve their knowledge and skills are prevented by a lack of access to teacher-
education programs featuring the best in up-to-date, research-based instruction. Master’s degree programs represent an extensive form of staff development. Possession of a master’s
degree has long been highly valued in the field of K–12 education and perceived as an indicator of
effective teaching and higher student achievement by boards of education, teachers, and the public. This paper describes Walden’s Master of Science in Education with a focus on Elementary Reading
and Literacy, and its impact both on teachers who were enrolled in it and on their students. The
goal of the program is to provide an intensive, high-quality master’s degree program that results in
graduates who are more effective in the classroom and whose students have higher levels of
achievement. Course strategies and materials were designed to link theory and practice in three
ways: (a) educational experts present the latest research; (b) educational theory is modeled by
master teachers in classroom settings; and (c) texts, readings, and study guides encourage peer study
groups of practicing teachers to reflect on their teaching and to practice strategies in the classroom. Background—Commitment to Research
For over 30 years, Walden University has been known for providing innovative graduate distance-
education programs that adhere to the highest academic standards. To better determine the success
of Walden’s M.S. in reading program, the university has engaged in this research study to verify
program impact on student achievement. Walden’s program combines solid educational theory with practical strategies to help teachers teach
and help children learn. The program curriculum is based on the most recent reliable and valid
research in areas ranging from adult-learning modalities to practical teaching strategies, utilizing the
knowledge of top experts in the field. About This Paper
This paper describes the impact of a distance-learning Master of Science in Education program
with a focus on Elementary Reading and Literacy available through Walden University. The remaining sections of this paper address the following topics:
• “Overview of the Walden University Elementary Reading and Literacy Specialization,” including both course content and distance-learning features. • “Design of the Research Study,” including study participants, procedures, and potential limitations of the study results. • “Research Findings,” including teacher ratings of program effectiveness, program impact on classroom practices, and program impact on student reading achievement. • “Conclusions” that can be drawn from the research and plans for future research. 32 © 2005 Walden University Overview of the Walden University Elementary Reading and Literacy
Specialization Walden University’s Master of Science in Education program with a focus on Elementary Reading
and Literacy offers practical, research-based instruction in a distance-learning format on key topics
related to reading instruction. Students complete 10 required courses over five semesters while
continuing with their own teaching. Students accepted for the master’s program must have a
bachelor’s degree from a regionally accredited institution, a teaching certificate or two years of
teaching experience, and access to a classroom where they can apply the strategies taught in the
program. At the conclusion of the program, students must submit a portfolio documenting
accomplishment of the identified program outcomes. Course Content
Program courses include the following:
• Teacher as Professional—Develops skills and strategies that help teachers perform at their highest level. • Foundations of Reading and Literacy Development—Provides background knowledge to understand topics and issues relevant to reading instruction, including basic tenets of a balanced
approach to literacy instruction. • Effective Teaching Using Learning Styles and Multiple Intelligences—Helps teachers design instruction that supports diverse learning styles and multiple intelligences. • Strategies for Literacy Instruction, Part 1—Addresses word knowledge, phonemic awareness, phonics, vocabulary, and fluency. • Habits of Mind: Thinking Skills to Promote Self-Directed Learning—Focuses on attributes of self-directed learners and critical thinkers in three contexts: teachers’ professional growth,
students’ growth as learners, and growth of the learning organization as a whole. • Strategies for Literacy Instruction, Part 2—Addresses topics such as prior knowledge, metacognition, and reading as a constructive process; active reading; comprehension before,
during, and after reading; guided reading; and integrating reading and writing. • Collaborative Action Research—Provides tools to help teachers study issues and develop plans for action arising out of their research. • Supporting the Struggling Reader—Provides a working knowledge of common reading difficulties, methods for diagnosing those difficulties, and guidelines for accessing appropriate
resources to provide instructional support for students. • Instructional Models and Strategies—Introduces research-based models of instruction and presents strategies for using them to design curricula, instruction, and assessments. • Planning and Managing the Classroom Literacy Program—Discusses a variety of concepts and techniques, including flexible grouping, language-rich classrooms, and techniques for time-
management, instructional pacing, and transitions. 33 © 2005 Walden University Course strategies and materials were designed by Walden University. The curriculum is directly
related to the nationally accepted standards for teaching reading for classroom professionals as set
forth by the International Reading Association (IRA) in Standards for Reading Professionals (1998)
and is consistent with the recommendations of the National Reading Panel (2000). The courses
link theory and practice in three ways:
1. Educational experts present the latest research.
2. Educational theory is modeled by master teachers in classroom settings.
3. Texts, readings, and study guides encourage peer study groups of practicing teachers to reflect on their teaching and to practice strategies in the classroom. Courses are designed to provide teachers with the knowledge, skills, and tools to implement an
effective, developmentally appropriate, balanced literacy program. Teachers learn a broad variety of
tested instructional and assessment strategies to meet the range of needs of students in their
classrooms. Each course focuses on developing the teachers’ abilities to assess and monitor student
progress, and to select, from their expanded repertoire, the literacy-development strategies that are
best suited to increasing student achievement. Distance-Learning Features
The majority of courses follow a distance-learning model that features
• clear outlining of objectives.
• invocation of teachers’ prior knowledge through introductory activities.
• presentation of new information through videotapes or DVDs, readings, etc. Each of the 10 graduate courses includes approximately 15 instructional videos, each of which features lectures
and interviews with nationally known experts in reading. • modeling of best practices through demonstrations by master teachers in real classrooms, which also appears on the instructional videos. • discussion and collaboration in a peer study group consisting of three to eight teachers taking the same courses. These study groups are designed to provide practice, feedback, and coaching for
participants. Students also complete assignments in the study group. Study groups can meet face
to face or communicate online; most groups are formed within a school or district. Within this
study, all participants took part in face-to-face study groups. • application of concepts in teachers’ own classrooms, as directed in assignments from the course study guide. • reflection on teachers’ experiences applying the new concepts.
• assessment of learning. Assessment is both formative and summative, and includes both peer feedback in the study group and feedback from Walden University faculty via phone, fax, and
email. 34 © 2005 Walden University Design of the Research Study Evelyn Ogden, Ed.D., an independent research consultant, was contracted to study the impact of
the Walden University distance-learning master’s specialization in Elementary Reading and Literacy.
Specifically, the study was designed to
• determine the extent to which teachers taking part in the program increased their use of practices associated with the effective teaching of reading. • determine if students of teachers enrolled in the program experienced gains in reading achievement that were sufficiently large to be of practical significance. • compare the achievement of students taught by teachers enrolled in the Walden University program with that of students taught by graduates of traditional on-campus master’s degree
programs. Study Participants—Teachers
The study followed a quasi-experimental pre-post design. Two groups of full-time first- through
fifth-grade teachers from two school districts in the state of Washington (Peninsula and Tacoma)
and 14 elementary schools participated in the study. The experimental group consisted of 32 teachers who enrolled in the Walden University Master of
Science in Education distance-learning program with a focus on Elementary Reading and Literacy
starting in the summer 2002 term and who completed the program in the fall 2003 term.
• The Walden teachers had previously completed an average of 2.8 undergraduate reading courses as part of their preparation for teaching. They rated the impact of these courses on their reading
instruction in the classroom at an average of 5.8 on a 10-point scale. • These teachers had taught an average of 7.5 years, with a range of experience from 1 to 26 years. Teachers in the experimental group had already completed two of their courses from Walden
University at the time of the student pretest in September 2002. During the course of the
2002–2003 school year, they completed an additional seven courses from Walden. The final course,
Planning and Managing the Classroom Literacy Program, was completed after the student posttest
and final teacher survey had been administered in June 2003. The comparison group consisted of 14 teachers who already had master’s degrees in education from
a source other than Walden University at the time the study began.
• Comparison-group teachers had completed an average of 2.5 undergraduate reading courses and an additional 3.4 graduate courses in reading. They rated the impact of these courses on their
reading instruction in the classroom at averages of 5.3 and 7.0, respectively, on a 10-point scale. • These teachers had a range of teaching experience from 2 to 30 years, with an average of 17 years teaching experience. Analysis showed, however, that years of experience were not significantly
correlated to achievement of students as reported in this study. 1 35 © 2005 Walden University Study Participants—Elementary Students
Student participants in the research were 805 children who were enrolled in reading classes taught
by the experimental and comparison group teachers during the 2002–2003 school year.
• Student profiles indicated that both treatment and comparison groups were similar in terms of socioeconomic background, with about three-fourths of the students in both groups classified as
low socioeconomic status, based on the percentage of students receiving free lunch. • All classes consisted primarily of general education students. Many classes also included a few (2– 4) special education or ESL (English as a second language) students. Procedures
Three surveys were used to gather information from both groups of teachers related to students in
their classes, strategies utilized in reading instruction, and background courses in reading
instruction. Specifically,
• Survey 1, administered in June 2002, was used to collect information about participants’ teaching practices at the time the study began (focusing on the two weeks prior to administration of the
survey). • Survey 2, administered in October 2002, was used to analyze the demographics of classes being taught by study participants. • Survey 3, administered in June 2003, was used to collect information about participants’ teaching practices at the time the study ended. This information was compared with information from the
first survey about teaching practices at the beginning of the study. • Responses to the third survey were also used to determine how many participants had changed their reading instruction during the 2002–2003 school year, how many planned to change their
reading instruction for the 2003–2004 school year, and how participants rated the impact of
their master’s degree program on their classroom practices. Elementary student reading skills were measured using the Degrees of Reading Power (DRP)
Primary (grade 1) and Standard (grades 2–5) test forms, which were administered to all students in
classes taught by experimental and comparison group teachers in September 2002 and again in
June 2003. 2 Limitations of the Study
Specific elements in the study design and implementation may have affected the outcomes or may
otherwise limit the conclusions that can be drawn from the study: 3 • Selection of teachers who took part in the study may have affected the results. In some cases, teachers were excluded because they taught students who did not complete reading assessments
(e.g., kindergarten teachers). In other cases, teachers were reluctant to dedicate time to respond to
questionnaires and complete multiple testing of their students. • Due to the quasi-experimental design of the study, it is impossible to guarantee that the experimental group and comparison group (both teachers and students) are truly comparable. • The number of comparison-group teachers is considered low (i.e., below 20) for deriving statistical results. 36 © 2005 Walden University • Since teachers in the experimental group had already completed two courses in the Walden program before the first survey, and had not yet completed the final course by the time of the third
survey, study results do not reflect the full before-to-after impact of the Walden University
program. However, it seems likely that the impact of the full program would be at least as great
as the impact of partial completion of the program. • Since teachers in the comparison group had already completed their master’s degrees before the study started, it is impossible to directly compare the before-to-after changes resulting from the
Walden University program with before-to-after changes resulting from other, more traditional
master’s programs. • Because the study covered only the year that teachers were involved in the Walden program, it is impossible to say what the lasting impact of the program might be. Ongoing tracking of student
test-score results into future years is planned to help determine what the lasting outcomes may be. • Student achievement results have been calculated based on the number of students, as opposed to the number of teachers. This may have made it easier to reach statistical significance in the
results. Research Findings Teacher Ratings of Program Effectiveness
Teachers in both groups were asked to rate the effectiveness of their master’s program in terms of
actual impact on their classroom practices using a scale from 1 (“Little real impact on my classroom
practices”) to 10 (“A great deal of impact on my classroom practices”). Walden teachers rated their
master’s program significantly higher than control teachers rated their programs (means of 8.3 and
6.2, respectively). 4 Teachers who were enrolled in the Walden program were also asked to rate the relevance of
individual courses they had taken in terms of impact on their classroom instruction. Eight out of
nine courses completed by participants at the time of the final survey received a mean rating of 7 or
above on a scale of 0 to 10. These results demonstrate not only the perceived relevance of
individual courses in the Walden University program, but also Walden’s commitment to collect
research information that can be used to gauge the effectiveness of individual program components
on an ongoing basis. Program Impact on Classroom Practices
Teachers in the Walden program were asked in June 2003 if they had changed the way they taught
reading during the previous year and if they planned to change how they taught reading for the
next year. Very large majorities answered yes to both questions (90.6% and 87.5% respectively for
the 2002–2003 and 2003–2004 school years). At the end of both the 2001–2002 and the 2002–2003 school years, teachers were asked to use a
checklist to identify instructional strategies they had used during the previous two weeks with a
whole class, small group, or individual students. 5 These strategies were mapped to five areas identified by the National Reading Panel as effective in teaching literacy: phonics, fluency, 37 © 2005 Walden University vocabulary, guided reading, and comprehension (National Reading Panel, 2000). Comparison of
the data showed that in all five areas, Walden teachers used significantly more strategies than the
comparison group after a year in the Walden University program. 6 Taken together, these results suggest a positive impact on teachers’ instructional behavior from their participation in the Walden
University courses. Program Impact on Student Reading Achievement
Students in classes taught by both Walden teachers and comparison group teachers took the
Degrees of Reading Power (DRP) tests in September 2002 and again in June 2003. Results of the
tests were as follows:
• Students taught by Walden teachers improved significantly between the pretest and posttest, both overall and at every grade level. 7 Students taught by comparison group teachers improved significantly overall at grade levels 1–3, but not at grades 4 and 5. 8 • Overall, students taught by Walden teachers improved significantly more than students taught by comparison group teachers. Grade-by-grade analysis showed that this difference was significant
for first grade, but not for other grades. 9 • Interpreted in terms of effect size, overall improvement of students taught by Walden teachers reached the level of “exemplary” educational significance (ES = .51), while overall improvement
of students taught by comparison group teachers reached the level of “significant” (ES = .37). 10 • Broken down grade by grade, effect sizes were “exemplary” for both Walden and comparison group teachers at grades 1 and 2, “significant” at grade 3, and not significant at grades 4 and 5. 11 • In terms of norm-referenced results, both groups of students improved their standing from pretest to posttest, jumping 19 percentiles for students of Walden teachers and 14 percentiles for
students of comparison group teachers. Taken together, these results demonstrate levels of reading improvement among students of teachers
enrolled in the Walden program that were at least comparable to achievement gains among students
of teachers with more traditional master’s degrees. Overall and at grade 1, gains were significantly
higher for students of Walden teachers. Conclusions Possession of a master’s degree has long been viewed by boards of education, teachers, and the
public as an indicator of effective, high-quality teaching, leading to higher student achievement.
States and local boards of education report the number of teachers with graduate degrees as an
indicator of the quality of schools. School districts across the country recognize the value of
graduate degrees through higher salaries and often subsidize tuition for teachers enrolled in
graduate programs. While some teachers come to the teaching profession with a master’s degree, more commonly
teachers pursue graduate work on a part-time basis while continuing to teach. In the past three
decades, new options have emerged in the form of distance-learning graduate programs,
unbounded by geography and rigid schedules, which make the pursuit of a master’s degree easier 38 © 2005 Walden University for many teachers. However, both teachers and school districts often wonder about the quality of a
distance-learning degree program and
its potential to improve teaching. The current study provides evidence supporting the value of the distance-learning Master of Science
in Education program with a focus on Elementary Reading and Literacy available through Walden
University on two grounds:
• The research demonstrates that the Walden University master’s program had a real, positive effect on classroom practices of teachers who took part in it, increasing their likelihood to use strategies
that are linked to effective, research-based reading instruction. • The research shows that students taught by teachers enrolled in the Walden University program improved at least as much in their reading achievement as students taught by teachers who had
completed more traditional master’s programs. In other words, Walden University graduates met
or exceeded the standard set by non-distance-learning master’s programs. These results represent a validation both of the distance-learning format for teacher education and
of the specific curriculum of the Walden University program in elementary reading and literacy.
The results constitute good news for schools and teachers who are interested in taking advantage of
distance-learning opportunities to improve their teaching. The goal of this research was to report the effects of teacher education on both teacher practices in
the classroom and student performance, with the implication that changes in student performance
are at least partly the result of changes in teacher practices. However, it should be kept in mind that
such two-tiered relationships are inherently difficult to demonstrate. Given the complexities of
classrooms and the many factors that affect instruction, the research results presented in this study
should be considered as suggestive, not conclusive. Plans for Future Research
Walden University remains actively committed to sponsoring research that examines the impact of
its educational products and services. One of the goals of future research will be to replicate and extend the results of the study reported
here with graduates in other geographical areas. Such research is valuable both to Walden University
and to its students and stakeholders to provide verification of program quality for educators who
seek to improve their own teaching, and also as a tool for Walden to continue improving the
quality of its teacher-education offerings. 39 © 2005 Walden University References Cunningham, P.M., and Allington, R.L. (2003). Classrooms that work: They can all read and write.
Boston: Allyn and Bacon. International Reading Association, Professional Standards and Ethics Committee and the Advisory
Group to the National Council of Accreditation of Teacher Education Joint Task Force (1998).
Standards for reading professionals. Revised edition. Newark, DE: International Reading Association. Koslin, B.L., Zeno, S., and Koslin, S. (1987). The DRP: An effective measure of reading. New York:
The College Entrance Examination Board. National Reading Panel. (2000). Report of the National Reading Panel: Reports of the Subgroups.
Washington, D.C.: National Institutes of Health (NIH). Ogden, E.H. (2003). Linking teacher learning to student success: The impact of Walden University’s
master’s program in Elementary Reading and Literacy, A distance-learning program. Los Angeles:
Laureate Education, Inc. Ogden, E.H., and Jirele, T. (2001). Understanding the components of effective teaching: An assessment
of the Marygrove College Master in the Art of Teaching distance learning degree program. Los Angeles:
Canter & Associates. Ogden, E.H., and Secolsky, C. (2002). The role of the master degree in teacher effectiveness:
The impact on teacher classroom performance of the Marygrove College Master in the Art of Teaching
distance learning degree program and traditional master degree programs. Los Angeles:
Canter & Associates. United States Department of Education, Office of Educational Research and Improvement (1999).
National Review Panel elementary scoring guidelines 1998–1999: Blue Ribbon Schools Program. 40 © 2005 Walden University Endnotes 1. r = .08. 2. The DRP, published by Touchstone Applied Science Associates, is a paper-and-pencil test that yields both criterion-referenced and norm-referenced interpretations. The norm-referenced
interpretations were used in this study. Studies of reliability of the DRP yield pretest-posttest
correlations between r = .83 and r = .88 (Koslin, Zeno, and Koslin 1987). Other studies have
shown that DRP tests are sufficiently sensitive measures of change to be used in studies and
have sufficiently low measurement error to detect gain over a school year (Koslin, Zeno, and
Koslin, 1987). 3. Limitations listed here include issues mentioned by the researcher and by independent experts from Pacific Resources for Education and Learning (PREL) who were invited to review the
study. 4. Mann-Whitney U = 97.5, z = -2.762, p = .01 (two-tailed test). 5. Strategies identified in the checklist represent a composite of strategies typically associated with research-based reading instruction by a variety of sources, including the National Reading Panel
(2000) and Cunningham and Allington (2003). 6. For phonics, the pre-post mean gain was 4.03, sd = 1.11 (t = 3.64, df = 31, p = .00). For fluency, the mean gain was 3.16, sd = .62 (t = 5.11, df = 31, p = .00). For guided reading, the
mean gain was .56, sd = .28 (t = 2.01, df = 31, p = .05). For vocabulary, the mean gain was
7.34, sd = 1.33 (t = 5.53, df = 31, p = .00). For comprehension, the mean gain was 5.78, sd =
1.14 (t = 5.06, df = 31, p = .00). 7. Overall, the pre-post mean gain was 9.72, sd = 15.46 (t = 14.59, df = 538, p = .00). For grade 1, the pre-post mean gain was 27.35, sd = 20.57 (t = 11.52, df = 74, p = .00). For grade 2, the pre-
post mean gain was 11.45, sd = 13.69 (t = 9.69, df = 143, p = .00). For grade 3, the pre-post
mean gain was 9.02, sd = 10.39 (t = 8.50, df = 95, p = .00). For grade 4, the pre-post mean gain
was 2.24, sd = 11.42 (t = 2.26, df = 131, p = .03). For grade 5, the pre-post mean gain was 4.70,
sd = 10.42 (t = 4.32, df = 91, p = .00). 8. Overall, the pre-post mean gain was 7.33, sd = 19.99 (t = 7.99, df = 266, p = .00). For grade 1, the pre-post mean gain was 15.86, sd = 19.51 (t = 5.52, df = 45, p = .00). For grade 2, the pre-
post mean gain was 10.33, sd = 15.48 (t = 4.95, df = 53, p = .00). For grade 3, the pre-post
mean gain was 7.95, sd = 12.98 (t = 5.37, df = 76, p = .00). For grade 4, the pre-post mean gain
was .15, sd = 10.45 (t = 0.12, df = 64, p = .12). For grade 5, the pre-post mean gain was 1.54,
sd = 8.99 (t = 1.84, df = 23, p = .41). 9. Overall, t = 2.06, df = 803, p = .04. For grade 1, t = 3.04, df = 119, p = .00. 10.The United States Department of Education (1999), in its guidelines for assessing student achievement in schools nominated for national recognition, has designated an effect size of .33 41 © 2005 Walden University (one-third standard deviation) or greater as educationally significant, while an effect size of .50
or greater is considered exemplary. Effect sizes were calculated using methods specified by the
National Reading Panel (2000). 11.Effect sizes for the experimental group: 1.60 at grade 1, .58 at grade 2, .44 at grade 3, .13 at grade 4, .25 at grade 5. Effect sizes for the comparison group: .81 at grade 1, .60 at grade 2, .36
at grade 3, .01 at grade 4, .09 at grade 5. 42 © 2005 Walden University 43 © 2005 Walden University Linking Teacher Learning to Student Success: The Impact of Walden University’s Master’s Program in Elementary Reading and Literacy A Distance-Learning Program Evelyn Hunt Ogden, Ed.D. December 2003 Complete Study 44 © 2005 Walden University Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Purposes of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Design of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Walden University Master’s Degree in Reading Program . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 The Structure of the Walden University Master’s Degree in Reading Program. . . . . . . . . . . 52 Study Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Methods and Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Analysis of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Limitations of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Results—Impact on Classroom Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Results—Impact on Reading Achievement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 45 © 2005 Walden University Abstract The Impact of the Walden University Master of Science Degree with a Focus on Elementary
Reading and Literacy on Teacher Performance and Student Achievement in Reading Evelyn Hunt Ogden, Ed.D. The ultimate demonstration of effective teaching is student achievement. During the past 45 years,
hundreds of studies have examined the role that teaching behaviors, abilities, and practices play in
learning outcomes for students. Some of these studies, such as those focusing on teacher personality
traits and class size, have yielded few consistent findings. However, a strong body of research
suggests that high-performing teachers differ in systematic, measurable ways from their less
successful colleagues. Furthermore, there are well-established links between effective teaching
practices and student achievement. An independent research consultant was contracted to study the impact of the Walden University
Master of Science in Education degree with a focus on Elementary Reading and Literacy on teacher
performance and student achievement in reading. The year-long study involved 46 teachers and 805
students from 14 elementary schools in two large districts in the state of Washington. The purposes
of the study were to determine
• the extent to which those completing the degree used the practices associated with the effective teaching of reading; • if there were gains in reading achievement, beyond those that would be expected in the course of a school year, made by the students of those teachers completing the program; and • if the achievement of students taught by teachers enrolled in the Walden University program would equal or exceed the achievement of students of traditional on-campus master’s degree
graduates. The study followed a quasi-experimental design. The samples consisted of two groups of teachers:
one (the experimental group) that enrolled in and ultimately completed the Walden reading master’s
degree program; and one (the control group) that, prior to the study, had completed master’s degree
programs in education with a concentration in reading from other universities. The study resulted in two main findings:
• Walden University teachers changed how they taught reading. They documented increased use of tested strategies in each of the areas (phonics, fluency, guided reading, vocabulary, and
comprehension) identified by the National Reading Panel (2000) as effective in teaching reading.
During the same period, teachers with master’s degrees from traditional university programs
made changes solely in their use of strategies to teach fluency. • Walden University teachers gave a high rating (8.3 on a 10-point scale) to the Walden University master’s degree program for positive impact on their classroom instruction. In comparison,
teachers with master’s degrees from other universities rated their graduate programs significantly
lower (6 on a 10-point scale) regarding classroom impact. 46 © 2005 Walden University The study provided persuasive evidence that the Walden master’s degree program not only had a
positive impact on teachers, but also, through the teachers, on the students they taught. Student
achievement results were viewed from three perspectives: statistical significance, effect size and
educational significance, and relative placement of mean scores in relationship to national norms.
Viewed through each of these lenses, students in classes taught by teachers completing the Walden
University master’s degree in reading program
• made significant overall gains, as well as significant gains in each of the five grade levels, beyond those gains expected, based on fall and spring norms; • exceeded or equaled the achievement of students in classes taught by teachers with traditional master’s degrees from other universities; • made exemplary, educationally significant gains in reading ability that exceeded .5 standard deviations above the expected spring norms; and • moved as a group from the 40th mean national percentile in the fall to the 59th mean national percentile in the spring. 47 © 2005 Walden University Introduction The ultimate demonstration of effective teaching is student achievement. During the past 45 years,
hundreds of studies have examined the role that teaching behaviors, abilities, and practices play in
the learning outcomes of students. Some of these studies, such as those focusing on teacher
personality traits and class size, have yielded few consistent findings. However, a strong body of
research suggests that high-performing teachers differ in systematic, measurable ways from their less
successful colleagues. Furthermore, there are well-established links between the teacher’s knowledge
in the content area and knowledge of how to teach the subject, and student engagement and
achievement. The No Child Left Behind (NCLB) legislation has renewed the commitment of school districts to
improving teacher effectiveness and increasing student achievement. The legislation provides
funding for vital programs; however, it also carries serious consequences for schools that fail to meet
established standards. NCLB takes the position that three strategies are necessary to improve
student achievement: 1. Strategies and programs used in schools should be research-based.
2. Teachers of core subjects (including reading/English) must meet high-quality teacher (HQT) content knowledge standards. 3. Uniform standards for student achievement must be set by each state, with a timetable of escalating standards designed to reach the goal of 100% of students meeting the standards within
12 years. Each subgroup of students (special education, limited English proficiency, ethnic/racial,
low socioeconomic status) must meet the same academic standards. Schools in which students as a
whole or, in any subgroup, do not currently meet the achievement standards must make specified
annual yearly progress (AYP) toward achieving the standards. Failure to make these gains results in
consequences, including school choice, required tutoring, and school restructuring (Riddle, 2003). Studies show that both the kind and extent of professional development matter for teaching practice
and student achievement (Darlene-Hammond, 2000). A review of 60 achievement studies
concluded
that spending on teacher education was the most productive investment for schools, outstripping
the effect of teacher experience and reduced pupil/teacher ratios (Greenwald, Hedges, & Laine,
1996). Master’s degree programs represent an extensive form of staff development. Possession of a master’s
degree has long been highly valued in the field of K–12 education and perceived as an indicator of
effective teaching and higher student achievement by boards of education, teachers, and the public.
Teachers cite improving their teaching skills as the most important reason for pursuing a master’s
degree (Ogden & Jirele, 2001; Ogden & Secolsky, 2002). Boards of education and states report the
number of teachers with graduate degrees as an indicator of the quality of schools. School districts
across the country recognize the value of graduate degrees through higher salaries and often
subsidize tuition for teachers enrolled in graduate programs. In some states, teachers are required to
obtain a master’s degree within so many years of beginning their teaching career. With such value 48 © 2005 Walden University placed on the master’s degree, it is logical that the quality of advanced degree programs, in terms of
improving teaching and increasing student learning, is of major concern to the teachers who enroll
in the programs and to the districts that seek to improve classroom teaching by supporting graduate
study. Although some teachers come to the teaching profession with a master’s degree, more commonly
teachers pursue graduate work on a part-time basis while continuing to teach. Limited by
geography, teachers have sought out and enrolled in graduate programs within commuting distance
from home and school. In the past three decades, new options have emerged in the form of
distance-learning graduate programs, unbounded by geographical concerns. However, questions
arise from both teachers and school districts concerning the quality of a distance-learning degree
program, its potential to improve teaching, and the relative merits of the distance-learning program
compared with traditional on-campus master’s degree programs. The goal of the Walden University M.S. with a focus on Elementary Reading and Literacy is to
provide an intensive, high-quality master’s degree program built on research concerning reading,
teaching, and learning, through multiple learning modalities, that does not require attendance at
on-campus classes and that results in graduates who are more effective in the classroom and whose
students have higher levels of achievement. Course strategies and materials were designed by Walden
University to link theory and practice in three ways: (a) educational experts present the latest
research; (b) educational theory is modeled by master teachers in classroom settings; and (c) texts,
readings, and study guides encourage peer study groups of practicing teachers to reflect on their
teaching and to practice strategies in the classroom. Walden University has over 30 years of experience in providing graduate-level distance-learning
opportunities. The university offers master’s and doctoral degree programs, is accredited by The
Higher Learning Commission, and is a member of the North Central Association, 30 North LaSalle
Street, Suite 2400, Chicago, Illinois, 60602, 312-263-0456. 49 © 2005 Walden University Purposes of the Study The purpose of this study was to explore the impact of the Walden University M.S. with a focus on
Elementary Reading and Literacy (a distance-learning program) to determine
• the extent to which those completing the degree used the practices associated with the effective teaching of reading; • if there were gains in reading achievement, beyond those that would be expected in the course of a school year, made by the students of those teachers completing the program; and • if the achievement of students taught by teachers enrolled in the Walden University program would equal or exceed the achievement of students of traditional on-campus master’s degree
graduates. Design of the Study Evelyn Ogden, an independent research consultant, was contracted to conduct the study. The study
followed a quasi-experimental pre-post design. Two groups of teachers from two school districts and
14 elementary schools participated in the study. One group of teachers (the experimental group)
was completing the Walden reading master’s degree program, and one group (the control group)
consisted of teachers who had completed, prior to the study, master’s degree programs in education
with a concentration in reading from other universities. The students of each group of teachers
constituted the student treatment and control groups. Walden University Master’s Degree in Reading Program The program includes six reading-specific courses that are aligned with the research-based
recommendations found in the Report of the National Reading Panel (2000). The courses are designed
to provide teachers with the knowledge, skills, and tools to implement an effective, developmentally
appropriate, balanced literacy program. Teachers learn a broad range of tested instructional and
assessment strategies to meet the range of student needs in their classrooms. Each course focuses on
developing the teacher’s ability to assess and monitor student progress, and to select from their
expanded repertoire of literacy development strategies those most appropriate and likely to increase
student achievement. Instructors include nationally known educators in the field of reading: Dorothy S.
Strickland, David Rose, Richard Allington, Dolores Gaunty-Porter, Gerald D. Duffy, Peter Afflerbach,
Susan Mandel Glazer, Carol Minnick Santa, Richard Allington, Donald Bear, Susan Watts-Taffe, and
Patricia M. Cunningham. 50 © 2005 Walden University Table 1
Correlation of the Content of Walden University Reading Courses and the Findings of the National
Reading Panel on Evidence-based Reading Instruction National Reading Panel Findings
and Conclusions* Conclusion concerning teacher preparation and
comprehension strategies instruction Teaching reading comprehension strategies to students at
all grade levels is complex. Teachers not only must have
a firm grasp of the content presented in text, but also
must have substantial knowledge of the strategies
themselves, of which strategies are most effective for
different students and types of content and of how best
to teach and model strategy use (pp. 15–16). Research studies showed that teachers required
instruction in explaining what they are teaching,
modeling their thinking processes, encouraging student
inquiry, and keeping students engaged.
Data…indicated clearly that in order for teachers to use
strategies effectively, extensive formal instruction in
reading comprehension is necessary…(p. 16) Walden University Reading Courses † Foundations of Reading and Literacy provides
foundational knowledge and principles relevant to
reading instruction taught in subsequent courses. The
course includes study of historical and contemporary
perspectives on reading, implications of brain research,
and introductions to language development, reading
processes, and selection of text material. Supporting the Struggling Reader is designed to give
teachers a working knowledge of common reading
difficulties, methods for diagnosing those difficulties,
and guidelines for accessing appropriate resources
to provide instructional support for students. Informal
diagnostic tools are introduced. Research-based
intervention programs, including classroom
interventions, are discussed. Guidelines are offered for
communicating with parents and other members of
the school community regarding a child’s reading
difficulties. Planning and Managing the Classroom Literacy Program
provides guidelines for planning, organizing, and
managing a balanced literacy program. Flexible
grouping is introduced as a structure for differentiating
instruction to meet the varying needs of students.
Recommendations are provided for combining and
implementing strategies in all areas of literacy instruction.
Techniques for time-management, instructional pacing,
and transitions, as well as activities to establish a
language-rich classroom environment are discussed.
Guidelines for establishing a classroom climate in which
all students feel comfortable taking risks necessary for
literacy development are provided. Collaborative Action Research is a tool to assist teachers
in the investigation of school or classroom-based
problems. Teachers in the course study modifications
and expansion of theories related to instruction and
learning. 51 © 2005 Walden University Conclusions of the reading panel concerning
research-based strategies by topic: Phonemic awareness—Explicit instruction in
phonemic awareness improves students’ phonemic
awareness, reading, and spelling. Phonics—Systematic phonics instruction produces
significant benefits for students in kindergarten
through grade 6 and for children having difficulty
learning to read. Fluency—Guided oral reading that includes
guidance from teachers, peers, or parents has
a positive impact on word recognition, fluency,
and comprehension across grade levels. Vocabulary—Both direct and indirect vocabulary
instruction leads to gains in reading comprehension. Comprehension—A combination of strategies
(comprehension monitoring, cooperative learning,
use of graphic and semantic organizers, question
answering, question generation, and story structure
and summarization) is necessary to improve reading
comprehension. Strategies for Literacy Instruction, Part 1 focuses
specifically on the area of word study. Key topics such
as the stages of developmental word knowledge, the
roles of phonemic awareness and phonics instruction
in an effective reading program, the elements of
vocabulary instruction that promote active and
independent reading and learning, and the
importance of fluency are covered in depth.
Numerous strategies for explicit instruction in
phonemic awareness, phonics, spelling, vocabulary, and
building reading fluency are presented. Strategies for Literacy Instruction, Part 2 presents
effective strategies for facilitating and monitoring
students’ comprehension throughout the reading
process. Key topics explored include comprehension
prerequisites, strategies to promote active reading
behaviors, and the role of ongoing assessment.
Pre-, during, and post-reading strategies are presented.
The reading-writing connection and the writing
process are introduced. * Report of the National Reading Panel: Teaching Children to Read: An Evidence-based Assessment of the Scientific Research Literature on Reading and Its Implications for Reading Instruction. D. N. Langenberg, Chair, et al. National Institutes of Health NIH Pub.
No. 00-4769, Dec. 2000. † Walden University course catalog, Master of Science in Education with a focus on Elementary Reading and Literacy, 2001. 52 © 2005 Walden University Table 2 Course Descriptions of the Four Walden University Courses Focusing on How Children Learn and on Effective
Teaching Practices The Walden University program also includes four other research-based courses that focus on
understanding how children learn and on effective teaching practices. Instructors include Art Costa,
David Johnson, Sue Teele, Patricia R. Wolfe, Harvey Silver, Bena Kallick, Jo Gusman, and Carlos
Cortes. The Structure of the Walden University
Master’s Degree in Reading Program Students accepted for the master’s program must have a bachelor’s degree from a regionally
accredited institution, a teaching certificate or two years of teaching experience, and access to a
classroom where they can apply the strategies taught in the program. Students are required to work
in collaborative peer study groups of three to eight students taking the same courses. These study
groups are designed to provide practice, feedback, and coaching for participants. Most groups are
formed within a school or district. Each of the 10 graduate courses includes approximately 15 instructional videos. Featured in each
video are lectures and interviews with nationally known experts in reading, and instructional
practice and classroom demonstrations by practicing master educators. Study guides include
assignments for the peer group designed to help students integrate concepts learned into everyday Teacher as Professional examines teacher efficacy and develops skills and strategies that enable teachers to
perform at high levels. Effective Teaching Using Learning Styles and Multiple Intelligences prepares teachers to design more
effective instruction to enable all students to learn. Two learning models, multiple intelligences, and
learning styles are explored and integrated in the context of implementation into instruction,
curriculum, and assessment. Habits of Mind: Thinking Skills to Promote Self-Directed Learning focuses on “intelligent behaviors,” also
known as “habits of mind,” identified in the work of Art Costa. These behaviors, which characterize the
attributes of self-directed learners and critical thinkers, are explored in depth. The course emphasizes
application of these behaviors in three educational contexts: the teacher’s professional growth, the
students’ growth as leaders, and the growth of the learning organization as a whole. Instructional Models and Strategies introduces several models of instruction that are foundational for
building an effective teaching practice to meet diverse learning needs. These research-based models work
in concert with each other to evoke different types of thinking processes in students, providing
opportunities for high-level learning and achievement. Strategies that integrate and balance these models
are presented for designing curriculum, instruction, and assessment. 53 © 2005 Walden University teaching. Appropriate team assignments are sent to a faculty mentor for evaluation. The study
guides include specific classroom application assignments to ensure that students practice and apply
concepts in the classroom. Textbooks are also used to extend the concepts taught in each course.
Assessment of the effectiveness of implementation is provided through study team member analysis
and the required portfolio. An essential component of the distance-learning model is peer feedback.
In addition, Walden University faculty members communicate regularly with students through phone,
fax, and email. Assessment of student progress is both formative and summative. At the conclusion of the two-year graduate program, each student must have successfully completed
each required course and submitted a professional portfolio documenting accomplishment of the
identified program outcomes. Study Groups Participants in the study were 46 teachers in two large school districts, Peninsula and Tacoma, in the
state of Washington. The participants were full-time, grade 1–5 teachers representing 14 elementary
schools. The Walden teachers (the treatment group) consisted of 32 teachers who enrolled in the Walden
University reading master’s degree program starting in the summer 2002 term and who completed
the program in the fall 2003 term. The Walden teachers had previously completed an average of 2.8
undergraduate reading courses as part of their preparation for teaching. They rated the impact of
these courses on their reading instruction in the classroom an average of 5.8 on a 10-point scale.
The range of teaching experience was 1 to 26 years, with an average of 7.5 years teaching
experience. The Walden teachers had completed two of their graduate courses, Teacher as Professional and
Foundations of Reading and Literacy, prior to the administration of the student pretest. During the
year and prior to the posttest they completed seven additional graduate courses, including three
additional courses directly related to reading instruction. The teachers took the final course,
Planning and Managing the Classroom Literacy Program, after the administration of the student
posttest and postteacher survey, in June 2003. Therefore the treatment group did not have the
advantage of the total program during the school year and the duration of the study. Participants in the control group were 14 teachers with master’s degrees in education from
institutions other than Walden University. These teachers had an average of 2.5 undergraduate
reading courses and an additional 3.4 graduate courses in reading. They rated the impact of these
courses on their teaching of reading as 5.3 and 7.0 respectively. The range of teaching experience
was 2 to 30 years, with an average of 17 years teaching experience. Control teachers had more years
of teaching experience; however, in this study, years of experience was an insignificant factor related
to achievement of students (r=.08). 54 © 2005 Walden University Student participants were 805 children in the reading classes of the treatment and control teachers
during the 2002–2003 school year. The profiles of the students indicated that both treatment and
control groups were similar in terms of socioeconomic background, with the majority of students in
classes considered as low socioeconomic status, based on the percentage of students receiving free
lunch. Table 3
Classification of Classes by Socioeconomic Status of Students Socioeconomic Makeup of Classes Treatment % of Students Control Low SES >33% in class 75% 77% Mid SES 12% 13% High SES <10% in class 13% 10% The majority of classes of both treatment and control teachers included general education students
as well as some classified and/or ESL (English as a second language) students. In four cases there was
an in-class support special education teacher. In these classes the treatment or control teacher was
the general education instructor. Table 4
Classification of Classes by Types of Students Type of class Treatment % by type Control % by type Only general education 9% 7% General and special education 59% 57% In-class support 9% 14% General and ESL 22% 21% Methods and Data Collection The evaluator conducted the study independently. The methodology consisted of pre- and
posttesting of student reading achievement, responses to questionnaires, and reports of classroom
activities. Instrumentation The Degrees of Reading Power (DRP) Primary (grade 1) and Standard (grades 2–5) test forms were
administered to all children in the classes of Walden and control teachers in September 2002 and
again in June 2003. The DRP, published by Touchstone Applied Science Associates, is a paper-and-pencil test that yields
both criterion-referenced and norm-referenced interpretations. The norm-referenced interpretations 55 © 2005 Walden University were used in this study. Studies of reliability of the DRP yield pretest-posttest correlations between
r =.83 and r =.88 (Koslin, Zeno, & Koslin, 1987). Other studies have shown that DRP tests are
sufficiently sensitive measures of change to be used in studies and have sufficiently low
measurement error to detect gain over a school year (Koslin, Zeno, & Koslin, 1987). Three surveys were used to gather information from teachers concerning the students in their
classes: strategies utilized in instructing reading, background courses in reading instruction, and
ratings of the impact of such programs on their classroom instruction. The surveys were
administered to the teachers in June 2002, October 2002, and June 2003. Analysis of Results Several types of analyses were conducted on the data collected in this study. Data for each study
group were summarized and compared in order to establish comparability between groups. NCE
scores based on Degrees of Reading Power (DRP) derived fall and spring norms were used to
calculate gains in achievement. Student achievement comparisons were done using t tests for total
grade 1–5 data and for each grade level. In addition to determining statistical significance of differences between groups, differences were
also assessed in terms of effect size and educational significance. Effect sizes of gains were calculated
using the standardized mean difference formula: (M1 –M2)/0.5 (sd1 + sd2), where M1 and M2 are
the means of respective treatment and control group or pre- and posttests, and sd1 and sd2 are the
respective standard deviations of the treatment and control or pre- and posttests (Report of the
National Reading Panel, 2000, p. 32). This analysis measures student gains on standardized tests
regardless of initial level of performance. Resultant estimates of student and teacher effects have
been demonstrated to be independent of socioeconomic effects (Sanders & Horn, 1995a, 1995b;
Wright, Horn, & Sanders, 1997). Educational significance is an estimate of the educational importance of statistically significant
changes in terms of actual student performance. In studies involving large numbers of students,
small differences in means can be statistically significant; however, these differences may have little
significance in terms of actual student performance in the classroom. In this study, educational
significance was evaluated using the standards established by the U.S. Department of Education to
assess the educational value of student gains. An effect size of .33 (one-third standard deviation) or
greater was judged to be educationally significant, and an effect size of .50 (one-half standard
deviation change) or greater was judged an “exemplary” change (U.S. Department of Education,
1999). Summary data were provided for questionnaire data. This information consisted of frequency
distributions, means, and standard deviations. In some cases, t tests were used to determine if
differences existed between groups and from pre- to posttreatment. In addition, all data were
checked for accuracy and corrected for data-entry errors and coding errors. 56 © 2005 Walden University Limitations of the Study Certain limitations in the study design and implementation may have affected the outcomes.
Teachers are often reluctant to participate in studies that require time to respond to questionnaires
and involve multiple testings of their students. While 32 of the teachers enrolled in the Walden
master’s degree in reading program participated in the full study, 23 other teachers did not
participate in the student assessment part of the study. These included kindergarten teachers who
were excluded from the study because the districts did not formally assess kindergarten students,
fifth-grade teachers from one of the districts that would not allow pre/post testing of fifth-grade
students, and a few teachers enrolled in the program who taught higher grades or were not assigned
to classrooms. The data necessary to explore possible differential effects of the program on subgroups of students
were not available since disaggregated test data and accountability are required under NCLB. Another factor that may have affected the outcomes of the study was that the teachers began the
Walden reading master’s degree program in the summer of 2002 and did not complete the
program until December 2003. Therefore, these teachers did not have the benefit of the entire
program during the 2002–2003 school year. Results Impact on Classroom Practices
Teachers were asked if they had made changes in the ways they taught reading and, if so, to describe
those changes. In response, 90.6% of the Walden teachers said they had changed how they had
taught reading during the 2002–2003 school year and provided specific written examples of the
changes they had made, compared with 50% of the control teachers. Further, 87.5% of the Walden
teachers stated that they planned to make additional changes in how they would teach reading
during the next school year, compared with 57% of the control teachers. Table 5
Changes and Planned Changes in Instruction of Walden Master’s Degree Teachers Group N % Changed # Changed Changed their Reading Treatment 32 90.6% 29 Instruction 2002–2003 Control 14 50% 7 Plan to Change their Reading Treatment 32 87.5% 28 Instruction 2003–2004 Control 14 57% 8 In the summer of 2002, teachers were asked to identify strategies in the research-based areas of
phonics, fluency, vocabulary, guided reading, and comprehension that they had used in the
classroom during the last month of the previous school year. In the spring of 2003, the teachers
were asked the same questions, only their responses were based on the last month of the current
school year. 57 © 2005 Walden University The Walden teachers significantly increased their use of strategies in phonics, fluency, guided
reading, vocabulary, and comprehension—all areas identified by the National Reading Panel as
effective in teaching literacy. In contrast, during the study period, the control teachers reported a
significant increase only in their use of fluency development strategies. Table 6
Pre/Post Use of Research-based Reading Development Strategies by Walden Teachers Pre/Post Mean Strategies N Difference sd t Sig. (2-tailed) Phonics 32 4.03 .28 3.64 .00* Fluency 32 3.16 .62 5.11 .00* Guided Reading 32 .56 .28 2.01 .05* Vocabulary 32 7.34 1.33 5.53 .00* Comprehension 32 5.78 1.14 5.06 .00* * Statistically significant Table 7
Pre/Post Use of Research-based Strategies by Control Teachers Pre/Post Mean Strategies N Difference sd t Sig. (2-tailed) Phonics 13 1.31 .75 1.75 .10 Fluency 13 2.23 .75 2.96 .01* Guided Reading 13 .38 .51 .75 .47 Vocabulary 13 1.00 1.94 .52 .62 Comprehension 13 .23 1.27 .18 .86 * Statistically significant Teachers were asked to reflect on and rate the actual impact they believed their master’s degree
programs had on their instructional practices in the classroom. In terms of impact on their teaching,
Walden program teachers rated their master’s degree program significantly higher than the control
teachers rated their programs, 8.3 and 6.2 respectively on a 10-point scale. Table 8
Ratings of the Impact of Master’s Degree Programs Practices Program N Mean Sig. Walden Master’s Degree Program 32 8.3 .01* Other Master’s Degree Programs 14 6.2 * Statistically significant 58 © 2005 Walden University Finally, the Walden teachers were asked to rate the individual courses they had taken in terms of
course impact on classroom instruction. The highest ratings were given to those courses that dealt
most directly with strategies for teaching reading. Table 9
Rating of the Impact of Each Course in the Walden Master’s Degree Program on Classroom Practice Walden Master’s Degree Courses Mean Teacher as Professional 7.0 Foundations of Reading & Literacy 7.6 Learning Styles and Multiple Intelligences 7.6 Strategies for Literacy I 8.6 Habits of Mind 7.0 Strategies for Literacy II 8.7 Collaborative Action Research 5.3 Supporting the Struggling Reader 8.7 Instructional Models & Strategies 7.3 Planning and Managing the Classroom Literacy Program N/A* *Course taken after post-survey and student posttests Impact on Reading Achievement
Analysis of pre-posttest data shows that the Walden grade 1–5 students made overall significant gains,
based on appropriate fall-to-spring norms, and significant gains were made by the students in each
grade. The control students made significant gains overall and in individual grades 1–3 but not in
grades 4 and 5. Table 10
Comparison of Pre and Post Mean Gains for Treatment and Control Students Sig. Group N Mean sd t df (2-tailed) Treatment–PreNCE-PostNCE 539 9.72 15.46 14.59 538 .00* Control–PreNCE-PostNCE 267 7.33 19.99 7.99 266 .00* * Statistically significant 59 © 2005 Walden University Table 11
Comparison of Pre-NCE and Post-NCE Mean Gains for Treatment and Control Students by Grade Sig. Grade Group N Mean sd t df (2-tailed) 1 Treatment–PreNCE-PostNCE 75 27.35 20.57 11.52 74 .00* Control–PreNCE-PostNCE 46 15.86 19.51 5.52 45 .00* 2 Treatment–PreNCE-PostNCE 144 11.45 13.69 9.69 143 .00* Control–PreNCE-PostNCE 54 10.33 15.48 4.95 53 .00* 3 Treatment–PreNCE-PostNCE 96 9.02 10.39 8.50 95 .00* Control–PreNCE-PostNCE 77 7.95 12.98 5.37 76 .00* 4 Treatment–PreNCE-PostNCE 132 2.24 11.42 2.26 131 .03* Control–PreNCE-PostNCE 65 .15 10.45 0.12 64 .12 5 Treatment–PreNCE-PostNCE 92 4.70 10.42 4.32 91 .00* Control–PreNCE-PostNCE 24 1.54 8.99 1.84 23 .41 * Statistically significant Students of teachers enrolled in Walden University’s reading master’s degree program made
significantly greater gains in reading than the students of the control teachers. Table 12
Comparison of Mean Gains for Treatment and Control Students Sig. Group N Mean sd t df (2-tailed) Treatment 539 9.72 15.46 2.06 803 .04* Control 266 7.36 15.01 * Statistically significant Analysis by individual grade levels showed that first-grade students of the Walden teachers scored
significantly higher than their control counterparts. However, there were no statistically significant
differences between treatment and control students based on individual grade analyses. 60 © 2005 Walden University Table 13
Comparison of Mean Gains for Treatment and Control Students by Grade Sig. Grade Group N Mean sd t df (2-tailed) 1 Treatment 75 27.35 20.57 3.04 119 .00* Control 46 15.86 19.50 2 Treatment 144 11.05 13.69 .23 196 .82 Control 54 10.52 15.56 3 Treatment 96 9.05 10.39 .62 171 .54 Control 77 7.95 12.98 4 Treatment 132 2.24 11.42 1.24 195 .22 Control 65 .15 10.45 5 Treatment 92 4.68 10.41 1.35 114 .18 Control 24 1.54 8.99 * Statistically significant The educational significance of scores (i.e., the actual relevance to students in the classroom) is
estimated by the size of the effect. The U.S. Department of Education, in its guidelines for assessing
student achievement in schools nominated for national recognition, has set two standards for
judging the educational significance of gains in achievement, based on effect size. An effect size of
.33 (one-third standard deviation) or greater is considered educationally significant, whereas an
effect size of .50 or greater is considered exemplary. Using these standards, the overall gains made by
the Walden students were educationally significant at the exemplary level. As in the other analyses,
primary students made the most improvement. First- and second-grade students of Walden teachers
made exemplary-level educationally significant gains, 1.6 and .81 respectively, during the school
year. Exemplary-level gains were also made by the control group students in first and second grades.
Both treatment and control students in grade 3 made educationally significant gains. Table 14
Effect Size and Educational Significance of Gains for Students of Walden and Control Teachers Pretest Posttest Effect Ed. Pretest Posttest NCE NCE size of Sig. > Group N sd sd Mean Mean gain .50 Treatment 539 18.45 19.58 44.8 54.4 0.51 * Control 266 18.45 21.02 41.8 49.1 0.37 † * Education significance >.50 standard
† Education significance >.33 standard 61 © 2005 Walden University Table 15
Effect Size and Educational Significance of Gains for Students of Walden and Control Teachers by Grade Pretest Posttest Pretest Posttest Effect Grade Group N sd sd Mean Mean size 1 Treatment 75 14.57 19.57 33.47 60.82 1.60* Control 46 18.03 21.32 42.41 58.27 0.81* 2 Treatment 144 18.37 19.53 42.25 53.30 0.58* Control 54 15.99 18.47 40.06 50.38 0.60* 3 Treatment 96 20.52 20.57 48.12 57.14 0.44 † Control 77 21.34 22.31 40.36 48.31 0.36 † 4 Treatment 132 16.02 17.36 46.61 48.85 0.13 Control 65 18.23 20.34 42.80 42.95 0.01 5 Treatment 92 17.91 18.25 51.65 56.25 0.25 Control 24 15.27 18.48 46.04 47.58 0.09 * Education significance >.50 standard
† Education significance >.33 standard Another way to view changes in student achievement is to consider how the study groups
performed in comparison with the norm reference group for the Degrees of Reading Power (DRP).
It is assumed that with a year’s instruction and maturation students will increase their reading
ability. Therefore, norms for the fall are based on the performance of a large sample of students at
that time. Spring norms are based on a national sample of students who took the DRP in the
spring. Scores are equated for the test over time. National percentile (NP) scores are most often
used in reporting student achievement on standardized tests to parents and for reporting school
achievement to the community. They show the relative position of the student or group mean score
compared with national norms. All standardized tests have a mean NP of 50. Results showed that at the beginning of the study, in the fall of 2002, the student means for both
the treatment and control groups were below the 50th NP. The spring test results showed that
the mean for the Walden student group was at the 59th NP, meaning that the Walden group
scored higher than 59% of the students nationally on the test. However, the control group
student mean remained slightly below the 50th NP. Table 16
Change in Pre/Post Mean National Percentile National National Percentile Percentile Group N Pretest Posttest Treatment 539 40th 59th Control 266 35th 49th 62 © 2005 Walden University Table 17
Change in Pre/Post Mean National Percentile by Grade National National Percentile Percentile Grade Group N Pretest Posttest 1 Treatment 75 21st 69th Control 46 36th 65th 2 Treatment 144 36th 56th Control 54 32nd 51st 3 Treatment 96 47th 63rd Control 77 32nd 47th 4 Treatment 132 44th 48th Control 65 36th 36th 5 Treatment 92 53rd 62nd Control 24 44th 45th Conclusions The findings of the study support the conclusion that the Walden master’s degree with a focus on
reading had a positive impact on how teachers taught reading. They documented increased use of
tested strategies in each of the areas (phonics, fluency, guided reading, vocabulary, and
comprehension) identified by the National Reading Panel (2000) as effective in reading instruction.
During the same period teachers with master’s degrees from traditional university programs made
changes solely in strategies to teach fluency. In addition, Walden University teachers gave high
ratings (8.3 on a 10-point scale) to the Walden University master’s degree program in terms of
positive impact on their classroom instruction. However, teachers with master’s degrees from other
universities rated their graduate programs significantly lower (6 on a 10-point scale) in terms of
classroom impact. The study provided persuasive evidence that the Walden master’s degree program had a positive
impact not only on teachers, but also, through the teachers, on students they taught. Student
achievement results were viewed from three perspectives: statistical significance, effect size and
educational significance, and relative placement of student mean scores in relationship to national
norms. Viewed through each of these lenses, the performance of students in classes taught by
teachers completing the Walden University master’s degree in reading program exceeded or equaled
the achievement of students in classes taught by other teachers with traditional master’s degrees
from other universities. The students of Walden teachers made significant gains, beyond those expected, based on fall and
spring norms, overall, and at each of the five grade levels. Students of control teachers made
significant gains overall; however, analysis by grade showed significant gains only in grades 1–3. Overall, grade 1–5 students of teachers completing the Walden University master’s in reading
program significantly outperformed the students of teachers with master’s degrees from other
universities. 63 © 2005 Walden University Looking across the grades, the greatest improvements in achievement were made in the primary
grades, and the least in grades 4 and 5. Entering first-grade students bring little or no knowledge of
reading to the classroom; the impact of instruction can be and often is dramatic. As students
advance through the grades, their achievement is affected not only by their current teacher, but also
by all of their previous learning experiences. In the study, first-grade students of Walden teachers
statistically outperformed control students, while in the other grades there were no statistically
significant differences between the students taught by Walden University master’s degree teachers
and those with master’s degrees from other universities. The educational significance of scores, the actual relevance of student gains, is estimated by the size
of the effect. The overall pre-to-post reading gains made by Walden students were educationally
significant at the exemplary level, exceeding one-half standard deviation. As in the other analyses,
the primary students made the most improvement. First- and second-grade students of Walden
teachers made exemplary-level gains of 1.6 and .81 standard deviations, respectively. Third-grade
students made an educationally significant gain of .44 standard deviations. However, gains made by
fourth- and fifth-grade students (.13 and .25, respectively) were not found to be educationally
significant based on the criteria established by the U.S. Department of Education (1999). As a group, the Walden students made reading gains in comparison with the national norms for the
Degrees of Reading Power (DRP). In fall 2002, the Walden group mean at the 40th national
percentile was below the national mean percentile; in spring 2003, the group mean was at the 59th
national percentile, exceeding the national average for the DRP, while the posttest mean of the
control group was at the 49th national percentile. References Darling-Hammond, L. (2000, January). Teacher quality and student achievement: A review of state
policy evidence. Education Policy Analysis Archives, 8(1). Greenwald, R., Hedges, L. V., & Laine, R. D. (1996). The effect of school resources on student
achievement. Review of Educational Research, 66, 361–396. Koslin, B. L., Zeno, S., & Koslin, S. (1987). The DRP: An effective measure of reading. New York:
The College Entrance Examination Board. Langenberg, D. N., Chair et al. (2000). Report of the National Reading Panel: Teaching children to
read: An evidence-based assessment of the scientific research literature on reading and its implications for
reading instruction. National Institutes of Health (NIH) Pub. No. 00-4769. Ogden, E. H. & Jirele, T. (2001). Understanding the components of effective teaching: An assessment
of the Marygrove College Master in the Art of Teaching distance learning degree program. Los Angeles:
Canter & Associates. Ogden, E. H. & Secolsky, C. (2002). The role of the master degree in teacher effectiveness: The impact
on teacher classroom performance of the Marygrove College Master in the Art of Teaching distance
learning degree program and traditional master degree programs. Los Angeles: Canter & Associates. Riddle, W. (2003). K–12 Education: Highlights of the No Child Left Behind Act of 2001 (P.L.
107–110). Report for Congress, updated January 15, 2003. Sanders, W. L., & Horn, S. P. (1995). The Tennessee Value-Added Assessment System (TVAAS):
Mixed model methodology in educational assessment. In A. J. Shinkfield & D. Stufflebeam (Eds.),
Teacher evaluation: Guide to effective practice (pp. 337–350). Boston: Kluwer. United States Department of Education, Office of Educational Research and Improvement (1999).
National Review Panel elementary scoring guidelines 1998–1999: Blue Ribbon Schools Program. Wright, S. P., Horn, S. P., & Sanders, W. L. (1997). Teacher and classroom context effects on
student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in
Education, 11, 57–67. EVELYN HUNT OGDEN
Study Consultant
Evelyn Ogden received her Ed.D. from Rutgers University in the fields of educational and
psychological evaluation and statistics. She is the author of five books as well as major studies and
reports on effective and ineffective schools, programs that work, strategies to increase student
achievement, programs for at-risk students, and dissemination of successful practices. Dr. Ogden
served for many years on the U.S. Department of Education Blue Ribbon School Selection Panel.
She also served on the U.S. Department of Education Program Evaluation Panel, established by
Congress, which reviewed and accepted or rejected research-based claims of program effectiveness.
She served as deputy assistant commissioner for research, planning, and evaluation. Currently, she is
deputy superintendent of East Brunswick Public Schools, New Jersey. 64 © 2005 Walden University