Standard 2

Standard 2: 

The unit has an assessment system that collects and analyzes data on applicant qualifications, candidate and graduate performance and unit operations to evaluate and improve the performance of candidates, the unit and its programs. 

How does the unit (Newman University School of Education) use the assessment system to improve the performance of candidates and the unit and its programs?

Introduction

The Conceptual Framework [Exhibit CF.2b School of Education Conceptual Framework] provides a focus for the governance, curriculum design and teaching and learning, both initial and advanced, within the School of Education at Newman University and reflects the School of Education Mission. The School of Education seeks to inspire and encourage students to become caring, reflective practitioners who are intellectually and spiritually motivated to transform self, schools, and society. The School focuses upon the concept of the caring, reflective practitioner as integral to the possibility of transformative conditions within self, schools, and society, local and global.  In fulfilling the mission of educating caring, reflective practitioners, the School of Education expects graduates, initial and advanced, to be educators who are knowledgeable, caring, reflective, visionary, collaborative, and ethical. These six outcomes– knowledgeable, caring, reflective, visionary, collaborative, and ethical – are integrated into the preparation of candidates to meet the Professional Education standards established by the Kansas State Department of Education. Both initial and advanced programs are responsive to the mission of the School of Education and its relationship to the university’s mission. The liberal arts core, as well as professional and pedagogical studies and field experiences, are designed to graduate individuals who possess the aforementioned knowledge and skills, as well as the dispositions to act in accordance with these themes.  The unit assessment system serves as a monitoring system at strategic locations along the continuum from entry to the university to program completion and the completion of licensure requirements. 

Initial Program Process Key Decision Points

The decision point process for initial programs is provided in a systematic progression of clearly described steps and processes [Exhibit S2.1.a Initial Program Decision Point Matrix].  As noted in the matrix, the first decision process for the initial program (admission to the university) is addressed without direct involvement and simply insures that eligibility issues at the time of university admission have been verified.  In addition, a base of candidate information is available for future use.  Admission to the Teacher Education Program (TEP) represents the first involvement of the unit in the decision process [Exhibit S2.6.c Teacher Education Program Application Process]. This decision point combines a review of academic success (GPA), basic aptitude (assessment score review, external verification (references) and discussion of the conceptual framework (interview).  This balancing of critical factors continues through the process.  Admission to Teacher Internship Program (TIP) is the locus of decision point 3 and provides for a continuation of the balanced decision-making process but with more intensity [Exhibit S2.6.d Teacher Internship Program Application Process].  The inclusion of the academic review of student progress in education courses provides a cross reference to the assessment system for course review and standard consistency that is provided by the unit and provides verification of readiness for the internship process.  The hallmark of Decision Point 4 at the conclusion of the internship program is an in-depth and reflective view of total experiences with the combined involvement of candidate, cooperating teacher and the unit representative. Finally, Decision Point 5 combines two distinct processes; program completion and licensure.  Successful program completion provides for the institutional recommendation, one of three elements required for licensure.  The institutional recommendation for program completion combines with licensure assessment success to support the transition of the NU candidate into the world of work.  In all cases, the decision point process reinforces and supports the emphasis on the development and support of candidates who will be knowledgeable, caring, reflective, visionary, collaborative, and ethical educators. 

Advanced Program Process Key Decision Points

The decision points for advanced programs provide parallel steps but with accountability for a larger number of options [Exhibit S2.1.b Advanced Program Decision Point Matrix].  In Building Leadership and Reading Specialist emphasis areas, the programs are open for licensure only—individuals with previously earned advanced degrees in education—as well as degree completion students.  In addition, ESOL endorsement can be earned without degree completion but can also be a part of the M.S. Ed. degree in C & I with ESOL Emphasis.  Finally, the unit provides one degree program (M.S. Ed. degree in C & I with Accountability Emphasis) that is open to individuals who are not licensed teachers.   The first decision point for advanced program candidates (admission to the university) provides an illustration of this variability.  In all cases, this step confirms candidate involvement and provides documentation for additional steps.  Decision Point 2  provides the initial contact of the unit with candidates and also provides an example of program to program differences as ESOL endorsement only candidates need only provide evidence of a valid teaching licensure while candidates in other areas have responsibilities for providing more complete information.  GPA information for previous degrees and graduate work is reviewed as part of the advising process but there is no minimum requirement for matriculation.  Decision Point 3—admission to practicum experience requires a review of candidate success in previous courses in the program.  As part of that review process, previous course experiences and completed activities that serve as the basis for arranging a successful practicum experience are included in the review process.  Advanced Program Decision Point 4 provides for an in-depth review of portfolio documentation that has been developed as part of the practicum experience.  Successful completion of this terminal experience (practicum or research project is parallel with program completion except for one program (M.S. Ed. degree in C & I with ESOL emphasis). Finally, licensure is viewed as a decision process that is separate from program completion in the advanced program because:

  • One program (M.S. Ed. degree in C & I with accountability) does not provide licensure
  • One program (M.S. Ed. degree in C & I with ESOL emphasis) serves candidates who may have already satisfied licensure requirements
  • One program (M.S. Ed degree in Building Leadership) has work requirements which may not be satisfied at the time of program completion.

The goal of this process is to make certain that the educational program for advanced candidates supports the mission of the School of Education; to offer experiences and instruction, integrating sound educational theory, best practice and Christian values that empower students to use their knowledge and influence to transform schools and society.

Other aspects of the NU School of Education Assessment System

Attention to program consistency

The unit provides service in a number of different locations and the importance of program consistency—with maximum opportunities for students in all locations—is addressed through attention to assessment and feedback from the total system.

  • The initial program provides the elementary education major on campus and in outreach locations in both Southeastern and Southwestern Kansas.  In order to address program consistency, program data obtained through the assessment system is organized and reviewed for students who are served on the Newman University campus and for students who are served in outreach locations. Thus, all Elementary Program data is reported for “campus” and “outreach” categories as noted on all of the Elementary Education assessment reports.  Examples of this process in action are also noted in the “why is the data used?” section of this report.
  • The advanced program also serves students in a variety of locations and addresses the concept of program consistency in two different ways. 
    • On site instruction for the ESOL endorsement program is provided in a large number of separate community and/or school based programs. Separate support networks for enrollment and recruiting are provided for ESOL instruction in the Wichita area and in Western Kansas sites but resources that are provided to support independent instructional locations are consistent throughout the system.  The development of a protected web site support system for ESOL Adjuncts instructors (based on instructor feedback) is address in the “why is the data used?” section.  In addition, ESOL Licensure data is reported for “Wichita area” and “Western Kansas”.
    • A uniform instruction program is provided for all students in all locations for licensure programs in Building Leadership and Reading Specialist programs so the program data and feedback can be applied to all students in all locations in the same way.

 

Providing a voice for students

The unit provides a voice for students through attention to complaints, appeals and exceptions through effective utilization of Newman University policies and procedures [Exhibit CF.1 NU Catalog].  As noted on pages 40—42 of this publication, these statements provide a clear statement of procedures.  Within this system, the unit provides support in the forms of:

  • Making certain that procedures are communicated and that opportunities for student use of these procedure are not limited in the case of the many initial and advanced program candidates who are receiving instruction in different locations. 
  • Providing additional support through individual advising and through the work of the Admission and Progression Committee

The unique element that is provided by the unit to support the progression of initial program candidates is the Admission and Progression Committee [Exhibit S2.6.a Admission and Progression Committee Description].  The A & P Committee meets on a regular basis to address specific student questions and needs with a focus on decision points 2 (admission to teacher education) and 3 (admission to teacher internship).  The attention to detail provided by this process is illustrated by examples of the committee in action [Exhibit S2.6.b Illustrations of A and P function].  The role of providing attention to unique student concerns is somewhat different for advanced program candidates because admission to all programs (except for the M.S. Ed. degree in Curriculum and Instruction with Accountability emphasis) is limited to licensed teachers.  Many of these candidates need support of a different nature because virtually all candidates in advanced programs combine graduate study with full time employment.  Access to exceptions—when special circumstances dictate the need for changes in plans of study—and time extensions through the use of university policy for incomplete grades are critical support factors.  Effective use of opportunities for exception requests and time extensions (through the use of the incomplete grade policy) allow potential problems to be modified into problem solving examples.

An overview of the assessment system with attention to the concept, organization, operation and data collection aspects of the system

For initial candidates, a set of five decision points [Exhibit S2.1.a Initial Program Decision Point Matrix] are provided and the assessment system is designed to provide information which both verifies and informs to support candidate growth.  The first Decision Point—admission to the university—provides verification of candidates’ status within the university and does provide access to ACT score information which can be used to address an important requirement for the second decision point.  Decision Point 2 provides for the official entry into a teacher education program and provides a combination of assessment verification, task completion and verification of success within the university community [Exhibit S2.6.d Teacher Education Program Application Form].  The use of previous experience, process completion and assessment information [Exhibit S2.3.a PPST/ACT Entry Score Summary] in this decision step provides verification and indicate that test scores are not a limiting factor in this process.  Although Decision Point 3 seems to mirror the process of the previous point, the examination of candidate success in Education places an important focus on relationships of information. The matrices of courses, assessments and standards for programs of study for initial candidates [Exhibits S2.2.a, S2.2.b, S2.2.c, S2.2.d and S2.2.e] serve as the basis for review of assessment results that are paired with these matrices.  In some cases two sets of assessment are provided as changes were made and approved in fall 2012. Assessment summaries for Elementary Education candidates [Exhibits S2.3.c, S2.3.d, S2.3.f, S2.3.g, S2.3.h, S2.3.i, S2.3.j and S2.3.k] provide data over a previous three-year period and allow for examination of:

  • Program consistency when outreach and campus students are compared.
  • Uniformity of candidate success through the number of responses at different rubric categories.
  • Areas of program success and needs for improvement.

Summaries for secondary programs in English [Exhibit S2.2.c], History/Government [Exhibit S2.2.d] and Mathematics [Exhibit S2.2e] provide achievement and licensure information in single documents because small numbers of candidates are involved in these programs.  Achievement summaries are not provided for two dormant programs; Secondary Biology [Exhibit S2.2.f] and Secondary Chemistry [Exhibit S2.2.g]. Decision Point 4 increases the emphasis on objectively supported recommendations from cooperating teacher and unit supervisor with the use of the OAFTI (Observation and Assessment Form for Teacher Interns) [Exhibits S2.3.e.1, S2.3.e.2and S2.3.e.3].  This serves as an important contribution to the development of the institutional recommendation.  Decision Point 5 is a combination of two distinct processes as successful program completion provides for two—institutional recommendation and NUTPP completion—of the elements that are required for licensure.  Reports of the remaining elements for licensure can be seen in Praxis II Licensure scores [Exhibit S2.3.b], PLT Summary [Exhibit S2.3.l] and the NUTPP (Newman University Teaching Performance Portfolio) summary [Exhibit S2.3.m].  Thus, the decision making process is supported by an overlapping array of tools that combine objective verification with subjective content. 

For advanced candidates, a set of five decision points is provided [Exhibit S2.1.b Advanced Program Decision Point Matrix] as a method of systematically reviewing the total process. The first Decision Point—admission to the university simply provides the basis for developing a file of information that can be used during the matriculation of the candidate.  Decision Point 2 requires the development of a plan of study and provides a basis for matching prior educational experiences with the development of academic plans.  Only one measure—GPA verification—is introduced at this stage and is provided as the basis of future planning and advising rather than as a decision making factor.  Decision Point 3 is a critical step in the process and requires a careful review of the program of study to verify the successful completion of experiences in preparation for the practicum experience.  The matrices of courses, assessments and standards for programs of study for advanced candidates [Exhibits S2.2.i ESOL Matrix of Assessments and Standards, S2.2.h Building Leadership Matrix of Assessments and Standards and S2.2.j Reading Specialist Matrix of Assessments and Standards] serve as the basis for review of assessment results that are paired with these matrices [Exhibits S2.5.a Building Leadership Assessment Summary, S2.5.b ESOL Assessment Summary and S2.5.c Reading Specialist Assessment Summary].  As experienced and working teachers, most candidates in these programs of study are focused on task completion as specified in the courses of these programs of study.  The most valuable use of the process is to verify the course outcomes and expectations remain focused on standards and student needs.  Decision Point 4 combines completion of the practicum experience and program completion for all programs except for the M.S. Ed. degree in C & I with ESOL Emphasis and the M.S. Ed. degree in C & I with Accountability Emphasis.  For many who are completing the “ESOL Emphasis” degree, the ESOL practicum was completed as part of the non-degree ESOL endorsement program while students in the “Accountability Emphasis” degree program have the option of a research study or a practicum experience.  Feedback from these practicum experiences combines verification of candidate performance with information for program change and growth; this information can be viewed in the assessment summaries for these advanced programs [Exhibits S2.5.a Building Leadership Assessment Summary, S2.5.b ESOL Assessment Summary and S2.5.c Reading Specialist Assessment Summary].  Decision Point 5—licensure—is separate from program completion as successful performance on licensure assessments must be combined with program completion.  Licensure success rates are included as part of the assessment summaries for these three programs [Exhibits S2.5.a Building Leadership Assessment Summary, S2.5.b ESOL Assessment Summary and S2.5.c Reading Specialist Assessment Summary] and clearly show a high level of success for the two programs with a history of licensure assessment scores (Building Leadership and ESOL).  Thus, the decision making process is supported by a set of tools that focus on the measurement of course outcomes, provided both as a form of verification of outcomes for individual candidates and feedback in the form of group data to determine the consistency of student outcomes in course experiences. 

Explanation of Data Collection and Analysis

The role of the unit assessment program is to provide for verification of all types of data with attention to appropriate definitions of “success” for different tasks.  The unit assessment program is one of multiple measures in which:

  • Some data is based on task completion in which “success” is measured by meeting performance standards of which test scores and GPA requirements are examples.
  • Some data is based on the completing of tasks in which “success” is simply measured by completion in a timely manner
  • Some data is based on task completion in which “success” is attained by a combination of the task with a degree of minimum expectations such as successful completion of an interview or completion required prerequisite courses.

Maintenance of student data is a shared process that involves the School of Education office on the main campus and the NU Outreach Office in western Kansas.  The system combines paper and electronic documents so that paper documents are housed in an official folder for each candidate and electronic documents are housed in the secure Newman University electronic support system. Data collection is a shared process for faculty members.  Direction for the development of the assessment plan is provided by team leaders for different areas of emphasis and matrix that shows the relationships of standards, assessments and courses for embedded assessments for each program of study is evidence of this work.  [Exhibits S2.2.a ECU Matrix of Assessments and Standards, S2.2.b Elementary Matrix of Assessments and Standards, S2.2.c Secondary English Matrix of Assessments and Standards, S2.2.d Secondary Government History Matrix of Assessments and Standards, S2.2.e Secondary Mathematics Matrix of Assessments and Standards, S2.2.i ESOL Matrix of Assessments and Standards, S2.2.h Building Leadership Matrix of Assessments and Standards and S2.2.j Reading Specialist Matrix of Assessments and Standards] In order to facilitate data collection, an EXCEL based data collection form has been developed and is provided for each course section for the purpose of data collection.  Data for all assessments, both initial advanced programs, has been organized into assessment reports to support the use of data. 

How is the data used?

The use of data from a combination of sources is used in a continuous process for quality improvement.  The ultimate “litmus test” for program changes must be provided in the context of the Newman University School of Education Mission to prepare educators who can make a difference as knowledgeable, caring, reflective, visionary, collaborative, and ethical individuals.  Although a more complete discussion of these changes is provided for examination, this summary will provide a view of data and feedback based changes that have been made.

Although the ECU (Early Childhood Unified) program is new, feedback of three types has been used to support change in this program [Exhibit S2.7.a Examples of ECU changes].  A review of test scores (Praxis II sub-scores) was used as a basis for adjusting the content of courses in the program.  In addition, student feedback about the portfolio process was used to streamline course syllabi and eliminate redundant/unnecessary assignments.  Finally the review conducted in the preparation of a KSDE Program review resulted in adjustments in the field placement process. 

The Elementary education program used data and data collection as the basis for several notable adjustments [Exhibit S2.7.b Examples of Elementary changes].  Based on the review of Praxis II sub scores, a course alignment adjustment was provided to address a concern about Social Studies sub-score data while concerns about Science sub score data was the basis for a staffing change.  The review of Literacy Portfolio data was used as the basis for an adjustment to the relationship of instruction and assessment as a basis for increasing program consistency.  Finally, the lack of data and concerns about collecting data was the basis for the development and use of an additional assessment tool (NUTPP or Newman University Teacher Performance Portfolio) and the development of a new content course (Methods of Teaching Content)

The review of data from Secondary Teaching was used to support two improvements for these programs of study [Exhibit S2.7.c Examples of Secondary changes].  Student feedback resulted in the strengthening of the Principles of Secondary Teaching Unit Plan.  An examination of data about the timing of taking licensure assessment resulted in a case study review and the development of additional informational support when candidates are taking licensure assessments. 

Feedback of different types was used to reinforce and organize additional review of advanced programs [Exhibit S2.7.d Examples of Advanced Program changes].  Although the Reading Specialist program does not have a developed source of systematic data, informal feedback from students while in the study program reinforces the focus on the development of instructional strategies to meet the needs of limited English proficiency students in the program.  The use of data from the ESOL program has been used to address two issues.  First, data on licensure assessment passing rates clearly shows a high level of consistent success—throughout the program—according to this measure and provides a confidence measure to support current program emphasis.  Additional use of this data shows that many who take at least one NU ESOL course do not complete the entire program of study and provides support for continuing work with school districts and candidates to examine program value.  Finally, feedback for ESOL Adjuncts has been used to develop and renew an electronic support network for Adjunct Instructors in the program.  Finally, Building Leadership program data from recent years would seem to indicate that a number of program completers have delayed licensure by not taking the SLLA licensure assessment and may not be actively seeking employment as school administrators.  Should this be a measure of progress success and should it be used as part of the continuing renewal process for the program?