2c.1. In what ways does the unit regularly and systematically use data to evaluate the efficacy of and initiate changes to its courses, programs, and clinical experiences?

The Director of Research and Data along with the Assistant Dean of Assessment and Accreditation aggregate data from both initial and advanced programs during the first week of August. The data are aggregated from three main sources: 1) TaskStream (unit electronic assessment system), 2) campus computing information system, and 3) ETS. Each system provides a unique set of data. Signature Assessment, candidate disposition, exit survey, and field experience data are collected using TaskStream. Grade point average, certain student demographic data, and enrollment data are collected using the campus computing information system. Finally, PRAXIS scores are collected from data provided by ETS. Even though the data are aggregated once a year, faculty members have the capability to aggregate data (course or program) in the programs in which they teach at any time. The Assistant Dean of Assessment and Accreditation and the Director of Research and Data run routine checks on TaskStream to make sure candidates are submitting their signature assessments and other required elements on TaskStream, as well as faculty are evaluating candidates’ work in a timely manner.

Once data are aggregated by the Director of Research, working together with the Assistant Dean, the data are then summarized by individual programs. This data and summarized results (in Excel) are presented to the Assessment Committee during the second or third week in August for analysis. The analyzed report and summarized data are then presented to the program chair and faculty members in each program at the start of Faculty Week (usually the third week in August). Faculty have a week to review the presented data and analysis to then initiate changes based upon the data.

If signature or program changes are deemed necessary by faculty, they complete a change of signature assessment/program change form. A rationale for the change along with quantitative and qualitative data if applicable is attached to the signature assessment/program change form. This package is then presented to all faculty members teaching in the program as well as the program chair. Upon approval from these members, the Assistant Dean of Assessment and Accreditation then takes the package to the Assessment Committee. Upon approval by the Assessment Committee, the requested change is implemented. If changes involved rubrics or forms on TaskStream, the old data are backed up and a new rubric or form is implemented.

The program faculty members use data on a yearly basis to make changes to signature assessments in their courses. These changes occur for a variety of reasons, ranging from changes in state or national program standards, best practices, or identified needs. If faculty perceive current signature assessments do not sufficiently measure program standards covered in a particular course and data prove this, then changes are made not only to the course but also the signature assessment. Data are also collected to analyze unit functions on a yearly basis.

Data such as candidate knowledge, skills, and dispositions, standardized test scores (PRAXIS), and exit surveys are used to implement unit changes. For example, a new candidate disposition instrument is being implemented in the Fall of 2009 based on data collected during the past year. These data include faculty perceptions, analysis of current dispositions, and results from previous years’ disposition scores of candidates.

In the 2008-2009 academic year, unit data was collected via unit operations survey administered to candidates and faculty. The candidate survey addresses: quality of unit operations, professors and supervisors, advisement, learning experience, academic support and operations of academic unit outside of teacher education. The faculty survey addresses: the unit’s ability to address the objectives outlined in the mission statement, citizenship, teaching, research and collaboration. The data from these two surveys are collected in June. Changes are made based upon survey results. Changes that are required are brought before the Assessment Committee. Approved changes are then forwarded to appropriate faculty members to disseminate to candidates, faculty, and the professional community. (Exhibit 2a1.1 Unit Assessment Handbook, Exhibit 2a3.1 Assessment Change Forms and Signatures)

2c.2. What data-driven changes have occurred over the past three years?

Over the past three years, unit consideration of assessment data resulted in important changes, each designed to improve teaching and learning. Data-driven changes occurred in signature assessments and courses, programs, and the unit.

The changes in courses or assessment occurred in the past three years in the following courses:

*CURR 450
*ELED 340
*SPED 577
*EDLE 555/560
*EDFN 524
*READ 518
*CURR 302
*CURR 542

The following program changes have occurred in the past three years:

*Academically Gifted, changes to all key assessments
*Intern MAT rubrics change
*Student Teaching rubrics

The following unit changes have occurred in the past three years:

*Faculty evaluation instrument
*Policy and Procedures for online teaching and learning
*Advising procedures
*Candidate Dispositions

Details, including the nature of and process for accomplishing each cited change, are presented in Exhibit 2a3.1 Assessment Change Forms and Signatures.

2c.3. What access do faculty members have to candidate assessment data and/or data systems?

Faculty have two forms of direct access to candidate assessment data and the data assessment system. First, faculty can aggregate course (signature assessment & candidate dispositions) and program data (portal review & application to teacher education and student teaching) on TaskStream for courses and programs they teach. Faculty members teaching in initial or advanced programs get a free TaskStream account and are enrolled as evaluators in all programs in which they are instructors. Enrollment is done by the office of the Assistant Dean of Assessment and Accreditation.

The second form of access that faculty have to candidate data or the data system is to request data from the Director of Research and Data. Faculty members fill out a request for data form, which they submit to the Director of Research and Data. Requested data are delivered to faculty via email or hard copy, according to faculty preference.

In addition, faculty also access data when they participate in review of the annual data summary and analysis during Faculty Week. At that time, they may initiate data-driven changes by following established unit procedures described in section 2c1 of this report.

2c.4. How are assessment data shared with candidates, faculty, and other stakeholders to help them reflect on and improve their performance and programs?

Assessment data are given to faculty at the start of the Fall semester and given an opportunity to voice opinions on proposed changes and make recommendations. Faculty can view their program reports for the past three years on TaskStream which houses data collected, minutes taken during the analysis of the data, changes recommended, and actions taken.

Instructors, at the start of the course, notify candidates of program changes, such as changes to a signature assessment in a course. Advisors inform candidates of a unit change, such as with advising procedures or new disposition instruments and workshops are held in classes to inform candidates of a new disposition instrument or modifications to the conceptual framework and reasons for change.

Throughout initial programs, candidates meet regularly with supervisors to discuss performance in courses and field experience. After observations are conducted; the candidate and supervisor are able to discuss findings and scores to pinpoint strengths and weaknesses. Candidates are also able to see the scores and rubrics of their assessments through Taskstream. Candidates can see feedback notes written by the professor of the course, which are permanent records of interaction, and use for reflection and improvement. Also, initial candidates complete professional growth plans at various times for use in planning and improvement on classroom skills. Advanced candidates meet with their major professor to complete the required research report and in turn, faculty give the candidate feedback on the report and for reflection.

Summary data of program and unit changes are disseminated and presented by the Assistant Dean to various outside stakeholders through the PK-16 Council, Teacher Education Council, and the Dean’s Advisory Council. Feedback is welcome from stakeholder groups, and changes may be made based upon the feedback, in which other stakeholders are notified.

2c.5. (Optional Upload for Online IR) Tables, figures, and a list of links to key exhibits related to the use of data for program improvement may be attached here. [Because BOE members should be able to access many exhibits electronically, a limited number of attachments (0-5) should be uploaded.]

Optional

1. What does your unit do particularly well related to Standard 2?

• The unit does a thorough job of collecting, compiling, aggregating, summarizing, and analyzing data on candidate performance, unit operations, and program quality each year as demonstrated in the Unit Assessment handbook and SPA reviews.

• All stakeholders (candidates, faculty, administrators and professional community) take an active role in the unit assessment system. Information is consistently vetted through various avenues to the appropriate committees, professional communities, and to candidates, thus keeping everyone in the loop.

• The unit uses the full capabilities of its electronic assessment system (TaskStream) to implement and maintain the unit assessment system. Numerous institutions within the state and nationally have contacted the Assistant Dean’s Office for help in implementing an electronic assessment system.

• TaskStream allows candidates to be informed of all requirements (signature assessments, field experience, candidate dispositions, applications, etc.) for every stage of their program. Having a centralized system allows candidates, faculty, and administrators to work more efficiently in terms of meeting assessment and accreditation requirements.

2. What research related to Standard 2 is being conducted by the unit or its faculty?

• Sivakumaran, T., Heyning, K., Wishart, W., Holland, G., (2009) Teacher Education Candidates and Faculty Member’s Understanding of Standards-Based Assessment. 61st American Association of Colleges for Teacher Education (AACTE), Chicago, IL

• Sivakumaran, T., Heyning, K., Wishart, W., Holland, G., (2009) E-Portfolio Implementation, Maintenance and Support. 61st American Association of Colleges for Teacher Education (AACTE), Chicago, IL

• Mann, R., Cannon, M., Sivakumaran, T., (2009) Using Data from an Electronic Assessment System to Improve Student Success. 2009 Hawaii International Conference on Education

• Sivakumaran, T., Thompson, W., (2008) Outcomes Assessment through the TaskStream Accountability Management System. Southern Association College and Schools Annual Meeting, San Antonio ,TX

• Wishart, W., Sivakumaran, T., Heyning, K., Flowers, B., (2007). Roles of Mentor Teachers in Teacher Education Candidates Assessment. American Association of Colleges for Teacher Education (AACTE), New Orleans, LA.

• Sivakumaran, T., Schween, D., (2008). Beyond the One Person Shop: Sustaining and Extending Unit Database Information Systems. American Association of Colleges for Teacher Education (AACTE), New Orleans, LA.

• Heyning, K., Sivakumaran, T., Wishart, W., Glenda, H., (2008) Dynamic Data Management û Built in Flexibility. American Association of Colleges for Teacher Education (AACTE), New Orleans, LA.

• Heyning, K., Sivakumaran, T., Wishart, W., Glenda, H., (2008). Electronic Portfolios and Hiring Process: Administrators Perception of Teacher Candidates Submission. Hawaii International Conference on Education (HICE), Honolulu, HI.

• Schween, D., Sivakumaran, T., (2007). Digital Dilemma: Roles in Data Collection. American Association of Colleges for Teacher Education (AACTE), New York City, NY.