2b.1. What are the processes and timelines used by the unit to collect, compile, aggregate, summarize, and analyze data on candidate performance, unit operations, and program quality?

The Unit Assessment System reflects the unit’s commitment to implement a system that can ensure optimal performance of faculty, candidates, and the students whose lives they impact, while allowing for continual improvement in response to data-supported needs. Data are collected concerning applicant qualifications, candidate proficiencies, student success, post-graduation performance levels, effective unit operations, and program quality. Using multiple assessments from internal and external sources (Exhibit 2b1.1 Unit Assessment Workflow Matrix), the unit systematically collects data from applicants, candidates, K-12 students, recent graduates, faculty, administrators, and other members of the professional community which in turn delineates the timing of data collection and analysis as it occurs throughout the candidates’ program and throughout the calendar year.

Historically, assessment data have been collected in the unit, but the unit assessment plan implemented in 2002 increased structure and greatly improved the assessment process. However, even with the new plan, it was difficult to fully integrate a total unit assessment system because data were originally housed in multiple formats and in multiple places. New procedures were introduced in 2006 that allowed for more precise examination and cross-referencing of key assessments. Many of the earlier issues were resolved with the integration of the portfolio system, TaskStream.

In the Fall of 2006, TaskStream, a commercial system, was adopted for the College of Education and Human Development after several months of research into electronic assessment systems. TaskStream allows for the collection and dissemination of candidate data in an efficient and user friendly way. Candidates must pay a fee to Taskstream to register for up to three years at a time. Once students have created an account on Taskstream, they self enroll into a customized degree program along with a field experience program. Candidates add material and maintain their account and therefore have access to every Signature Assessment and other requirements due from start to completion of their program. At the end of each semester, candidates must submit their signature assessments as well as any other requirements due for the semester and assessment purposes (e.g., dispositions or application to teacher education or student teaching). Faculty must evaluate candidate signature assessments and dispositions before submission of the candidate’s final grade. Routine checks to make sure candidates and faculty are submitting and evaluating work respectively are done every semester. Candidates’ other requirements, such as application to teacher education and student teaching along with exit surveys, are submitted to the Director of Field and Clinical Experience. Candidates are able to see their scores and any comments made by faculty or supervisors as soon as the evaluation is released after grading. Candidates are made aware at each decision point whether they will be allowed to continue in the program. Faculty members also have the ability to aggregate course or program data for programs they teach.

Collect: Candidate data is collected at the end of each semester using TaskStream. This data includes signature assessments, field experience, exit surveys, etc. Collecting graduate follow up data, principal or school administrator data on graduates of the teacher education program, as well as faculty and candidate perception of unit operation is done through Survey Monkey. Survey Monkey is an online surveying service. Program and unit data are collected annually in June in collaboration with the office of University Planning and Analysis. This data includes enrollment, student hours, demographic information, etc. Additionally the Coordinator of Grants and Development along with the Business Facilities Manager provide data on budget and building operations. Course, program and unit data are collected in the same manner for both face-to-face and online programs.

Compile: Director of Research and Data, along with the Assistant Dean of Assessment and Accreditation, compile data annually in June. This includes candidate, program, and unit data for both face-to-face program and online programs. Microsoft Excel and TaskStream are used to compile data for all programs.

Aggregate & Disaggregate: Director of Research and Data aggregates the data for the unit, using Microsoft Excel, and disaggregates data by programs during the month of July.

Summarize: Director of Research and Data summarizes the unit and program data during the month of July. Data reports are posted on TaskStream program websites. This provides accessibility to the data for faculty anytime as long as they have Internet access.

Analyze: Director of Research and Data, along with the Assistant Dean of Assessment and Accreditation present the summary reports to the faculty members in each program in August for their analysis. Unit reports are given to various administrators and stakeholders. Appropriate faculty members present reports to Teacher Education Council and PK+-16 Advisory Council. Faculty can request data reports at any point during the year or can run the reports themselves. (Exhibit 2b1.1 Unit Assessment Workflow Matrix)

2b.2 How does the unit disaggregate candidate assessment data for candidates on the main campus, at off-campus sites, in distance learning programs, and in alternate route programs?

Since the inception of TaskStream in the fall of 2006, the unit has made a concentrated effort to centralize all data collection into one centralized system. TaskStream allows for this process to occur smoothly and efficiently. By having all possible data feed into TaskStream, it allows for both main campus faculty, distance learning faculty, and administration to access data anytime.

Every initial or advanced teacher education candidate is required to have a TaskStream account. Their program of study and field experience in TaskStream is aligned with their degree sheets and unit’s portal requirements.

Candidates submit signature assessments, field experience (observations and weekly logs), disposition assessments, and applications to teacher education and student teaching on TaskStream. Faculty provide evaluation and scores through TaskStream. Candidates have access to all evaluations and comments made by faculty members. TaskStream also keeps track of communication exchanges between candidates and faculty. Changes to signature assessments and programs are communicated to candidates via faculty members in their courses. Also advisors inform the candidates of program or unit changes.

Disaggregation of data for the main campus is done via multiple processes. Faculty at the start of every Fall semester get program and courses data distributed to them by the Director of Research and Assistant Dean of Assessment and Accreditation. Faculty can also request access to data or reports at any time during the year from the Director of Research and Data by simply filing out a request for data form. Faculty can also aggregate their own course or program data via TaskStream at anytime. TaskStream contains data from signature assessment scores, observations scores, final assessments, field experience, exit survey, etc. (Exhibit 2b2.1 Data Request Form)

Disaggregation of data for the distance learning program is conducted in the same manner as for the main campus programs. Any faculty member teaching in our distance learning programs (online M.Ed in Curriculum and Instruction, Educational Leadership and Master of Art in Teaching) have access to candidate data in terms of signature assessments scores, final assessments, candidate dispositions, field experience, graduate exit surveys, etc. via TaskStream. Similar to faculty teaching on the main campus, faculty can request data or reports from the Director of Research and Data at any time by simply filling out the data request form. (Exhibit 2b2.1 Data Request Form)

2b.3. How does the unit maintain records of formal candidate complaints and their resolutions?

The unit maintains a record of formal candidate complaints and documentation of their resolution in the Dean’s office (Exhibit 2b3.1 Academic Appeal Procedure). The Undergraduate and Initial Review Committee is responsible for reviewing appeals of undergraduate and MAT candidates while the Graduate Review Committees address appeals of graduate candidates (Exhibit 2a4.4 Undergraduate and Graduate Committee).

Candidate complaints are addressed in the unit in the following sequence: the Instructor, the Department Head within which the course/instructor resides, followed by the Dean. The Dean may request that the Associate Dean and/or the Assistant Dean hear a student complaint before being addressed by the Dean. Instructors and department heads keep records in their respective offices to document the complaint process. If the complaint is not resolved within the unit to the candidate's satisfaction, policies and procedures in the ULM Student Policy Manual (page 45) are followed. (Exhibit 2b3.2 Student Policy Manual)

2b.4. (Optional Upload for Online IR) Tables, figures, and a list of links to key exhibits related to the unit's data collection, analysis, and evaluation may be attached here. [Because BOE members should be able to access many exhibits electronically, a limited number of attachments (0-5) should be uploaded.]