1	Quantifying the Assessment Enigma: Analysis & Synthesis Skills in General Education College of DuPage March, 2004 Russell J. Watson, Ed.D. Peter T. Klassen, Ph.D.
2	General Education definition * The aims of general education are to enable students to understand and appreciate their culture and environment; to develop a system of personal values based on accepted ethics that lead to civic and social responsibility; and to attain the skills in analysis, communication, quantification, and synthesis necessary for further growth as a lifespan-learner and productive member of society. * College of DuPage Catalog.
3	Brief history – institutional level assessment Standardized assessments provided a rich source of information about Reading, Writing, Math, Science Reasoning, and Critical Thinking skills. Supplemental questions included with these tests provided a source for information about understanding, appreciations and values. The process lacked specific information about Analysis and Synthesis skills
4	Standardized testing Substantial progress was made with the ACT/CAAP assessments. Sampling from course-sections Representative sample controlled with characteristics from student tracking High participation and effort Rolling annual cycles in Fall & Spring Faculty cooperation Results corroborated with other data including focus groups and graduate surveys.
5	Standardized testing or local instrument Advantages of standardized testing Content, skills validated by external experts Benchmarks and comparisons are possible Advantages of local instruments Skills and criteria align with insti-tutional goals Process of designing an assessment promotes institutional development Provides a measure where none is published
6	Application of existing models Synthesis and critical thinking addressing open-ended questions. Essay responses provided for a wide range assessment of skills A range of theoretical and pragmatic assessments have been developed and tested.
7	Components in COD’s response Prompts Response form Rubric – a foundation for evaluation Feedback
8	Prompts Open-ended questions are choices among alternatives, with no right or wrong answer. Prompt needed to present a Pro/Con position on broad topics of general interest, attention, and media coverage. Selected for brevity, clarity, and strength of position statement.
9	Response form Similar in format to CAAP so it could be integrated into testing cycles. Meeting various implementation and quality issues requirements Editing and using the college icon Paper opacity for op-scan technology Keeping track of multiple page responses Encoding and aggregating data while minimizing cross-reader bias
10	Rubric development Reviewing of other rubrics in use Selecting of four dimensions to be evaluated Drafting of dimension criteria and descriptions for application with a diversity of responses Testing of rubric for effective use and sensitivity in discriminating among our students skills Current draft is a tentative working option
11	Rubric – a foundation for evaluation Communication of ideas Information literacy skills Synthesis / Critical thinking skills Ways of Thinking / Knowing Quantitative, Empirical Thought Rationality based on Authority, Reasoning, or Consequences Intuitive / Reflective Thought Multiple, mixed approaches or at low levels no clear way of thinking
12	Feedback Letters written with dynamic inserts to indicate: Thank your for participating… and general explanation of dimensions. Level of performance on each scale measured. An indication of the student’s performance and the percent distributions for each dimension criteria. Report to the faculty and institution
13	Progress to date Student response 141 responses from an opportunity sample of course sections of day-time general transfer students 40 responses evaluated, results reflected in the example letters, remaining in process Evaluators experiences Four dimensions of varied length is difficult, but manageable Dimensions “work” and student responses are arrayed along the criteria Criteria need to be edited for a more consistent harmony
14	Contacts and information Peter Klassen – p.klassen@DocumentingExcellence.com www.documentingexcellence.com/links.htm Russ Watson – watson@cdnet.cod.edu College of DuPage -- 630.942.2800 425 22^nd St., Glen Ellyn, IL 60137