Documenting Excellence
A consulting practice focusing on working with colleges', organizations', and individuals' utilization of quantitative and qualitative assessment tools to analyze and document their quality outcomes through providing staff development, research design and analysis, and psychometric evaluations.

Responses to CAAP Results

Home Up Assessment Links Statistical Tools Contents Glossary Search Feedback

TOC Workbook Workbook Questions General Education Exploring CAAP Objectives Responses to CAAP Results Envisioning the Future Feedback and Assessment

Further Information

Up

Primer of Statistical Tools

FAQ
Tool List (TOC)

Section Three: Exploring the Fall Quarter 1998
CAAP Test Results

The Fall Quarter 1998 round of CAAP tests had a population of 609 students across the six tests. Entering freshmen were 43% of the sample and returning sophomores were 39% of the sample. It is only the freshmen population that can be clearly measured and classified in this initial round. The classification of the remaining students requires aggregation of test results with students’ records.

The Fall Quarter 1998 sample contains students reasonably defined as Freshmen at College of DuPage and students at other points throughout an educational range. Because of this diversity, it is not appropriate to statistically compare the fall sample to any reference norms. However, using general characteristics, the typical student taking the test (based on mode characteristics) is a white (73%), female (55%), age 19-20 (39%), who speaks English as her first language (87%), freshman (43%), enrolled at COD as a freshman (79%), attends classes full-time (72%), reports a GPA between 2.51 - 3.0 (24%), and plans to transfer to another school (52%). It can be reasonably concluded from examining the variation in student characteristics that any one characteristic is less than representative. For the purpose of this initial review, results are being presented in two ways. The following text describes observations and reasonable conclusions based on a statistical summary presented in Table 3.

Selecting from this total sample those students identifying themselves as freshmen, some comparisons to national freshman norms are possible. Test scores from the COD-freshman cohort and two referenced norms are summarized in Table 3. One set of norms are those for freshmen at 2-year public colleges, a category in which College of DuPage belongs. Also provided are norms for freshmen at 4-year public colleges and universities.

Based on these norms one can reasonably conclude that College of DuPage freshmen are statistically similar to 2-year public college freshmen on four of the tests: reading, critical thinking, science reasoning, and writing skills objective test. Our freshmen score significantly higher than freshmen at 2-year public college on two tests: mathematics and essay writing. A similar pattern emerges based on the sub-scores with our freshmen demonstrating no significant difference from 2-year public colleges freshmen in knowledge of mechanical usage, rhetorical skills, and reading in arts, literature, and social studies. The algebra sub-score from mathematics is statistically higher than the comparable norm.

A comparison of our freshmen with 4-year public colleges sets a slightly higher standard. Comparing our freshmen with 4-year public colleges and universities, our students scored significantly lower on all tests and sub-tests except essay writing. In essay writing they demonstrated no statistical difference from the 4-year norms. (Note that the ACT norms for 2-year colleges are lower than the norms for 4-year colleges, just as represented in our sample.)

At this point is it important to note that the COD findings are based on reasonable but small numbers of students. Before accepting these observations additional cases need to be added to these data. The next rounds of testing are scheduled for Spring-1999, Fall-1999, and Spring-2000. By fall of 2000 the patterns observed here will be expanded to include completing-sophomores. Then conclusions may be supported or modified based on larger numbers of students.

For those readers not interested in the supporting numbers may wish to move on to the response to worksheet 3. For those wishing to explore the supporting data in Table 3, the following may provide some explanation. For the COD data as well as each of the referenced norms a mean, standard deviation, and number of subjects are listed in the test and sub-test columns. Also listed is a margin of error. The margin of error may be added to and subtracted from the appropriate mean to determine an interval within which the mean of each cohort, may fall. If the interval calculated for our freshmen does not overlap with an interval of a referenced norm cohort then the two cohorts show a statistically significant difference. Now, for those who want the statistical details, the margin of error was calculated using the established procedure for estimating a .05 probability of Type I error, utilizing the formula 1.96 * standard error of the mean.

Table 3: College of DuPage -- Fall Quarter 1998 CAAP Results

COD- 1998 Fall

Institutional Summary

Writing Skills

Usage / Mechanical

Rhetorical Skills

Essay Writing

Math

Algebra

Reading

Arts / Literature

Social Sciences

Critical Thinking

Science Reasoning

Mean

61.3

15.9

15.6

3.3

57.1

14.6

59.6

14.5

15.2

59.5

58.4

S.D.

4.8

2.2

2.5

0.6

3.8

2.0

5.7

3

2.7

5.7

4.2

N

113

113

113

109

99

99

100

100

100

100

96

Margin of error

0.89

0.41

0.46

0.11

0.75

0.39

1.12

0.59

0.53

1.12

0.84

 

COD - Entering Freshman

Writing Skills

Usage / Mechanical

Rhetorical Skills

Essay Writing

Math

Algebra

Reading

Arts / Literature

Social Sciences

Critical Thinking

Science Reasoning

Mean

60.2

15.3

15.2

3.2*

57.4*

14.7

59.9

14.9

14.9

59.8

57.9

S.D.

4.4

2.2

2.2

0.7

3.8

2.0

5.8

3.0

2.9

5.6

4.2

N

48

48

48

42

45

45

39

39

39

48

46

Margin of error

1.24

0.62

0.62

0.21

1.11

0.58

1.82

0.94

0.91

1.58

1.21

 

ACT - 1997 2-Year Public Freshman

Writing Skills

Usage / Mechanical

Rhetorical Skills

Essay Writing

Math

Algebra

Reading

Arts / Literature

Social Sciences

Critical Thinking

Science Reasoning

Mean

61.3

15.8

15.6

2.9

55.8

13.8

59.0

14.4

14.6

58.8

57.6

S.D.

4.9

2.4

2.6

0.6

3.6

1.9

5.1

2.6

2.5

5.1

3.9

N

4085

4085

4085

1123

3655

3655

2018

2018

2018

6833

1825

Margin of error

0.15

0.07

0.08

0.04

0.12

0.06

0.22

0.11

0.11

0.12

0.18

 

ACT - 1997 4-Year Public Freshman

Writing Skills

Usage / Mechanical

Rhetorical Skills

Essay Writing

Math

Algebra

Reading

Arts / Literature

Social Sciences

Critical Thinking

Science Reasoning

Mean

65

17.5

17.5

3.1

58.9

15.4

63.1

16.2

16.5

63.3

60.7

S.D.

4.7

2.3

2.5

0.5

3.8

1.8

5.1

2.5

2.6

4.8

4.3

N

3177

3177

3177

699

2915

2915

2763

2763

2763

2777

2656

Margin of error

0.16

0.08

0.09

0.04

0.14

0.07

0.19

0.09

0.10

0.18

0.16

 
 

*   Statistically significantly higher than 2-year Public College Freshmen

  Statistically significantly lower than 4-year Public Freshman

Worksheet 3: Response to the Test Results

1.     What is your response to the general and specific characteristics of College of DuPage students’ general education competencies?

Our students are normal for community college freshman. Their higher scores in Essay writing and math reflect the high-schools they come from. I want to know what we add to their development.
I was pleased that our students are statistically similar to 2-year public college freshmen and four tests and significantly higher on math and essay writing.
It does not surprise me that our C.O.D. students score lower than the incoming freshmen at 4 year state schools.
I am upset that there are different norming standards for 4 & 2 yr. Public colleges.
Pleased that our students performed above other 2 year public colleges on math & essay writing. But would have
liked to have seen stronger performance on the other subtests.
Interesting results--I look forward to future study.
I can see where we need improvement as well as what we are doing well & need to continue to do.
For a community college students do not need any specific average to get in. For my language classes they all start with 101. Most students have the fundamental skills, but there is a continuous erosion. New students have less
That our COD students are "holding their own" when compared with other 2 yr. Public college freshmen.
Generally moderate competencies
The test scores seem to be higher than what I see students able to do in my classroom.
About what I expected.
It is not surprising that Cod's entering freshmen are lower than 4-yr. Freshmen. How do we respond to this?
Depressing but not surprising.
I guess I'm not surprised.
No response
Appropriate for community college.
I am not surprised by the results of the survey. Many of our students are not ready to compete with 4 year univ.
The first thought (questions) I had was how early in the Fall, 1998, were they tested - and whether or not we could assume that the 4-year freshmen were from a sample taken at a comparable time in Fall, 1998.
No surprises.
Mostly as expected. Positive surprise that difference to 4-year colleges is not bigger.
Is this more a measure of their high school preparation than COD experiences?
Surprised that above mean in math especially since many place in developmental math sequence.
I think the results show that our students may have to work harder at their transfer institution than native students to be successful.
I am not surprised to see our scores are lower than other institutions, although I was surprised by how much lower in some instances.
I am heartened by the fact that Cod freshman students scored slightly higher than general freshman students entering 2 yr. Institutions. I wish COD freshman were on a par with those entering 4 yr. Institutions.
Does not surprise me. Supports some of the experiences I've had the past 2 quarters.
I Think they seem better than what I expected.
Just like any place else, some are more competent than others.

2.     What specific information, if any, surprised or disturbed you, and why?

Surprise: I expected that we would be higher in more of the tests. Closer to the scores for 4-year freshmen. Of course, this is the disappointment expressed with many of our criticisms of student performance.

I was surprised that our students scored lower on all tests and subtests except essay writing than freshmen from 4-year public colleges and universities.

It does not surprise me that our C.O.D. students score lower than the incoming freshmen at 4 year state schools.

That our students score well/or at least better on the essay writing & mathematics tests.

Strength of performance on essay writing surprised me. Not consistent with most of my students' writing samples.

I do not see the skill level in writing that I feel is appropriate for a college level student. We may be in a favorable institution by comparison, however basic writing seems to be absent.

Curiously, students scored "lower on all sub-tests except essay writing." This is encouraging news! (from an Eng. 101,102,103 standpoint)

Writing & math scares, went against my assumption - esp. writing scares.

There is considerable discrepancies among students background and ability (I am referring to freshmen). Students are more heterogeneous than homogeneous.

That essay writing was higher than 2 yr. public college freshmen, but writing skills were lower than 4 year public

Lack of writing skills

None

None - though I thought we might have done worse in essay writing than 4 year colleges - glad we don't.

Zero margin of error for science reasoning. Never have seen that with such a large sample size.  [Editor's note, the s.e. = 0.84, thanks for noticing.]

None

None

The Essay Writing score seems unusually high, given what I see in my courses. Further analysis--even verification--is needed here.

Essay writing results - I would have guessed, based on my experience, the results would have been lower.

none

Higher goals for written competency and improved math skills. Actually, although 4 year transfer programs in architecture require advanced mathematics, the freshman students need help in basic math, algebra & geometry.

No real surprises. Nothing disturbing.

We were not surprised, we have observed the results as the students complete their assignments.

Nothing

None

I was not surprised.

Students at COD are having difficulty with a number of basic skills including the use of formal, standard English, literacy, reasoning/critical thinking which significantly reinforces the need for the developmental classes. I think there needs to be more communication among faculty and how they can help students learn these skills without "dummying (?) down" their curriculum including a discussion of the support offered by faculty teaching developmental classes.

Perhaps other disciplines also need tutoring centers like the math assistance center and the writing center. This increases the need to individual attention to our students.

Writing skills, usage, and rhetorical skills could be better at Cod - though the margin of error could explain the slightly lower achievement in these areas.

As a new faculty I don't have must experience at the community college level so I'm just accepting what comes right

Nothing surprised me. I am disturbed by the fact that there were so many adolescent students at Cod in Fall '98.

Some were probably tested. I'm sure the scores would be higher without them

3.     What change(s) would you consider an appropriate goal(s) to work towards?

Changes: I think we should include reading, writing, and quantitative literacy across the curriculum.

We could strive to do better when compared with 4-year public colleges and universities

As the college has already done with the reading courses, I believe mandatory placement into remedial writing courses (if appropriate) should be done!

We should eliminate or combine some of our math remedial courses to get students through the sequence faster.

Continued/enhanced emphasis on these skills across the curriculum.

Writing

There's always room for improvement--I'd like to see a concentrated effort among family to include these elements: reading, critical thinking, and writing skills as a useful part of all courses. In other words, reading strategies are not only taught in Eng. 081, 082, and 083 courses.

Address critical thinking across the curriculum.

The problems start in high school - they pass everybody even those who cannot read what's on their diploma.

Should a community college really have remedial non-credit courses?

Continue working on reading and critical thinking skills in all areas.

We need to bring up the general skills level of our students. Each student should be required to take a study skills class. Our students (about 90% of mine) do not know how to study effectively.

No inspiration here. There is of course always room for improvements.

Perhaps educating students to the need for their redmediation in areas where competencies are lower. COD offers the remediation courses, but many students do not persevere through these.

Honestly? I am interviewing elsewhere. It is getting worse. Increase the numbers of trained faculty in remedial skills areas - basic English and math. 

Continue to improve math and writing skills.

No response

none

It would be great to try to track a cohort group through the end of the sophomore year to compare with 4-year sophs - but I know from the past & the nature of the community college students how difficult this would be.

More reading, writing. More essay tests.

So far we should do nothing - we have to work for the second wave to see if our students learn S.U.

Writing skills

Improve critical thinking skills & science reasoning.

Get the N higher. You cannot statistically compare a sample with 39 elements to one of over 2,000 (or 48 to 6800.)

1) More communication among faculty to discuss these issues. 2) An institute day devoted to discussion of our student body and how all can best meet their needs. 3) Continued development of support systems to help students & faculty in meeting the demands of general education competencies.

Make instructors aware of techniques that can be used to hone skills that are not part of their specific discipline: for example, help science teachers find methods to sharpen rhetorical skills amount their students. Help social science teachers find ways to sharpen science reasoning skills among their students.

Improving writing & research skills. Critical thinking & reasoning skills.

Getting our scores as high as 4-yr. College students.

4.     What questions would you propose for future inquiry concerning students’ general education competencies?

What value is added by our educational efforts?

How can we reinforce these skills across the curriculum?

None come to mind.

Why our math pre-tests don't seem to match the national norms. I'm assuming these were post-tests after student had some C.O.D. math experience.would like to see comparison to private schools.

How can we track students for a longitudinal study?

How do we test oral communication or "performance" of "arts"?

Make sure all students who graduate from H.S. are really proficient in English, math science and social studies. With good basis, college classes are somewhat easy and the classes can be more homogeneous with good standards.

?

Should we be teaching college transfer courses or simply focusing on remedial skills?

No response

We need to know more about their preparation before attending Cod (H.S. experiences, job-related learning, etc.)

none

This is a valuable study - if only to confirm that we are just like everyone else. Can we compare to other schools with open enrollment only.

None

How do our students with a GPA of 3.0 (or some accepted level) succeed at their transfer school?

1) What specific deficiencies do students have in literacy, reasoning/critical thinking and the use of formal, standard English: 2) How will the skill level of COD students impact faculty's ability to help students fulfill the general education competencies? 3) What support will the college provide for faculty to help students fulfill the general education competencies. 4) What support will the college provide students so they can fulfill the general education

Compare general COD outcome to general 2 yr. College outcome (not just freshmen). Compare GPA's of Cod students that transfer to a given 4 yr. School to GPA's earned by transferees from other 2 yr. Colleges. Who earns the higher GPA after transfer?

What can be done to make students more observant? What can be done so students will learn about their own

A Summary Report on Responses to
Considering Course and Faculty Interaction with General Education:
An Asynchronous Workbook

Summary Abstract
Table of Contents
Section 1
Section 2
Section 3
Section 4
Section 5
Appendix

Copyright © 1999 College of DuPage
Student Outcome Assessment Committee
425 22nd St., Glen Ellyn, IL 60137
Disclaimer

 
Send mail to Peter T. Klassen with questions or comments about this web site.
Copyright © 2007 Peter T. Klassen
Last updated: 24 Nov 2007