Appendix A
Assessment Summary of the General Education Initiative
2000-2003
Acknowledging the need for formative and summative assessment throughout the three-year pilot program, Undergraduate Studies officials have created a conceptual model and assessment plan for General Education Initiatives. The Conceptual Model of UD General Education Assessment http://www.udel.edu/ugs/gened/images/GenEdFlowchart.doc illustrates the relationship between the ten general education goals and the academic programs created that accomplish these goals. Our assessment plan, further detailed to include other facets in the future, http://www.udel.edu/ugs/gened/Assessment_Efforts.doc includes both direct and indirect measures of student outcomes.
The information below summarizes what we have learned through the pilot period assessment. A full list of assessment reports is located on the Gen Ed website. http://www.udel.edu/ugs/pdfs/reports.html.
The three-year pilot period has enabled us to gain some
insights, but since assessment is seen as an ongoing cycle, several components identified
as important that should be included in recommendations for next year and
beyond.
Between
2000 and 2003, the impact of General Education was assessed by asking a variety
of questions, mostly through survey instruments.
LIFE
Pathways
Capstone and other Discovery-Based Learning
LIFE
Since the
inception of the LIFE program in the Fall 2000, 655
first year students have participated in the Learning: Integrated Freshman
Experience Program (LIFE). The program has grown each year. In Fall 2000 there were 123 students in 11 clusters; in Fall
2001 there were 178 students in 17 clusters; and in Fall 2002 there were 207
students in 22 clusters. LIFE also included Spring
2001 with 10 clusters and 49 students; Spring 2002 with 7 clusters and 49
students; a Winter 2002 with 1 cluster and 11 students; Winter 2003 with 1
cluster and 17 students; and Spring 2003 with 3 clusters and 21 students. There
are 36 clusters planned for Fall 2003 and 2 clusters
for Winter 2004. Table 1 below shows demographic data for all three
cohorts combined.
Table 1. Demographic and Academic Characteristics of 2000, 2001, and 2002 LIFE and Non-LIFE Students |
||
|
LIFE Students (N = 526) |
Non-LIFE Students* ( N = 8143) |
Female |
329 (63%) |
4881 (60%) |
Male |
197 (37%) |
3262 (40%) |
Black or African American |
46 (8%) |
409 (5%) |
Other Ethnic Minority |
51 (11%) |
643 (8%) |
Caucasian |
429 (81%) |
7091 (87%) |
Mean SAT Total |
1147 |
1135 |
Mean PGI |
2.79 |
2.76 |
* excludes Honors students
Near the end of each fall term, LIFE students were asked to complete a 70-item questionnaire to evaluate their satisfaction and perceived gains from the program. The alpha reliability coefficient for the total set of 73 items was .98, indicating that 98% of the variants in the scores was due to true, accurate measurement.
Across all three years, student responses have been overwhelmingly positive. For example, when asked if the program enhanced their academic and social abilities, LIFE students said yes. In fall 2002, 85% of those responding to the survey said they enhanced their ability to collaborate with others, 75% said the program helped them make connections between their courses and the world, and 71% said they took initiative to get answers to questions.
As a program that seeks to provide academic and psychosocial transitions into the college environment, students perceived it to be a success. The majority of students said that the LIFE program helped them acquire skills that will prepare them for future courses, clarified the major they had or hoped to have, and helped them feel like they belonged in classes as well as the UD community. The vast majority of LIFE students were also very strong in their support of their living-learning community in the residence halls. They reported that living in close proximity to other students in their cluster helped them form study groups, feel connected to peers, and feel like they belong at UD.
Although students have reported their experiences in the LIFE program to be positive, their responses highlight some areas that could be improved. The majority of LIFE respondents did not indicate that the program had enhanced their abilities in writing or using the computer, and only about half said that the UNIV101 seminar helped them learn how to approach and solve problems. Perhaps students believe their skills are not in need of improvement, or perhaps the program did not meet these needs.
For additional information, please see:
LIFE Student Evaluation Summary – Fall 2002
http://www.udel.edu/ugs/pdfs/Fall_2002_LIFE_Evaluation.doc
LIFE Student Evaluation Summary – Fall 2001
http://www.udel.edu/ugs/pdfs/Sum_Report_Fall 2001_LIFE.doc
LIFE Evaluation Survey Item Comparison-Fall 2000 and 2001
http://www.udel.edu/ugs/pdfs/UGS_comparison_report_f00-f01.doc
LIFE Needs Assessment Survey Summary-Fall 2002
LIFE Needs Assessment Survey
Summary-Fall 2001
http://www.udel.edu/ugs/pdfs/UGS_Fall_2001_Needs_Assessment_Survey.doc
LIFE student representative comments selected from a full set of comments
to try to represent a diversity of views.
One unexpected outcome of the LIFE program was the interest by a subset of students to continue their participation with LIFE in the spring. In the spring 2002, 49 students continued and in spring 2003, 21 continued their LIFE activities. Program coordinators thought it would be helpful to examine why some students chose to continue in the spring while others did not, and so a short web-based survey was sent in spring 2003 to all Fall 2001 and Fall 2002 LIFE students.
From the 79 respondents, we learned that the main reasons why students wanted to continue for another semester was due to the fun and friendly atmosphere as well as the opportunity to work further on their final group project. Some of these students who remained in LIFE in the spring believed that they continued to build strong friendships and the additional semester gave them the opportunity to learn more about UD and the campus community. Of those respondents who chose to not continue in the spring, some believed they had not received high value in the fall, and some others said they could not fit the UNIV102 or extra courses into their spring schedule. In addition, some others said that they believed they had adequately developed friendships and gained understanding of the campus in the fall, and thus did not see the need for their involvement beyond one semester.
For additional information, please see:
LIFE Web Survey to Fall 2001 and 2002 LIFE Students
http://www.udel.edu/ugs/pdfs/Survey_Fall 2001-02_LIFE_Students.doc
Program coordinators also thought it would be valuable to examine the comparison between what students perceived as their academic needs prior to matriculation and those skills they believed they received during their first semester. To that end, a comparison was made between responses to the UD Student Needs Assessment Survey and responses to the end of semester LIFE Evaluation. In most cases, the questions were not worded exactly the same, thus the comparisons made are done with great caution and some subjective decisions about the similarity between questions. These comparisons, however, can provide some insight into students’ perceived gains during their first fall semester and thus an indicator of the value of the LIFE program.
For both the Fall 2001 and Fall 2002 cohorts, students’ reported gain in academic and psychosocial skills exceeded their reported need. For example, the F02 mean score for perceived need to ‘budget time more efficiently was 2.32 (1=no need; 4=high need), while the mean score for skills enhanced during the fall semester to ‘budget my time more efficiently’ was 2.81. The higher mean score indicates that, as a group, respondents received slightly more enhancement in time management skills than what they thought they needed prior to beginning the fall semester. A series of comparison charts with greater detail are shown in both reports at the Gen Ed evaluation report website.
For additional information, please see:
LIFE Comparison of Perceived Needs with Skills Enhanced-Fall 2002
http://www.udel.edu/ugs/pdfs/UGS_F02_LIFE_Needs_Comparison.doc
LIFE Comparison of Perceived Needs with Skills Enhanced-Fall 2001
http://www.udel.edu/ugs/pdfs/UGS_F01_Needs&LIFE_comparison.doc
Based on responses from LIFE Faculty Contact survey completed in December 2002 as well as several breakfast and luncheon meetings over the past two years, faculty members who become involved with the LIFE program do so because they are rewarded by their interactions with students. The greatest challenge for their involvement is time. When asked to describe how the LIFE program affected students, many faculty perceived students to be gaining important interdisciplinary skills through hands-on assignments, demonstrating leadership skills, and having the opportunity to build trust and teamwork among peers.
LIFE faculty contacts believed strongly that peer mentors were an important asset, but some believed that a closer match between the peer mentor’s discipline and the cluster were important. When asked what resources would make LIFE smoother, three faculty respondents said that no additional resources were needed, but several others commented on the need for additional funds that could be used for field trips and guest speakers.
For additional information, please see:
LIFE Faculty Contact Survey Summary-January 2003
http://www.udel.edu/ugs/pdfs/LIFE_faculty_comments_03.doc
Faculty Contacts representative
comments selected from a full set of comments to try to represent a diversity
of views.
Based on responses from the Fall 2001 and Fall 2002 Peer Mentor Surveys, overall, peer mentors reported high satisfaction with their participation in and perceived value of LIFE to undergraduate students. Based on the Fall 2001 and Fall 2002 Peer Mentor Surveys, the overwhelming majority of respondents said they felt prepared and received adequate resources to teach the UNIV101 course. Peer mentors believed that freshmen gained important knowledge about the campus and an opportunity to develop peer friendships. In addition, peer mentors reported that they too gained important skills in becoming an effective leader and communicator.
Although their overall comments were more positive than negative, the two areas that appear to merit attention are the peer mentors’ relationship with the cluster faculty contact and residence hall assistant (RA). Some peer mentors recommended that faculty mentors be more actively involved in the cluster, complementing the role of the peer mentor in the UNIV101 course. The majority of peer mentor respondents also commented on their interactions (or lack thereof) with residence hall assistants. Respondents to both surveys commented on the sometimes duplication of tasks between the PM and RA or simply confusion about the role of PM versus RA. Based on these comments, discussions of the role of the peer mentor and the resident assistant may improve the quality of the LIFE program for students and peer mentors.
For additional information, please see:
LIFE Peer
http://www.udel.edu/ugs/pdfs/Fall_2002_Peer_Mentor_Table.doc
LIFE Peer
http://www.udel.edu/ugs/pdfs/Fall 2001_Peer_Mentor_Survey.doc
Peer
Students who choose to participate in the LIFE program are not significantly different from other non-LIFE, non-Honors freshmen. In the fall 2000 and fall 2001 groups, GPAs did not differ significantly, although LIFE students earned a slightly higher GPA. End of first and second term GPAs for the cohort, however were significantly higher for those students who participated in LIFE. This increase may indicate that as the LIFE program has become more solidified, students are netting measurable gains in academic success that may be due in part to the LIFE program.
|
Fall
2000 cohort |
|
Fall
2001 cohort |
|
Fall
2002 cohort |
|
|
LIFE
(n=119) |
Non-LIFE
(n=2583) |
LIFE
(n=200) |
Non-LIFE
(n=2712) |
LIFE
(n=207) |
Non-LIFE
(n=2848) |
Mean
SAT |
1151 |
1126* |
1142 |
1134 |
1149 |
1143 |
PGI |
2.82 |
2.74* |
2.76 |
2.74 |
2.79 |
2.80 |
Fall
00 gpa (eot) |
2.87 |
2.76 |
|
|
|
|
Spring
2001gpa (eot) |
2.91 |
2.73* |
|
|
|
|
Fall
2001 gpa (eot) |
|
|
2.78 |
2.73 |
|
|
Spring
2002 gpa (eot) |
2.91 |
2.86 |
2.79 |
2.76 |
|
|
gpa (eot) |
|
|
|
|
2.92 |
2.78* |
Spring
2003 gpa (eot,cum) |
2.97 |
2.91 |
2.89 |
2.88 |
2.93 |
2.80* |
*
Significant difference between LIFE and non-LIFE students. Analyses above omit
students with gpa=0.0.
For
additional information, please see:
Demographic and Academic Characteristics of Fall 2000, 2001, and 2002 First Year Students
http://www.udel.edu/ugs/pdfs/Demographic_Report.doc
LIFE
Comparison of GPA for LIFE vs. Non-LIFE Students- Spring 2002
http://www.udel.edu/ugs/pdfs/sp02_GPA_Comparison.doc
LIFE students displayed their final projects at the Fall 2001 and Fall 2002 LIFE Fest events. The final project is a culminating group project that enables students to synthesize work they have been addressing in the UNIV101 course as well as its connections to the other cluster courses. In Fall 2002, a group of UD faculty and staff served as judges to evaluate the final projects.
The vast majority of LIFE students who included open comments on their end of Fall 2002 semester evaluation said they would recommend the LIFE program to other students. Although a few (approximately 6-8 each year) students said they did not like nor benefit from living in the same room with another LIFE student, the majority of respondents noted the immediate value of building relationships with other students and having the opportunity to study and attend class with the same group of students in their cluster. When asked what UNIV101 activities built upon their learning, many respondents mentioned the final cluster project, guest speakers, and field trips. LIFE students also said they valued and perceived benefit from the living-learning environment in which they studied and socialized with a group of peers. The tight knit living-learning communities allowed students a level of comfort in which they could meet professors, ask each other questions, and in general, raise their level of academic self-concept that helped guide them in their first year success.
For additional information, please see:
LIFE Student Evaluation Summary-Fall 2002
http://www.udel.edu/ugs/pdfs/Fall_2002_LIFE_Evaluation.doc
LIFE Student Evaluation Summary-Fall 2001
http://www.udel.edu/ugs/pdfs/Sum_Report_Fall 2001_LIFE.doc
A total of 22
pathways courses have been offered during the past three years with a number
being offered more than once for a total of 39 Pathway course offerings. The
total enrollment of these 39 courses has been 2200 students (1032 freshmen and
1168 non-freshmen). Pathways courses are high quality and satisfy Goals 1, 2,
3, and 6 of General Education. The biggest difficulty with the growth of
Pathways courses has been developing enough capacity within department
resources so that these courses can be a numerically significant component of
the GEI. Many Pathways courses, developed with a significant investment of
faculty time, have only been offered for one semester.
To examine student perceptions of their Pathways course, a 13-item course evaluation was developed in Fall 2001 that specifically addressed four of the General Education goals. Alpha coefficient for internal reliability of the instrument was .931, indicating that 93% of the variants in the scores was due to true, accurate measurement.
Overall, students have reported positive marks for the Pathways courses. Student scores have not varied widely in the four semesters that Pathways evaluations have been used, thus the table below provides an example of student responses.
While the majority of students in Fall 2002 Pathways courses agree that the course has helped them strengthen skills, there were approximately 1/3 of the students who do not agree. For example, 43% of the f02 students did not believe that the course strengthened their written communication skills, and 48% did not believe that the Pathways course helped them determine future courses to take, an important goal of the Pathways program.
In both the multiple choice and open-ended responses, the vast majority of students said that their Pathways course enabled them to work with others in small groups. The majority of students benefited from small group work and discussion, saying that it helped them think about ideas from a different perspective, become more skilled in problem solving, express ideas, and reinforced time management skills.
Although the majority of students expressed positive remarks about their Pathways course, there were a small number of students who expressed dissatisfaction with the course. While there were comments in direct contradiction, a few students did not find the course to be intellectually interesting, too easy, or requiring too much writing. One question asked students, ‘This course is designated as a Pathways course. For you, what does this course lead to, or what is it a pathway to?’
Responses to this question revealed that some students received great value for the course—gaining more interest in the subject, a better understanding of the world, how material in this course relates to other information being learned. However, based on comments from other students, it appears that some students did not understand the intention of the Pathways course. One comment included by about 29 students was ‘I don’t know’ or ‘I don’t know what Pathways means.’
Table 1
Responses to Fall 2002 Pathways Evaluation
Percent
Agree/ Percent Disagree/
Strongly
Agree Strongly
Disagree
The course strengthened my oral communication 56.8 33.5
The course strengthened my written communication 56.8 43.2
Strengthened my information technology 64.0 36.0
Helped me think about issues & problems 80.1 19.9
Helped me integrate ideas 77.8 22.2
Helped me see connections across disciplines 63.7 36.3
Enabled work with students in groups 89.3 10.7
Format encouraged participation 61.1 38.9
I now have better idea of future courses 52.1 47.9
Made me aware of acad. opportunities outside class 42.7 57.3
Out-of-class activities enhanced learning 70.8 29.2
I used class info in other conversations 57.1 42.9
I’m now interested in taking related courses 59.0 41.0
For additional information, please see:
Pathways Course Evaluation Summary-Fall 2002
http://www.udel.edu/ugs/pdfs/F02_Pathways_Course_Eval_summary.doc
Pathways Course Evaluation Summary-Fall 2001
http://www.udel.edu/ugs/pdfs/f01_Pathways_summary_report.doc
Pathways Report on Writing Assessment-Spring 2003
http://www.udel.edu/ugs/pdfs/Pathways_report-03_lutz.doc
Pathway student representative
comments selected from a full set of comments to try to represent a diversity
of views.
· The course was a lot more interactive and interesting with more work given and more material covered.
· A different learning style and not just a straight boring lecture. More personal.
· The workload made the class less interesting.
· This class was more writing intensive and more detailed. My professor was helpful since it was more work.
· Lot of group work instead of books and tests. More interaction between my peers.
· This course helped more with presentations instead of just learning. I found the class to be more interesting and enjoyable. More active.
· You have to do something with the info in this class. A lot more busy work but more interaction. It was not as in depth as other classes.
· This class focused on what I did outside of class.
· It gives a different viewpoint for the same subject. It was better to get more than one perspective. It incorporates many areas into one.
· The class helped me get a better understanding of what I want to do. It provided a sense of the world around me.
· It helped me to connect subjects to one another.
· I’ve become more comfortable in front of students. Also a better understanding of how to work with others.
In Fall 2002, all UD faculty who had taught a Pathways course since 2000 were sent a short eight-item questionnaire to examine their perceptions of the Pathways program. Ten out thirty eight faculty members responded and were, overall, enthusiastic about their involvement. Many said that they themselves as well as the students received benefit from the program. Respondents said they were rewarded by working with faculty peers and having the opportunity to teach interdisciplinary courses.
Respondents believed that students benefited from learning to work with others, receiving a variety of learning experiences, and seeing connections between subject material and how it relates to other topics in the real world. Two major barriers related to faculty members’ time and the need to balance commitments to other courses in their department. Several respondents suggested that department chairs (and perhaps deans) be included in upcoming discussions to discern how Pathways courses can be offered in conjunction with other department requirements.
For additional information, please see:
Pathways Course Faculty Survey-Fall 2002
http://www.udel.edu/ugs/pdfs/UGS_Pathways_faculty_survey_report.doc
Pathways faculty representative
comments selected from a full set of comments to try to represent a diversity
of views.
In February 2003 an electronic survey was sent to all department chairs and program directors to query the department/program’s current use of capstone courses. Officials from 35 departments responded to the survey, seven of which have more than one program within that department. The table below shows that, of the departments that responded to the survey, which offer capstone courses and the percentage of majors that are required to enroll in a capstone experience.
There are a variety of activities included in the capstone courses. The three most frequent activities included in capstone courses were seminar format, thesis or other substantial writing assignment, and information on career preparation (all at 44%). Thirty-seven percent of the respondents said that an individual or team oral presentation was also part of this experience. Twenty-seven percent said that field experience was a component while 10 percent said some of their students participated in educational travel as part of their capstone experience. Respondents to this survey were also asked to rank a series of educational goals that were included in their capstone course. The rankings were somewhat distributed across many goals, but the most goal ranked highest was ‘fostering integration and synthesis within the major.” A more detailed summary of the capstone survey can be reviewed: Capstone Course Survey-Spring 2003.
This table represents the 34 departments and programs which responded to the survey and does not include all the departments and programs of the University.
Departments that offer one Percent majors Departments that
or more capstone courses: who
enroll: do not
offer capstone:
Anthropology 100 Art History
Art
Conservation 100 Biology
Business
Administration 100 Economics
Chemistry
75* Food & Res Economics
Civil Engineering 100 Museum Studies
Computer Info Sci 100 Psychology
Curatorial
Apprentice program w/out
majors
Disability
Studies (HEPP) 100
Education 100
Engineering
Technology 100
English 100
Entomology 0**
Geography
75
Geology
80
Health,
Exercise Science
90
History 100
Honors 100
Hotel, Rest, Instit. Mgmt 100
Legal
Studies 100
Mechanical
Engineering 100
Medical
Technology 100
Music 100
Philosophy 100
Plant
& Soil Sciences
50
Political
Science 100
Sociology 100
Theatre 100
Women’s
Studies 100
*
some percentages are an average based on differential
percentages for two or more programs in one department
**
course offered but only 2 students signed up.
For additional information, please see:
http://www.udel.edu/ugs/pdfs/sp03_Capstone_Surveys.doc
Based on the student, faculty, and support staff comments as well as some limited direct measures of student outcomes, data shared above (additional detailed information online http://www.udel.edu/ugs/pdfs/reports.html) indicates overall support and success of the UD General Education pilot program from 2000-2003. Information gathered over the past three years will be incorporated in recommendations currently being drafted that will be included in the full report on the pilot program submitted to the Faculty Senate in September 2003.