The Master of Science in Evaluation is an interdisciplinary degree intended to prepare students to contribute to human service, education, and other program and policy areas through the thoughtful, effective, and ethical use of evaluation models and methods.[1] The program combines coursework in evaluation theory and methods with hands-on experience in the practice of evaluation.

B. Program organization

An interdisciplinary degree, the MS in Evaluation cuts across academic units in the College of Human Services, Education, & Public Policy (CHEP). The School of Education (SOE), School of Urban Affairs & Public Policy (SUAPP), and the Department of Human Development and Family Studies (HDFS) are collaborating in the development and implementation of the program. (Letters of support and commitment from the Dean and Department Heads are provided in Appendix A.)

Administering unit: The CHEP Dean’s Office will be the administering unit. Staff in the Dean’s office will be responsible for interacting with the Office of Graduate Studies and handling all student paperwork. Locating the program in the Office of the Dean follows the models of the interdisciplinary undergraduate degree in Human Services, Education, & Public Policy, and of the Masters and PhD in Energy and Environmental Policy, which are inter-College degree programs administered by a non-academic unit (Center for Energy & Environmental Policy), whose head reports to the Dean.

Faculty: The program faculty is comprised of faculty representing each of the participating departments. (See Appendix B for a list of core and affiliated faculty.) The eight core faculty teach at least one of the required courses in the program each year and have agreed to advise at least one student. The program also has four affiliated faculty who teach one or more of the area of emphasis or elective courses. ~~All faculty who have primary academic appointments in one of the three participating units will have joint appointments in the Evaluation program.~~ The Academic Director of the Evaluation program has no input to faculty workloads or evaluations.

Faculty governance: A Faculty Governance Committee consisting of the core faculty will be formed. The Committee will:

· Be responsible for curricular and admissions decisions and for ensuring that program decisions are consistent with College and Faculty Senate policies and guidelines;

· Always include at least one member from each of the three collaborating units;

· Select an Academic Program Director to coordinate program activities; and

· Adopt by-laws governing its own procedures.

The by-laws will establish the responsibilities and length of term of the Academic Program Director and will include an explicit requirement for consultation and coordination with the participating departments through the Advisory Committee (see below). Programmatic decisions about curriculum, size, admissions requirements, or configuration of the program will be submitted to the Advisory Committee for approval.

Curricular decisions: Courses that are under the purview of participating departments will continue to be the sole responsibility of the participating department. Almost all the courses required for the Evaluation program are also required for one or more degree programs in SOE, HDFS, or SUAPP, so they are expected to be stable.

The courses that will be under the governance of the Faculty Governance Committee and will be taught under the EVAL prefix are: Evaluation Models and Approaches (3 credits); Advanced Seminar in Evaluation (3 credits); Special Projects (2 credits); and Portfolio and Competency Analysis Paper (1 credit). The first two of these courses will be cross-listed with the participating departments; Special Projects and the Portfolio will be restricted to students in the Evaluation program. Changes to specific components of these courses (e.g., changes to the portfolio requirements (described in appendix C)), do not require approval of the Advisory Committee.

Admissions decisions: Admissions decisions, including offers of financial aid, are the purview of the Faculty Governance Committee. Each year an admissions committee will be formed, including at least one representative from each participating department. As stated above, decisions about admissions requirements or about changes in the anticipated size of the program will be subject to Advisory Committee review and approval.

Advisory Committee: To ensure that the interests of the three participating units are appropriately considered in decisions related to the Evaluation program and not already under the purview of one of the departments, an advisory committee made up of the three unit heads – the Director of the School of Education, the Director of the School of Urban Affairs & Public Policy, and the Chair of the Department of Human Development & Family Studies, or their designee -- will be asked to review and approve curricular and other programmatic decisions made by the Faculty Governance Committee.

C. Learning outcomes

The successful graduate of the MS in Evaluation will:

a) Be able to explain the historical and philosophical underpinnings of evaluation and their implications for evaluation practice;

b) Be able to apply the ethical standards and guiding principles of the profession, including striving for cultural competence;

c) Be able to explain the logic of evaluation/research design, including mixed-method designs;

d) Have basic skills in the collection and analysis of qualitative and quantitative data individually, and intermediate skills in at least one;

e) Develop specialized knowledge of a program/policy area (e.g., welfare, education, health) or context (e.g., government, nonprofit), or advanced technical skills (e.g., cost-benefit analysis, multivariate data analysis, ethnography);

f) Have intellectual humility in the form of an understanding of the programmatic and methodological areas in which s/he does not have expertise; and

g) Be able to conduct an evaluation, including negotiating evaluation questions, developing program theory, matching evaluation plan to questions and resources (money, time, staff), collecting and analyzing data, interpreting and reporting results, and disseminating and encouraging the use of findings.

D. Assessment and benchmarks for success

Outcomes (a) – (e) will be assessed by successful completion of required coursework and electives identified in consultation with an advisor. Outcome (g) will be assessed by successful completion of fieldwork experiences provided during core courses. All the outcomes (including (f)) will be assessed in the final semester of a student’s program by the compilation of a portfolio and preparation of a competency analysis report, in which the student rates him/herself on the above outcomes.

At the midpoint in a student’s program, his/her program of study will be reviewed to determine if he or she is making satisfactory progress through the program. A minimum average of B (GPA of 3.0) is required for successful completion of the program.

II. Program Rationale and Demand

A. Institutional Factors

Compatibility with Academic Priorities of the University: The program relates to The Professional Education Initiative outlined in the Strategic Planning Report issued in April 2008, which states: “Our greatest impact can be achieved by educating and training the professionals and leaders who serve society in essential roles, and by closer integration of public service with research and education.” Evaluators serve an essential role. When evaluations are conducted by skilled evaluators, they provide credible evidence about the efficiency and effectiveness of social interventions, evidence which is used to inform decisions in the public, non-profit, and for-profit sectors. Evaluation is a form of applied research that is practiced to support the public good, and the MS in Evaluation will work closely with CHEP’s applied research centers, thus integrating the research and public service missions of the university.

By providing a unique interdisciplinary degree, the MS in Evaluation also supports the Academic Priority of strengthening graduate programs in areas of demonstrated comparative advantage, particularly the item, “Invest selectively in interdisciplinary graduate programs in areas of demonstrated comparative advantage and community need” http://www.udel.edu/provost/Priorities.html). The program’s comparative advantage lies in the paucity of other evaluation graduate programs in the mid-Atlantic, and the existence of only two other truly interdisciplinary graduate degrees in evaluation in the country. Community need is demonstrated by the existence of two thriving regional affiliates of the American Evaluation Association (the Eastern Evaluation Research Society and the Washington Evaluators), the large number of evaluation projects engaged in by CHEP’s public service centers (e.g., Center for Community Research & Service, Delaware Education Research & Development Center, and the Center for Disability Services), and the proximity of major employers (state and federal agencies, nonprofit agencies, and foundations). A program specifically focused on evaluation will raise the visibility of the University of Delaware as a resource for the evaluation needs of local, regional, national, and international organizations. The diversity of opportunities in evaluation also indicate the value of such a program: Potential sources of graduate assistantships include evaluations of educational and human service programs, visitor studies for museums and botanical gardens, evaluation of research programs, among other possibilities. Moreover, the need for evaluation professionals is likely to increase in response to the continued growth of attention to accountability for public funds and the increased interest in evaluation internationally. (Universities in the Netherlands and Italy are in the process of developing similar programs.)

Planning Process: This proposal is the result of an awareness of the demand for graduates with evaluation skills and experience on the part of several key people, including former CHEP Dean Timothy Barnekov and Interim Dean Michael Gamel-McCormick, the Directors of the SOE and SUAPP, the Chair of the Department of HDFS, and the Director of the Delaware Education R&D Center (DERDC). In response to the demand, a new Public Service faculty member with expertise in evaluation, Dr. Leslie Cooksy, was hired with a joint appointment in SOE and DERDC in Fall 2007. On October 19, 2007, the SOE, SUAPP, and DERDC Directors and the HDFS Chair, plus faculty representatives from each unit, met and agreed to propose a MS in Evaluation as an interdisciplinary degree.

In Fall 2007, a review of the existing graduate programs in the United States found a gap in opportunities for graduate education in evaluation, with only 17 universities in the United States claiming to offer a degree or concentration in the discipline. Interviews with the directors of six of these programs were conducted to determine the demand and to learn about each program’s curriculum. These interviews supported the value of the proposed program and the expectation of demand. (For example, the University of Illinois at Urbana-Champaign is currently developing an on-line Masters program to supplement their doctoral program in order to meet demand for a professional degree.)

To gauge interest in the program beyond CHEP, Dr. Cooksy met with directors of the Organizational Effectiveness, Development, & Change degree in the College of Business & Economics and the Health Promotions degree in the College of Health Services to gauge their interest in the program. She also consulted with directors of other interdisciplinary programs, such as the Masters in Energy & Environmental Policy, to learn how cross-departmental programs are structured.

Impact of proposed curricula on other programs of the University: The proposed curricula builds on existing courses. The academic units offering those courses have been consulted and approve the cross-listing of their courses with the new program. Based on conversations with faculty across CHEP and in other colleges, the two new courses that will be developed for the program are likely to be of interest to students in other programs.

B. Student Demand

The program anticipates an enrollment of 1-2 students in the first two years, growing over time to 10-12. These enrollment figures were derived by interviews with directors of evaluation programs in other universities, and the commitments of core faculty. Six directors were contacted. The largest Masters program (Claremont Graduate University) reported starting with only two students enrolling in the first couple of years. They now have 30. The other programs fell within that range. When asked specifically about possible difficulties recruiting students for a Masters degree (instead of a doctorate), the directors were uniformly positive about the need and demand for Masters-level training.

· Estimated number of credit hours to be generated: 37 credit hours

· Number of program graduates: None so far, see anticipated student demand for expected enrollment.

· Number of new majors: The MS in Evaluation will be a new graduate program, not a modification of an existing program.

Two of the four new courses required for the program are expected to serve the needs of students in the School of Education, the Department of Human Development and Family Studies (HDFS), and the School of Urban Affairs & Public Policy. These courses are Evaluation Models & Approaches and Advanced Seminar in Evaluation. (Evaluation Models & Approaches, taught as an experimental course in Education by Cooksy in Spring 2008, had students from Education, Urban Affairs, Marine Policy, and Business. Students from Education, Urban Affairs, and HDFS have registered for an experimental course scheduled for Fall 2008.)

The students in the new program will also be required to take 6 courses that are already offered and will be cross-listed with the new program.

The program will be most attractive to currently employed professionals or those returning to school after participation in a volunteer service program (e.g., Peace Corps, Americorps). Courses will be taught in the evening or in block format to enable the participation of working students.

C. Transferability

We do not expect students to transfer into the program from other UD programs or programs at other universities. Students may change their classification from masters to doctoral status without receiving the masters if they are accepted into the doctoral program in Research Methodology & Evaluation (RME) in the School of Education, which requires some of the same courses required by the MS in Evaluation.

D. Access to graduate and professional programs

The program is designed as a terminal degree. While some students may choose to pursue a doctorate, most are expected to graduate and begin practice as professional evaluators.

E. Demand and employment factors

This program is specifically intended to prepare students for careers in evaluation. Programs, organizations, personnel, products, policies, and other entities can be evaluated. While most students will go into jobs in program or organizational evaluation, the skills learned will be transferable to other kinds of evaluation.

Audience for the program: It is anticipated that most of the students in the program will already have some work experience as service providers or managers. In their work environments, they discover the importance of evaluation for accountability and program improvement. This discovery leads to the desire for increased knowledge about evaluation so that they can pursue careers in the field.

Unique career paths: There are three likely career paths for the graduates of the MS in Evaluation.

· First, the experience of the School of Urban Affairs & Public Policy, which places all its students in graduate assistantships intended to provide practical experience in professional settings, indicates that assistantships often grow into full-time positions after graduation. The Delaware Education Research & Development Center has agreed to work with the new program to find graduate assistantships that will give the evaluation students similar opportunities. Given the demand for evaluation in the area, the program is expected to generate new assistantships that will not replace opportunities for students in existing programs.

· A second likely career path for the graduates of the program is to return to the organizations that they came from, now equipped with the skills in evaluation.

· A third path is finding evaluation positions through such sources as the American Evaluation Association’s job listing service. On May 9, 2008, AEA’s website had 108 jobs listed, with close to 40% (43) of those listed in the previous month. On Monster.com, the popular job search internet website, 1,240 jobs are listed under “evaluation” in nine categories (e.g., education, government and policy, nonprofit). (More than 5000 are listed if the search is not limited to specific categories.)

F. Regional, state, and national factors

Comparable programs in the state/region: There are no comparable programs in Delaware or the mid-Atlantic region. University programs in evaluation are listed on the website of the American Evaluation Association. While evaluation is growing as a profession, there is a gap in educational opportunities for people who want to become evaluators. On the American Evaluation Association website, 17 U. S. universities have self-identified as having graduate programs in evaluation. Of these, only two are interdisciplinary (University of Minnesota and Western Michigan University). Only three others are widely known for their evaluation program (Claremont Graduate University, Florida State University, and the University of Illinois).

Influences on the proposed curriculum: There is no accreditation process for graduate evaluation programs or licensing requirements for professional evaluators in the United States.

G. Other strengths

The University of Delaware is uniquely poised to compete for the top students in evaluation. It would be the only program in the mid-Atlantic region. It has a strong base of applied researchers with experience in evaluation already on faculty. CHEP’s public service/applied research centers are another strategic resource; they are a likely source of field experiences for the evaluation students. In addition, Leslie Cooksy, a nationally renowned evaluator (recently voted incoming (2010) President of the American Evaluation Association), was hired as a Public Service faculty in the School of Education with the specific charge of developing and coordinating an interdisciplinary graduate program in evaluation.

III. Enrollment, Admissions, and Financial Aid

A. Enrollment

The program will be effective as of the 09-10 academic year, with recruitment beginning in Spring 2009. Enrollments will be limited based on faculty availability for advisement. Of the seven core faculty, 7 have committed to advising at least one student; one (Noble) has committed to advising at least two students; and one (Cooksy) has committed to advising four students, which will limit enrollment to 12. Enrollment will also be limited by the availability of financial support through research assistantships and other sources. (The program does have a commitment from the Education R&D Center to support students.) In addition, enrollment will be limited to those students who meet the admissions requirements.

B. Admissions Requirements

The admissions criteria identify those applicants who are likely to be successful evaluation professionals. Specifically, the program seeks to attract applicants who have: (1) a demonstrated commitment to social change and betterment through effective programs and other interventions, and (2) an academic and/or professional background that indicates the ability to successfully complete the program. Acceptance to the program is based on a composite of the applicant’s scholastic record, standardized test scores, letters of reference, and personal statement. Relevant work experience may also be taken into consideration. Admission is selective and competitive based on the number of well-qualified applicants and the limits of available faculty and facilities. Those who meet standard minimum academic requirements are not guaranteed admission, nor are those who fail to meet these requirements necessarily precluded from admission if they offer other appropriate strengths. Applicants for the Master of Science in Evaluation will apply to the Office of Graduate Studies.

The specific criteria are:

1. Applicants must have a minimum of a baccalaureate degree. Evaluation is an interdisciplinary field, so the discipline in which the applicant received his or her degree is not necessarily a decisive factor in admissions.

2. Applicants must submit a written statement of the reasons for their interest in evaluation, their motivation to pursue a graduate degree, and their professional goals and objectives.

3. Applicants should have an overall undergraduate Grade Point Average (GPA) of 3.0 or higher (on a scale of 4.0 = A).

4. Applicants should have a GPA of 3.2 or higher in their undergraduate major.

5. Applicants must take the Graduate Record Exam (GRE), which will be evaluated as part of the overall admissions process. A minimum GRE score of 1100 is required for admission, but may be waived if other academic or professional achievements are exceptionally strong.

6. If English is not an applicant’s first language, then the applicant must demonstrate a satisfactory command of English. The TOEFL (Test of English as a Foreign Language) is required of all foreign applicants. A minimum score of 600 (paper-based test), 250 (computer-based test), or 100 (TOEFL iBT) is required for consideration for admission.

7. Applicants must provide letters of recommendation from three (3) people familiar with the candidate’s academic record and/or professional achievement.

C. Student Expenses and Financial Aid:

Students will be expected to have access to a personal computer. There are no other required expenses beyond the traditional books and supplies. Financial aid will be available to some students through research assistantships in the applied research centers in CHEP or other sources.

IV. Curriculum Specifics

A. Institutional Factors:

The program will award a Master of Science in Evaluation. This is the appropriate degree because the program will develop the students’ technical skills in evaluation procedures and methods. The emphasis will be on professional practice rather than on evaluation theory and research.

B. Curriculum

1. Requirements:

a) The MS-Evaluation requires the student to complete a minimum of 37 credits. The curriculum includes (a) core courses in evaluation theory and practice (6 course credits, plus 2 credits in special projects and a 1 credit portfolio), (b) core courses in applied research methods (16 credits), (c) elective courses in an area of methodological specialization (6 credits), and (d) elective courses in a program area or advanced technical skills (6 credits).

b) To graduate from the program, each student will write a competency analysis paper, describing their level of achievement on a set of core competencies (evaluation theory, research methods (including measurement and data collection methods), program evaluation design and management, program evaluation implementation, data analysis, and communication of evaluation results. The competency analysis paper will be accompanied by a portfolio supporting the student’s assessment of his or her competencies. (See Appendix C for description of the portfolio requirement. This approach was decided on based on conversations with directors of other programs. It follows the model used by Florida State University.)

c) Each student will be required to have a committee for his or her Portfolio Review and Competency Exam. The committee will be comprised of two graduate faculty members, the chair of which must be a core or affiliated faculty member in the Evaluation Program. To provide guidance on the development of the portfolio, the committee will be developed one calendar year before the expected date of graduation, and the chair will be responsible for advising the student, including guiding the selection of and approving electives.

d) The curriculum is designed to be completed in 2 years of full-time study (a minimum of 9 hours/semester). However, there are no full-time residency requirements. Although there will be no separate part-time track, the program will be open to part-time students. As noted earlier courses will be offered in the evening or in block schedule format to increase access by working students.

e) Students are unlikely to be involved in research projects. However, approval of all research projects involving humans, even administration of a survey, must be obtained prior to beginning any study. Information about obtaining approval may be found on Human Subjects in Research (http://www.udel.edu/OVPR/humans/humans.html). If a project involves animal subjects, an Animal Use Proposal must be completed and submitted to the Institutional Animal Care and Use Committee (http://www.udel.edu/OVPR/animals/animals.html).

f) Insurance indemnity for students is included under the general University program.

2. Sample curriculum following the format in the Graduate Catalog: See Appendix D.

3. Curriculum Satisfies UD and CHEP Requirements: Yes

C. Appendices

A – Letters of support [provided in separate files]

B -- Core and Affiliated Faculty

C -- Description of Competency Analysis Paper and Portfolio

D -- Curriculum following the format in the Graduate Catalog

E – Letter from Susan Brynteson, May Morris Director of Libraries [provided in separate files]

F -- Detailed description of faculty

G -- Map of Curriculum to Program Outcomes

V. Resources Available

A. Learning Resources:

The library has assessed the needs of the program. The letter concludes that “The University of Delaware Library is well able to support the MS in Evaluation.” (See Appendix E.)

B. Faculty/Administrative Resources:

See Appendix F for a full description of the qualifications of the core faculty. The Office of the Dean is providing staff resources to process student paperwork and act as liaison with the Office of Graduate Studies. All paperwork and decisions associated with courses offered through participating departments will be the responsibility of those departments.

C. External Funding:

Funding for students will be provided through research assistantships in the Delaware Education Research & Development Center, and other CHEP public service centers. The program is expected to generate new assistantship opportunities with no net effect on research assistantships for existing programs. Part-time students will not be eligible for assistantship support.

VI. Resources Required

Learning Resources: Access to key journals and texts in the field are required. As stated above, the library has an appropriate collection, with access to all the recognized journals in the field available through e-journals.

B. Personnel Resources: No new faculty positions are required.

C. Budgetary Needs: Because the program relies primarily on courses that are already offered and that will be open to the MS Evaluation students, the program has minimal budgetary requirements. The two new courses and development of special projects opportunities will be the responsibility of the program director, Leslie Cooksy, who was hired specifically for the development of this program. Thus, adding two new courses will not impact the availability of faculty resources for other courses.

VII. Implementation and Evaluation

A. Implementation Plan: One of the new courses – Evaluation Methods & Approaches – will be taught as an experimental course (as EDUC 667) in the Fall and Spring of the 2008-09 year. The second new course – Advanced Seminar in Evaluation – will be taught for the first time in Fall 2009. Other courses are already being taught and the cooperating programs have agreed to accept Evaluation students in the courses. When the MS in Evaluation proposal is approved, core courses will be cross-listed with the EVAL prefix.

B. Assessment Plan: See Attachment G for curriculum map to program outcomes and methods of assessment. A more detailed assessment plan will be developed with the core faculty when the program is approved.

Appendix A

Letters of Support & Commitment from the CHEP Dean and Heads of Participating Units

Appendix B

Core and Affiliated Faculty

Core Faculty

(All have agreed to advise at least one student unless otherwise noted.)

School of Education

Leslie Cooksy, Associate Professor – will advise up to 4 students

Joseph Glutting, Professor

Robert Hampel, Professor

Ratna Nandakumar, Professor

Department of Human Development and Family Studies

Donald Unger, Professor and Interim Chair of HDFS

School of Urban Affairs & Public Policy

Audrey Noble, Director,

Delaware Education Research & Development Center – will advise up to 2 students

Paul Solano, Associate Professor

Danilo Yanich, Associate Professor

Affiliated Faculty

School of Education

Joan Buttram, Policy Scientist, Delaware Education R&D Center

Department of Human Development & Family Studies

Michael Gamel-McCormick, Interim Dean of CHEP

School of Urban Affairs & Public Policy

Maria Aristigueta, Professor and School Director

Edward Ratledge, Director, Center for Applied Demography & Survey Research

Appendix C

Competency Analysis Paper & Portfolio Description

The capstone for the MS in Evaluation is the preparation of a short (5-10 double-spaced pages) paper in which the student assesses his or her competency on each of the following criteria. (The criteria are adapted from those used by the MS in Evaluation at Florida State University, and are based on empirical studies of expected competencies in evaluators (e.g., King, Stevahn, Ghere, & Minnema, 2001; Stevahn, King, Ghere, & Minnema, 2005; and others). The competency analysis paper will refer to a portfolio that supports the analysis. Students are not expected to excel on each of the following criterion, but are expected to be able to demonstrate at least a moderate degree of competency on each.

1. Demonstrated knowledge of evaluation theory, models, and approaches, including knowledge of the American Evaluation Association’s Guiding Principles (www.eval.org) and The Program Evaluation Standards (Sanders et al., 1994). Key elements include the identification of different models and approaches, ability to discuss theoretical issues associated with each, skill in developing evaluation questions and evaluation designs appropriate for at least two models. These skills can be demonstrated by deliverables for coursework and evaluation projects.

2. Demonstrated knowledge of research methodologies applicable to systematic, rigorous evaluations. Demonstrated knowledge of quantitative, qualitative and mixed data collection methodologies, including data analysis, can be shown by deliverables for coursework (e.g., research proposals, measurement assignments, statistics exams, qualitative evaluation projects).

3. Demonstrated competency in conducting program evaluations. Key skills include stakeholder identification, development of evaluation questions, the use of planning tools such as logic models and design matrices, match of evaluation plan to questions and resources (money, time, staff), quality assurance/quality control of data collection and analysis. These can be demonstrated through on-the-job experience or research assistantships, assignments conducted for core evaluation courses, and special projects.

4. Demonstrated competency in a specialized area. The specialized area may be methodological (e.g., economic evaluation) or programmatic (e.g., educational leadership, family studies). Demonstrated competency in the substantive area could include deliverables from elective courses, directed independent study, or supervised research projects that apply program evaluation to the substantive area.

5. Demonstrated competency in reporting research and evaluation outcomes in ways that encourage the use of evaluation findings.

Students will be asked to rate themselves on each of the above criteria using a rating scale of highly skilled, moderately skilled, minimally skilled, not at all skilled. The competency analysis report discusses the skills the student feels that he or she has acquired or has failed to acquire while a student in the Evaluation MS program. The report will have 5 sections, one for each competency. Each section will be no more than 3 pages long. Where a moderate or high rating is given, the discussion will describe how relevant items in the portfolio support the rating. The number of documents/items in the portfolio will be limited to 12. The student will be expected to include an annotated table of contents for the portfolio, including (1) the nature of the item, (2) the semester during which the item was produced, (3) the course or project for which it was produced, and (4) the student’s role in developing the item.

References

King, J. A., Stevahan, L., Ghere, G., & Minnema, J. (2001). Toward a taxonomy of essential evaluator competencies. American Journal of Evaluation, 22, 229 - 247.

Stevahan, L., King, J. A., Ghere, G., & Minnema, J. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26, 43 – 59.

Appendix D

SAMPLE CURRICULUM IN CATALOG FORMAT

MS IN EVALUATION

Telephone: (302) 831-6872

http://www.udel.edu/...

PROGRAM OVERVIEW

The Master of Science in Evaluation prepares professionals for the successful design, implementation, and management of program, policy, and organizational evaluation. Students are prepared in the science and art of evaluation. The specific skills that will be developed are: identifying evaluation needs, negotiating evaluation questions, developing program models, designing evaluation studies that are responsive to information needs, and carrying out and reporting the results of these studies in ways that improve the ability of programs, policies, and organizations to address social problems. The program grounds these skills in the theoretical foundations for evaluation and the ethical principles of evaluation practice.

The program is designed to meet the needs of both traditional graduate students and working professionals, with graduates having the skills and knowledge to work with a wide variety of populations and in diverse settings. The program provides opportunity to pursue specific areas of emphasis within Evaluation, including (but not limited to): measurement, quantitative design and analysis, qualitative evaluation, economic evaluation, or customized study focused on a specific program area (family studies, community development, educational leadership, organizational development, health promotion).

REQUIREMENTS FOR ADMISSION

Students will be admitted to the program based upon enrollment availability and their ability to meet the following entrance requirements.

2. Applicants must submit a written statement of the reasons for their interest in evaluation, their motivation to pursue a graduate degree, and their professional goals and objectives.

3. Applicants should have an overall undergraduate Grade Point Average (GPA) of 3.0 or higher (on a scale of 4.0 = A).

4. Applicants should have a GPA of 3.2 or higher in their undergraduate major.

5. Applicants must take the Graduate Record Exam (GRE), which will be evaluated as part of the overall admissions process. A minimum GRE score of 1100 is required for admission, but may be waived.

6. If English is not an applicant’s first language, applicants from abroad must demonstrate a satisfactory command of English. The TOEFL (Test of English as a Foreign Language) is required of all foreign applicants. A minimum score of 00 (paper-based test), 250 (computer-based test), or 100 (TOEFL iBT) is required for consideration for admission.

7. Applicants must request letters of recommendation from three (3) people familiar with the candidate’s academic record and/or professional achievement.

Admission is selective and competitive, based on the number of well-qualified applicants and the limits of available faculty. Those who meet stated minimum academic requirements are not guaranteed admission, nor are those who fail to meet those requirements necessarily precluded from admission if they offer other appropriate strengths. See the Admission Information chapter in this catalog for additional information.

REQUIREMENTS FOR THE DEGREE

The Master of Science in Evaluation requires 37 credit hours of coursework at the 600 and 800 level. The 30 credits of coursework must include 25 credits of required courses, and 12 credits of advisor approved coursework.

Credit Requirements:

Core Credits (Areas A + B) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . …. 25

Area of Emphasis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Electives (advisor approved) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Total number of required credits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

A. Courses Required Within Evaluation

· HDFS 637 Program Planning, Assessment, & Evaluation (3) – to be cross-listed with EVAL prefix

· EVAL 755 Evaluation Models and Approaches (3)

· EVAL 756 Advanced Seminar in Evaluation (3)

· EVAL 757 Special Projects (2)

· EVAL 758 Portfolio and Competency Analysis Paper (1)

Total Credits from Area A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

B. Courses Required From Other Programs

· Measurement (3) [EDUC 865 Educational Measurement Theory or HDFS 655 Measurement Techniques for Children & Families]

· Research Methods (3) [EDUC 607-Educational Research Procedures; HDFS 815: Research Issues & Designs; UAPP 800-Research Methods & Data Analysis]

· Statistics (3) [EDUC 665-Elementary Statistics or UAPP 815-Public Management Statistics]

· Qualitative Methods (3) [EDUC 850: Qualitative Research in Educational Settings]

· Survey Design (1) [UAPP 676 – Survey Research]

Total Credits from Area B…………………………….13

C. Area of Emphasis [6 credits to be selected with approval of advisor]

Qualitative Methods

· UAPP 804 Qualitative Methods for Program Evaluation

· EDUC 858 Advanced Qualitative Methods

Quantitative Methods

· EDUC 812 Regression and Multivariate Models in Education

· EDUC 856 Introduction to Statistical Inference

· EDUC 862 Design and Analysis of Experiments

· EDUC 874 Multivariate Data Analysis

Economic Evaluation

· UAPP 834 Economics in Public and Nonprofit Sectors

· UAPP 827 Program and Project Analysis

D. Electives [6 credits in a program area to be selected with approval of advisor]

· Program areas include, but are not limited to:

o Adult education (EDUC 670, 699, 883)

o Community development (UAPP 608, 639, 645)

o Family services (HDFS 670, 850, 855, 870, 875)

o Management (UAPP 819, 684).

o Policy analysis (UAPP 806, 652)

Appendix E

Formal Library Assessment

Appendix F

Description of Faculty Resources

Core Faculty

School of Education (SOE)

Leslie Cooksy, Associate Professor, Delaware Education Research & Development Center (DERDC), has taught program evaluation, research methods, and qualitative methods for several years, and has conducted several evaluation projects for Delaware state agencies as well as for federal and international organizations. She has held leadership positions in the American Evaluation Association, including serving as Associate Editor of the American Journal of Evaluation (AJE) and a member of the Board of Directors, and is currently incoming President (2010). She has published in New Directions for Evaluation, Evaluation & Program Planning, and AJE. In the MSMS in Evaluation, she will teach the two new courses – Evaluation Models & Approaches and the Advanced Seminar in Evaluation.

Joseph Glutting, Professor, is part of the SOE’s Research Methodology & Evaluation doctoral specialization. His areas of expertise include school psychology, psychoeducational assessment, and educational measurement. He has published in numerous journals, including Applied Neuropsychology and the Journal of Special Education. His statistics courses are part of the program core and the area of emphasis in quantitative methods for the MSMS in Evaluation.

Robert Hampel, Professor and DERDC Faculty Affiliate, teaches program evaluation and qualitative methods. His areas of expertise include contemporary educational policy and the history of education. He has contributed to the Encyclopedia of Educational Reform and Dissent and the Encyclopedia of the Social and Cultural Foundations of Education , and has published in the Phi Delta Kappan, Education Week, International Journal of Educational Reform, and other journals. In the MSMS in Evaluation, Professor Hampel teaches the core course in qualitative methods (EDUC 850).

Ratna Nandakumar, Professor, is currently the coordinator of the Research Methodology & Evaluation faculty group in the SOE’s doctoral program. She teaches courses in the application of statistics and psychometrics in education and social sciences. Her research focuses on Item Response Theory in psychometrics: Dimensionality and Differential item functioning (DIF), including the development and modifications of statistical methodologies. She has published in Applied Psychological Measurement, the Journal of Educational Measurement, and other journals. In the MS in Evaluation, Professor Nandakumar teaches a core course in measurement as well as the courses for the optional area of emphasis in measurement.

Department of Human Development & Family Studies

Donald Unger, Professor, Acting Department Chair, and Faculty Affiliate in the Center for Disability Studies, teaches courses on the development, delivery, and evaluation of services for individuals and their families. His research has focused on the development and evaluation of services for families with children and/or parents with special needs, teen parent families, families in poverty, and families with children at risk for child abuse and/or substance abuse. He currently serves on the editorial board of Family Relations, has also been on the editorial boards of Marriage and Family Review, the Child, Youth, and Family Services Quarterly, and Children's Services, has published scholarly articles and book chapters, has edited two books, and was awarded Fellow status in the American Psychological Association. Dr. Unger teaches one of the core courses in the MS in Evaluation --- HDFS 637: Program Planning, Assessment, & Evaluation.

School of Urban Affairs & Public Policy

Audrey Noble, Assistant Professor, Director of the Delaware Education Research & Development Center, has led numerous policy studies for the state of Delaware and participated in national and international reviews. Her research interests include educational policy analyses, specifically related to the consequential validity of testing in public schools, and qualitative methodologies. In the MS in Evaluation, she teaches the core course in qualitative methods offered by SUAPP, and the advanced courses for the qualitative methods area of emphasis offered by SOE.

Paul Solano, Associate Professor and Director of the Health Services Policy Research Group, has conducted numerous health and health economic studies for NIH, federal agencies, and state agencies and non-profit organization, and specializes in econometric analyses of health program and policies. He has published an array of health and finance articles in books and journals including Health Policy, and The Journal of Mental Health Policy and Economics. He has served many times as NIH grant reviewer and evaluator for NIH site visits as well as reviewer for several health economic journals. His research interests include public finance and health economics. In the MS in Evaluation, Dr. Solano teaches courses in the area of emphasis in economic evaluation.

Danilo Yanich, Associate Professor, Director of the Urban Affairs & Public Policy Program, and Policy Scientist in the Center for Community Research & Service, conducts research, teaching and public service in several areas of urban policy including: (1) the relationship between the media, citizenship and public policy; (2) crime and criminal justice policy and (3) comparative urban governance. He has published in the Journal of Urban Affairs, Crime & Delinquency, and Crime, Law, & Social Change, among other venues. In the MS in Evaluation, he co-teaches the core course in research methods.

Affiliated Faculty

Maria Aristigueta, Professor, SUAPP Director, and Senior Policy Fellow with the Institute for Public Administration, conducts research primarily in the areas of performance management and organizational behavior. These areas of expertise are also the focus of her teaching. Her publications include Managing for Results in State Government, and many publications in scholarly journals or book chapters on such topics as the use of social indicators, performance measurement, teaching of intrapersonal skills, and the use of strategic planning for economic development. For the MS in Evaluation, Dr. Aristigueta teaches courses in the elective program area of management.

Joan Buttram, Educational Researcher, Delaware Education R&D Center and School of Education, conducts program evaluation and research studies related to school improvement, educational leadership, and professional development. In addition to numerous third-party evaluation and policy studies, she has published in Educational Leadership, the American Journal of Evaluation, and Research in Rural Education. She teaches the evaluation course in the School of Education’s doctoral Educational Leadership and Curriculum, Technology, and Higher Education programs.

Michael Gamel-McCormick, Professor, HDFS, and Interim Dean of CHEP, leads research and evaluation studies in the areas of early childhood programs for children with and without disabilities, teacher and professional preparation to work with young children with disabilities, and others. His teaching in research issues and design will contribute to the MS in Evaluation.

Edward Ratledge, Associate Professor, SUAPP, and Director, Center for Applied Demography & Survey Research, has conducted opinion research and statistical analysis in areas such as health and transportation policy as well as general demographic analysis for state and local government and the private sector. His research interests include management information systems, econometrics, and criminal justice systems. His course on survey design is a core course in the MS in Evaluation curriculum.

Appendix G

Assessment Plan

The learning objectives for the MS in Evaluation are that students will:

1. Be able to explain the historical and philosophical underpinnings of evaluation and their implications for evaluation practice;

1. Be able to apply the ethical standards and guiding principles of the profession, including striving for cultural competence;

2. Be able to explain the logic of evaluation/research design, including mixed-method designs;

3. Have basic skills in the collection and analysis of qualitative and quantitative data individually, and intermediate skills in at least one;

4. Develop specialized knowledge of a program/policy area (e.g., welfare, education, health) or context (e.g., government, nonprofit), or advanced technical skills (e.g., cost-benefit analysis, multivariate data analysis, ethnography);

5. Have intellectual humility in the form of an understanding of the programmatic and methodological areas in which s/he does not have expertise; and

6. Be able to conduct an evaluation, including negotiating evaluation questions, developing program theory, matching evaluation plan to questions and resources (money, time, staff), collecting and analyzing data, interpreting and reporting results, and disseminating and encouraging the use of findings.

Table F-1 shows how these objectives map onto the curriculum.

Table F-1. Curriculum-Objectives-Assessment Matrix

Objective	Course Requirements	Method of Assessment
Evaluation history & philosophy	Evaluation Models & Approaches [new course]	P, OP, G, C
Ethical standards	Evaluation Models & Approaches [new course]	P, C
Evaluation practice	Program Planning, Assessment & Evaluation (HDFS 637)	P, FE, D
Research design	Research design requirement (EDUC 607; UAPP 800; or HDFS 815)	H, Q
Qualitative data	Qualitative methods requirement (EDUC 850)	P, O, FE
Quantitative data	Measurement requirement (EDUC865; HDFS 655) Quantitative methods requirement (EDUC 665; UAPP 815)	Q, L, H
Program or method specialization	Electives in program or methods area	P, G, D
Intellectual humility	Competency analysis	P, Pf
Evaluation implementation	All courses	Pf

Assessment Key: P = Paper, Q = Exam/Test Ques., O = Oral Presentation, Pf = Portfolio, H=Home Work, G=Group Work, FE = field experience, Discussion = D. All objectives require Attendance and Class Participation.

[1] The American Evaluation Association (www.eval.org) defines evaluation as the assessment “of the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness.”