Institutional Effectiveness and Accreditation

Institutional Effectiveness

Structure | Enhancement Cycle | Committee | Committee Responsibilities |
Effectiveness Collaboratives | Process Phases/Timeline | Resources | IE Home

Resources

The Office of Institutional Effectiveness and Accreditation, with the Collegewide Institutional Effectiveness Committee, has developed an Institutional Effectiveness Manual for Academic Programs and Non-Academic Units/Educational Support Services Units. This includes the College’s Institutional Effectiveness Model, the six steps of program student learning outcomes assessment with examples, the review of assessment reports, process phases and timeline, glossary and other resources. These publicly posted manuals do not include the appendices. The full version of these same manuals are posted in the WEAVEonline Collegewide Document Repository with the complete appendices.

Assessment Sampling for Academic/Educational Programs

The designated Effectiveness Process Facilitators are responsible for selecting the appropriate sample for their program. The list of these facilitators can be found by selecting links embedded in either of these Web pages:

We recommend the following guidelines to assist Effectiveness Process Facilitators in their Institutional Effectiveness assessment activities.

Program Size

  • For degree and certificate programs, program assessment generally focuses on data collection of student artifacts from ‘end-of-program’ students who are at or near graduation, such as those who have completed 75 percent or more of the program or are enrolled in a specific end-of-program course like the Capstone/Internship.
  • Programs with more than 30 ‘end-of-program’ students should collect and analyze assessment data from a random representative sample of ‘end-of-program’ students.
  • Since General Education is part of the A.A., which includes more than 30 students who are near the end of their program, sampling should include distance education course sections/students.

Sampling

  • As a general guideline a 30-50 percent random sample of the total is suggested (note—using a columnar list of all cases (reference numbers, students or their artifacts for a specific assessment measure), Microsoft Excel can be used to generate a random sample*). As a general rule, more cases are better than less, especially in instances where there are less than 300 total cases. If the total group size is 30 or less, then all cases should be included.
  • Sampling should include students/course sections taught by full-time and part-time faculty; all locations (ie. North, South, NAS Jax, XX Company offices, dual enrollment, distance education, etc.)
  • Sampling should include students/course sections taught via distance education even if less than 50 percent of the program is offered via distance education.

* Please note that Associate of Arts/Liberal Arts and Sciences, and General Education Assessment samples are constructed by the Office of Student Analytics and Research. All other academic programs select their own program assessment samples. If assistance is needed, please consult the Office of Student Analytics and Research.

Analysis and Reporting of the Findings

  • If 50 percent or more of the program is taught via distance education, data should be discussed overall (entire sample) and also disaggregated by delivery method. (The results for XXX outcome across all students in the sample are ... The results for XXX outcome for students sampled from distance learning are ... The results for XXX outcome for students sampled from face-to-face class sections are ...)
  • If 50 percent or more of the program is taught at more than one location, data should be discussed overall (entire sample) and also disaggregated by location. (The results for XXX outcome across all students in the sample are ... The results for XXX outcome for students sampled from XX Campus are ... The results for XXX outcome for students sampled from NAS Jax are ...)

Definition of Distance Education for Purposes of Assessment

  • A combination of the new state definitions for hybrid (HB) and distance education (DL) courses. The state now defines Hybrid as 50 to 79 percent of the seat time for a course is not “face-to-face.” The state now defines Distance Education courses as courses in which 80 percent or more of the seat time is not “face-to-face.” Since SACS’s definition of distance education is at the 50 percent mark, we are combining the state definitions of hybrid and distance education for assessment purposes.

Resources

  • The designated Effectiveness Process Facilitators should contact E-Dean Sheri Litt (Open Campus), if they plan to sample distance education courses encoded by Open Campus, in order to develop a collaborative approach that will involve Open Campus in planning for the assessments it will be responsible for administering.
  • The designated Effectiveness Process Facilitators should contact Steve Kruszewski (Student Analytics and Research), if they have questions about sampling in general.

*Please see the attached brief Web demonstration to provide tips on selecting a random sample of cases. If you have questions about this demo, please contact the Office of Student Analytics and Research.

Assessment Plan Sampling for Non-Academic Units and Educational Support Services (student success units)

When designing your assessment measures, please collect the data in a manageable and practical manner. The most important tip is to collect data that will provide evidence that is relevant to the specific outcome.

Please also include a representative sample that includes all campus locations/centers, off-site locations, and distance learning, as appropriate.

If possible, collect the requested data from every student, employee, community member, or client for a given time period (ie. If you are conducting a survey of student and employee satisfaction with the cafeteria, consider providing the survey to every student or employee who uses the cafeteria in a given period of time. If you are conducting a workshop, consider providing the workshop evaluation form to each person who participates in the workshop.)

If it is not practical to collect data from every individual who uses your department’s services, consider these questions:

  • What is the population or group of individuals who can provide the information that you need to adequately assess the measure?
  • What information do you expect these individuals to provide?
  • Is there a minimum number of responses that you need to give you adequate evidence?
  • What resources do you have to administer the survey, quiz, or other assessment measure?  Consider the cost and time involved. 
    • Will you be printing and mailing forms or surveys?  Will you be sending an email with a web link to a survey or other form at no cost?
  • What time period would provide you the most useful information?
  • Can you combine your survey or tool with another department who needs to survey or contact the same group of individuals?

If you determine that you need a more ‘statistically sound or correct’ sample size for your assessment measure, please contact the Office of Student Analytics and Research.