Accountability Methodology

An accountability report organizes aggregate statistics for undergraduate involvement in a certain activity type. An Activity models a person’s participation in some sort of trackable activity. The primary purpose of the Activity models is to compile and generate accountability statistics for the provost and the legislature. We currently focus on two metrics for undergraduates: public service and research.

How activities are stored

Activity is subclassed by a number of Activity varieties, as depicted in the table below.

Subclass Types of activities
ActivityCourse For tracking an entire UW course for a particular quarter
ActivityProject For tracking an ongoing project, such as Students in Service, that lasts several weeks or even several quarters
ActivityEvent For tracking a one-time event, such as the MLK Day of Service, that has a more limited scope.

Sources

The following data sources are used to calculate accountability statistics:

  • Course lists provided by departments, including course_branch, course_no, dept_abbrev, and section_id for each course along with a Y/N for each quarter that the course should be counted. (Note: if section_id was not specified, we automatically pull all sections for that course. In some cases this will result in duplicates but they will be removed during duplication processing).
  • Individual student lists provided by departments
  • Carlson Center service-learning participation (as tracked by EXPO) — only for the public service report
  • Applicants and group members of applications whose Offering has an ActivityType defined (currently, this includes Mary Gates Research scholarship [research], Mary Gates leadership scholarship [public service], and the Undergraduate Research Symposium [research]).
    • For session-based events (e.g., UGR Symposium), session presenters (all group members) are counted.
    • For non-research-based award offerings (e.g., Mary Gates Leadership), only awarded applicants are counted.
    • For research-based award offerings (e.g., Mary Gates Research), all complete applications (mentor approved) are counted.

Intended Results

This system generates the following general types of statistics wherever possible:

Student-quarters
The number of quarters a student spent working in the given activity. One student for one quarter=1 student-quarter. One student for three quarters=3 student-quarters. Three students for one quarter=3 student-quarters.
Number of students
The total number of unique students who participated
Number of hours
A sum of all total number of hours that students participated
Coverage
The number/percentage of departments reporting information

Accountability Methodology

This section describes the methodology used to calculate accountability statistics. All statistics are generated in groups based on the ActivityType of the given activity.

  1. Raw Data Accumulation: For each data source, pull relevant records. Calculate number of students and hours for each record.
    1. Student participation is gathered for department course lists:
      1. Each ActivityCourse that meets the requested criteria (e.g., “year = 2009”) is validated against the SDB to verify that it is a valid course.
      2. For each course, the class enrollment is pulled, resulting in individual Student records.
      3. Students who withdrew, failed, or did not receive credit for the course are removed.
    2. Student participation is gathered from department individual participant lists.
    3. Student participation is gathered from EXPO’s service-learning courses (ONLY for public service report).
    4. Student participation is gathered from EXPO’s application system (ONLY matching ActivityTypes)

    Final result: hash with Students as keys and an activity array for each quarter, e.g.:

           Student_1_system_key   =>  {
             AutumnQuarter => [ActivityCourse, MaryGatesLeadershipApplication],
             SpringQuarter => [ServiceLearningCourse],
                            },
           Student_2_system_key   =>  {
             WinterQuarter => [ActivityEvent]
           }
           Student_3_system_key   => {
             :no_quarter => [ActivityProject]   # in rare cases, we don't know what quarters a project spanned
           }
           ...
  2. Removal of non-undergraduates: For each student in the data hash, check each quarter of activity to ensure that the student was enrolled as an undergraduate during that quarter. Non-undergraduate quarters are removed. To do this, we look at the student’s transcript record for that quarter.
    • If the quarter being evaluated is summer, the record is never removed, even if the student was not enrolled or not listed as an undergraduate for that quarter.
    • If there is no transcript record for that quarter and it is not summer, we remove it from the data hash.
    • If the “class” code for that quarter is higher than 5 and it is not summer, we remove it from the data hash.

    Notes:

    • For this report, “undergraduate” is any student with a class standing of 1, 2, 3, 4 or 5 (note that we do not include non-matriculated students but we do include “5th years”).
    • If removing a quarter from a student in the data hash results in that student having no more valid quarters, the student is completely removed from the data hash and will not count toward any totals.
    • Any :no_quarter listings are still analyzed based on the first quarter of the academic year being evaluated.

    Final result: same as in step 1, but only with the students’ quarters that are valid.

  3. Duplication Processing: For each student in the data hash, process each duplication rule (see rule definitions below). As the processing happens, records that are removed are saved in a duplicates array for later review.
    1. Same quarter, same course
    2. Same quarter, multiple courses
    3. Same quarter, multiple activities

    Final result: same as in step 2, but only with the activities that should be counted in the final tally.

  4. Statistics Calculation
    1. Student-quarters: sum of the count of each student’s Quarter objects, plus any :no_quarter listings.
    2. Number of students: size of student data array.
    3. Number of hours: sum of number of hours per student; calculated differently based on the data source.
      1. Course list: number of credits the student earned for the course, multiplied by the constant COURSE_CREDIT_TO_HOURS_MULTIPLIER and the WEEKS_PER_QUARTER (10).
      2. Individual list: Hours are specified by department (as either hours per week or total hours per quarter)
      3. Carlson Center service-learning: 20 hours per quarter is used, as recommended by the Carlson Center (this is the minimum requirement by the CC)
      4. Application processes: use hours_per_week as specified by the student, if available. If hours_per_week is not available, we calculate the average number of hours for all students in the given activity type.
        • For award-based processes, multiply the calculated-hours-per-quarter times the number of requested award quarters
        • For event-based processes, the number of hours is only counted for the quarter during which the event occurred.
  5. Additional Reports
    • Department Sponsorship Breakdown: in what departments are these activities happening?
    • Department Cross-Pollination: e.g., how many BIO majors are doing research sponsored by medicine?
    • Demographic Analysis: majors, gender, class standing, and race/ethnicity of students participating.
    • Coverage: how many departments are represented/reported, broken down by college, discipline, and academic/admin.

Rules for duplicates

These rules define how we handle students who have multiple records in a given quarter. Note that activities are always evaluated through the lens of a specific ActivityType, so these rules only apply to duplicates within the same ActivityType. For example, if a student participates in two research projects and one public service course in the same quarter, the public service course is not factored into the duplication processing of the two research projects.

Any time duplicates are removed, a Duplicates Report or Discrepancy Report is generated to allow for manual inspection and override.

  • Same quarter, same course: If a student is enrolled in the same course for the same course_no (e.g., they are enrolled in both a lecture section and a quiz section for the same course), only the section with the larger number of credits are counted.
  • Same quarter, multiple courses: Students receiving credit for multiple courses in the same quarter are not de-duped, unless the faculty mentor/instructor is listed as the same for both course.
  • Same quarter, multiple activities: Students who receive credit for a course and are also listed under an ActivityProject or ActivityEvent for the same quarter will only count the larger of the number of hours.

Comments are closed.