The Texas A&M University System delivers a common set/embraces a common view of important outcomes and is accountable for sustained measurement.
For all TAMU System universities, the rationale for assessing student learning outcomes (SLOs) originates primarily from efforts to maintain institutional effectiveness, which is defined as a process of identifying outcomes, assessing the extent to which they are achieved, and providing evidence of improvement based on their analysis.
Upon completion of their degree program, students will be able to synthesize knowledge from general and specialized studies.
All criteria met and results exceed expectations with little room for improvement.
Most criteria met and results indicate mastery of objective with some room for improvement.
Acceptable number of criteria met and results meet expectations with room for improvement.
Some criteria met and results indicate need for improvement.
Few criteria met; results indicate need for significant improvement or no/insufficient results reported to measure performance of objective.
TAMU (incl. TAMUG and TAMUQ)
Student Experience in the Research University (SERU) Survey: Every 2 - 3 years, undergraduates are asked to complete a comprehensive survey focused on their experiences at a research university. Two questions are included on this survey related to students' ability to integrate and synthesize knowledge from across their coursework and other learning experiences. Specifically, respondents are asked to indicate how often they (1) incorporated ideas or concepts from different courses in course assignments; and, (2) brought up ideas or concepts from different courses during class discussions (where 1 = never to 6 = very often). In the spring of 2017, 6,369 students responded to the survey (representing a response rate of apx. 11%), of whom 3601 where juniors and seniors (representing a response rate of apx 12%). The responses from juniors and seniors specifically are reported here.
Proficient: The average response among juniors and seniors to the first item related to incorporating concepts from different courses in course assignments was 4.75 (SD = 1.19); the average response on the second item related more specifically to class discussion was 3.42 (SD = 1.40). These findings would suggest that students are able to synthesize and integrate knowledge gained across general and discipline-specific coursework as demonstrated by (self-reported) classroom discussions and assignments. Approximately 25% of the respondents reported having brought up ideas or concepts from different courses in class discussion either often or very often. In addition, approximately 65% indicated having incorporated concepts from different courses in to course assignments either often or very often. Interestingly, the mean responses from students who had participated in high impact practices (e.g., first year seminars, capstone experiences, study abroad) were consistently higher on these two survey items than for students who had not participated in such experiences.
When comparing responses on the two items from students completing the survey in 2014 v. 2017, the mean responses show no differences between the two years (2014: ave response of 4.78, SD = 1.11 for the first item and ave response of 3.43, SD = 1.39 for the second item).
Given the apparent difference in students' perception of their ability to integrate/synthesize knowledge gained across coursework based on whether or not they had participated in high impact learning experiences, programs are continuing to broaden the range of high impact learning experiences available for their students and identifying strategies to encourage participation in such activities to strengthen students' ability to synthesize knowledge from general and specialized studies.
TAMU (incl. TAMUG and TAMUQ)
Senior Survey: Each year, graduating seniors are asked to complete a Senior Survey immediately prior to their graduation. Although an indirect measure of learning, findings related to students' self-assessment of their ability to integrate knowledge gained through their studies and other learning experiences are nonetheless valuable. The responses from the May graduates who completed the survey (n = 3906; representing a 56% response rate) on items related to integrative learning are reported here. These items asked students to indicate the extent to which TAMU contributed to (1) their ability to integrate knowledge from outside their field of study; and, (2) their ability to synthesize knowledge across courses and other experiences in their discipline (where 1 = none to 5 = very much).
Proficient: The average response for the first question was 3.89 (SD = .967) and 4.23 (SD = .77) for the second. Interestingly, students who had participated in high impact learning experiences (e.g., first year seminars, capstone experiences, study abroad) consistently had higher means on these two items than students who had not participated in such. The increased attention in recent years to the benefits of participating in high impact learning experiences (as described previously) may well have contributed to the recognition of the university's contribution to the development of this particular learning outcome.
When comparing responses on the first item from May 2014 graduates with May 2017 graduates, there are no differences with respect to students' perceptions of TAMU's contribution to their ability to integrate knowledge (note: the second item was not included on the Senior Survey in 2014). In both instances, the average response was 3.89 (SD = .94 in 2014 and SD = .97 in 2017).
The emerging benefits of participating in high impact practices in strengthening learning outcomes such as the ability to synthesize knowledge gained across coursework, both inside and outside their chosen profession, reinforce the university's commitment to ensuring students have access to a wide array of high impact learning experiences designed to enrich student learning. Experiences such as study abroad, capstones, internships/field experiences, undergraduate research, and co-curricular experiences such as being a student leader (as presented above) provide students direct opportunities to engage in integrative learning.
TAMU (incl. TAMUG and TAMUQ)
Quality Enhancement Plan (QEP): Aggies Commit to Learning for a Lifetime: As a part of the TAMU 5-year QEP focused on the value of high impact learning experiences (HILEs) on student learning, students having completed either a capstone experience or an internship/field experience (both considered to be HILEs) submitted artifacts (e.g., e-portfolios, field experience reflections) for review. The artifacts were scored using a similar rubric to that described above for Integrative Learning, thus providing a direct measure of student learning. A total of 143 artifacts were evaluated for integrative learning specifically (note: over 1000 artifacts were scored across 4 identified learning outcomes in an effort to assess directly the impact of participation in HILEs on student learning; the results reported here pertain only to those scores related specifically to integrative learning).
Sufficient: The artifacts submitted by students having completed a field experience reached the Benchmark level (based on an average score of 2.57 on an 8-point scale where 0-2 = Failed to Benchmark, 2 - 4 = Benchmark, 6 - 8 = the highest level of proficiency or Capstone). Students achieving the Benchmark level demonstrated the ability to identify connections between inside and outside class experiences and to use skills/knowledge gained in one experience to address a problem or challenge faced in another situation. Each of the final projects submitted following a capstone experience reached the Capstone level with respect to integrative learning; these students demonstrated a more sophisticated ability to synthesize and draw conclusions across a range of experiences and disciplines to help guide problem solving in a unique situation.
The projects and reflections used to assess integrative learning among undergraduates suggest not only that students are able to make connections across experiences and use these connections to address problems or situations encountered, but also that participating in high impact learning experiences contributes to the this ability to synthesize knowledge gained across the curriculum. It should be noted that the scoring of student artifacts as a direct measure of integrative learning has been challenging over the years given the range, depth, and applicability of the artifacts submitted as well as the availability of appropriate artifacts across programs to be scored. Additional assessment strategies and rubrics are under develop in hopes of improving the quality and number of artifacts submitted on a routine basis for assessment purposes.
Beyond actions to strengthen assessment strategies, programs across the university are committed to facilitating student participation in high impact learning experiences as a means through which to strengthen students' ability to synthesize knowledge and address real world problems.
TAMU (incl. TAMUG and TAMUQ)
Student Leadership Assessment: The Division of Student Affairs asked student leaders from across the university to reflect on their experiences. Specifically, leaders were asked to submit responses to the following: "Describe what you have learned from your leadership experience and how it changed you. How would you apply this in your major/classes and career goals?" 307 student leaders submitted essays that were scored based on the VALUE rubric for Integrative Learning (focused on students' abilities to connect relevant experiences and academic knowledge, to make connections across disciplines, and to adapt and apply skills, abilities, theories, or methods gained in one situation to a new situation).
Sufficient: The average score across the reflections was 1.85 on a 5-point scale. Of those, 50% scored at the Milestone 2 level (demonstrating their ability to articulate strengths and/or challenges within specific events to increase effectiveness across contexts), with an additional 16% scored at the Milestone 3 level (demonstrating their ability to recognize their own learning over time and complex contextual factors), and 2% at the Capstone level (demonstrating their ability to solve problems in new and original ways).
Overall, students were able to articulate aspects of what they had learned from their leadership role with many demonstrating an ability to connect and apply these lessons to future endeavors. When comparing the current results from those reported in 2014 (also based on reflections among student leaders), a noteworthy improvement in the ability to reflect integrative learning has occurred (2014 ave response of 1.33 v. 2017 ave response of 1.85). Since 2013-14, more training opportunities have been provided to student leaders focused on the value of reflection for purposes of making stronger connections between curricular and co-curricular experiences in preparation for their future.
Advisors of student organizations are being encouraged to meet with the leaders specifically (and the membership of the student organizations at large) on a routine basis to challenge students to make connections across learning experiences and environments in preparation for their future. Advisors are also being encouraged to train the student leaders to implement reflection opportunities within the student organizations, thus strengthening the culture of reflection and integration.
TAMU (incl. TAMUG and TAMUQ)
Quality Enhancement Plan (QEP): Aggies Commit to Learning for a Lifetime – Freshmen and Senior Reflections from High Impact Learning Experiences (HILES) evaluated with Integrative and Lifelong Learning (modified) VALUE Rubric.
Sufficient: Seniors: Connections – 2.2/4, Transfer – 2.3/4, Reflection and Self Assessment – 2.5/4. Freshmen: Connections – 1.8/4, Transfer – 2/4, Reflection and Self Assessment – 2.2/4.
For each “Integration” category of the Aggies Commit rubric, Seniors scored higher than freshmen when reflecting on significant learning experiences. These results include participation from each college as well as branch campuses in Galveston and Qatar. Reflections were also analyzed in co-curricular experiences, comparing student leaders to general students (1.33/5 vs. 1.15/5).
College/campus-specific reports were sent to each college and branch campus. Program improvements are the responsibility of each college and/or department. Moving forward, Texas A&M will use this data as baseline information for future planning of its QEP for 2018.
Content-analysis of reflections also revealed approximately 1/2 of all reflections discussed integrative learning gains in the co-curricular (prompt was not specific).
TAMU (incl. TAMUG and TAMUQ)
Student workers in the Division of Student Affairs were given a pre-survey (September) and post-survey (April) about their learning on the job and its relationship to their academic work. Some students also were interviewed by their supervisors during the year.
Proficient to exemplary: the pre-mean to post-mean for each question increased .o8 to .24 (on a 5-point scale), depending on the question. The post means ranged from 3.39 to 4.32 on a 5-point scale.
Students were asked about their job in relation to several outcomes: working with a variety of people, conflict resolution, written communication, time management, oral communication, and critical thinking. Students were also asked about their recognition of connections between their job and their academic work, career options, and full-time employment. Qualitatively, the post-survey students were better able to articulate how they have applied their coursework to their job and vice versa. Students articulated what skills they learned that make them successful at Texas A&M.
For some students, particularly in more technical fields, making the connections were more difficult, especially as seen in the pre-survey. Students who had the opportunity to reflect with their supervisors were helped to make connections as well. In 2014-2015, the Division will provide more training to supervisors about helping students make connections.