Outcomes assessment—building from the ground up

Print Friendly

Summary

Our campus project has an established pilot or moderate stage of using ePortfolio to support Outcomes Assessment. Portfolios across a range of disciplines have been structured to align with student learning outcomes, and departments are reporting ePortfolio data within the program review processes. Some departments are also referencing portfolios as a way to reflect on the curriculum and competency requirements.

Authors: Ruth Cox and Alycia Shada

eportfolioposter_print_full

Setting the Stage

SF State has 22 departments using ePortfolios for a range of purposes. In conjunction with Academic Technology, portfolios have been clearly structured to align/map to core requirements/outcomes in the discipline. Assessment protocols for specific outcomes vary and are being conducted at the local/program/departmental level.

The office of Academic Planning and Assessment is responsible for coordinating the overall outcomes assessment at SF State, accredited by WASC.

As a 4-year public institution, outcome assessment is conducted at the department level through program review processes. SF State just concluded the Educational Effectiveness Review (Fall 2012). A number of departments included ePortfolio data in their program reviews, (Journalism, Metro Academies, Masters in Public Health, Liberal Studies, Special Education). In addition to regular departments, 28 SF State programs are accredited by special accrediting agencies. Some of the ePortfolio-using departments (e.g. Public Health, Nursing, Public Administration) fall into this category.

Most departments have piloted or applied a range of outcomes assessment protocols (including the VALUE rubrics) to a selection of student ePortfolios. A number of departments have included ePortfolio data in their program review reports to the Office of Academic Planning and Assessment.

There has not been a formal planning process for a campus-wide ePortfolio assessment system at SF State. In the fall of 2012, the General Education Taskforce identified ePortfolios as the preferred pathway for tracking Gen Ed outcomes (to begin with a pilot in Liberal Studies). The groundwork for this work has not yet begun, but will represent the first direct connection between campus-wide GE outcomes and portfolio use.

SF State’s office of Academic Planning defines assessment as “a continuing process aimed at understanding and improving institutional, program level and classroom level achievement involving: Making expectations explicit and public; Setting appropriate criteria and high expectations for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; Using the resulting information to document, explain, and improve performance. Assessment is a dynamic process.”

Developmental Story

Building a “culture of learning” is happening in a myriad of ways at the program level and those using portfolios have reported (to the Acad. Tech team) that they are employing assignments that deepen inquiry, critical thinking, reflection and integration. In program review cycles some departments are referencing portfolios as a way to reflect on the curriculum and competency requirements.

Specifically, within the Metro Academies Faculty Learning Community, there were several facilitated meetings over the past 2 years focused the development of ePortfolio structure, assignments and integrative learning exercises.

This presentation presents an overview of Academic Technology’s millennial learner-centered ePortfolio rationale and assessment strategies. We offer two case stories we hope will illustrate early attempts at working to improve retention, persistence, graduation rates and moreover an understanding (from students and faculty) of the importance of outcomes.

Metro Academies, (8 minutes into recording) and Liberal Studies (outcomes mapped and example of course/dept. process @ 17 minutes in recording) both illustrate projects we’ve been supporting:

ePortfolio evaluation SF State, November 2011 

echo.pres.screenshot

Persistence rates: Metro students persist at a 19% higher rate into their 7th semester (senior year) compared to all first-time freshmen at SF State, when we take an average of all Metro cohorts. A 5% increase has been considered significant in the past, so we are thrilled with this news.

Graduation rates: Metro students’ four-year graduation rate is almost ten percentage points higher than their peers. Their five-year rate is expected to be almost 30 percentage points higher (five-year graduation rates are projected for spring 2014 based on current course completion data).

We will be looking at the C2L Metro survey data more closely to find indicators of ePortfolios as a high-impact contributing factor to this new data.

Conceptual Framework

Catalyst and Connector

ePortfolio in Metro has created an opportunity for disciplines to connect to one another and to connect to GE. With a cohort of students taking linked courses each semester (one discipline-specific and one GE), we have had an environment rich with integration and collaboration. The ePortfolio has helped immensely with assessment of learning outcomes such as integrative learning, critical thinking, and writing. The ePortfolio is helping create opportunities for assessment of oral communication and quantitative reasoning, but at this point, we are only in the discussion stages of assessing these learning outcomes as part of the ePortfolio and Metro.

Metro has a faculty workgroup that meets 2-3 times a year as part of the Connect to Learning grant. This workgroup took the several pages of detailed student learning outcomes written by Metro’s leadership team, and refined them down to five easy-to-use and concise learning outcomes. This team also adapted VALUE rubrics to fit Metro’s learning outcomes and reflected on classroom efforts to achieve these learning outcomes.

Evidence

At the institution level, Academic Technology is supporting the individual assessment efforts of various programs. The Metro Academies program, with the support of the Connect to Learning grant, has worked for over two years to develop and pilot student learning outcomes assessment. We have discussed these efforts with the larger faculty learning community, smaller faculty subgroups, and in one-on-one conversations with faculty. With their feedback, we are revising and implementing assessment protocols this upcoming year for our expanding program. We are in the midst of consulting with three separate outside experts to assist in this process, each of whom has a specific, but related area of expertise (e.g., program evaluation, assessment of student learning outcomes, education evaluation).

Conclusion

We hope to gain further support from our campus leadership through inclusion in national partnerships such as Connect to Learning. We believe that having strong examples of ePortfolio from different disciplines to share locally with our President, Provost, Deans, and Chairs will help further our mission of integrative learning, reflection, and assessment.

As an AAC & U VALUE leadership campus, we see great potential in working more deeply with faculty to apply specific VALUE rubrics, and develop meaningful evaluation processes. We are encouraged by the three-year timeframe in the Connect to Learning model, as we believe that we can now track observable stages in the growth of successful ePortfolio initiatives, and are prepared to document these using the Metro Academies project.

Our ePortfolio project is thriving at program levels, but we have not yet reached an institution-wide policy that would support ePortfolios for all students. A challenge that Metro faces is that its students do not necessarily continue to use their ePortfolios in their junior and senior years, after they have completed the Metro program.

 

Speak Your Mind

*

Skip to toolbar