Faculty Research and Assessment of Scholarship

Traditional definitions of faculty work frame contemporary views of its assessment. Faculty work generally falls into three categories: (1) research, which is the discovery or creation of knowledge through systematic inquiry; (2) teaching, which is the transmission of knowledge through class instruction and other learning-focused activities; and (3) service, meaning service to others through application of one's special field of knowledge. Assessment protocols have considered, to a varied extent, scholarly activities performed in each of these areas. Faculty assessment is conducted for purposes of reappointment, promotion, the awarding of tenure, and professional development.

During the last decades of the twentieth century, a societal focus on the work of college and university faculty as a measure of return on the public's investment in higher education stimulated a reevaluation of how faculty performance ought to be measured and assessed. The work of Charles Glassick and his colleagues (1997) has significantly influenced scholarship focused on developing workable evaluation systems. Their work is rooted in the broadened definitions of scholarship proposed by Ernest Boyer in 1990, and on how these definitions impact assessment systems. As scholarship is being conceived more broadly, traditional assessment systems are being reexamined to include all work conducted by faculty.

The development of workable assessment systems is difficult largely due to the fact that the value of assessment is often controversial. The controversy stems from several factors, according to Larry Braskamp and John Ory's 1994 study. First of all, faculty are not used to fully describing or judging their work. Secondly, systems that have been developed are often criticized by faculty as inadequate, particularly as assessment applies to teaching and service work. Lastly, a preoccupation with how to assess has resulted in a lack of attention to what is to be assessed. Successful assessment systems are based on clearly specified performance assessment criteria and standards for faculty performance in relevant areas of work. Each of these will be covered in turn.

Performance Assessment Criteria

While faculty are typically assessed based on their performance in their teaching, research, and service roles, the weight assigned to each of these areas varies depending on the type of institution and its associated expectations for faculty involvements in each area. For example, the assessment process for tenure-track faculty in research-focused institutions may place relatively more emphasis on the performance of the research role. An exception is for faculty whose appointments are not tenure-eligible, such as adjunct faculty or those in full-time appointments with a specified term. In these cases, assessment is typically weighted according to the specified terms of each individual's contract, or according to expectations for performance as determined by institutional policy or tradition. For faculty in teachingoriented liberal arts colleges or community colleges, evaluation processes may focus more on teaching.

Assessment criteria also vary by academic discipline to accommodate the sometimes disparate intellectual products. The intellectual work of faculty in management and education, as applied disciplines, may be best evidenced by corporate or institutional testimonial of improved organizational effectiveness as a result of faculty involvement. The intellectual products of the so-called pure disciplines, such as history, are more appropriately characterized by written articles, reviews, and monographs.

Performance Standards and Quality

Standards for performance quality and assessment protocols are to a large extent determined by the governing bodies of each institution. Weighting of performance in teaching, research, and service is variable as well. Largely due to the disproportionate value placed on the research role of faculty, promotion and tenure guidelines have been clearer in articulating standards for research performance. Relative to assessment of teaching and service, the standards for judging research performance are clear and systematically linked to guidelines for promotion and the awarding of tenure. Performance is typically measured in terms of productivity, relying largely on the use of quantitative measures such as the number of publications or other creative works produced over a specified period of time.

Standards for teaching performance on the other hand, particularly in research-focused institutions, are less clear. The inherent difficulty in judging teaching performance lies in the fact that effectiveness is best measured by how much students learn. Because evidence of learning is often subjective or qualitative, it is more complicated to measure. This leads to questions of what measures are most appropriate for purposes of faculty evaluation, particularly when faculty compensation, if not entire careers, may rest on reliable assessment processes. This dilemma continues to fuel debates around the best way to assess teaching performance and contributes to faculty mistrust of evaluation systems. In many cases, efforts to develop workable measurement systems have stalled.

Evaluation of service remains illusive. Consequently, when it comes to tenure and promotion, service is often forgotten. This has led, unwittingly, to a devaluing of service work, and "virtually no institution has yet figured out how to quantify such work" (Glassick et al., p. 20). Reliable assessment of the service work of faculty must therefore rely on qualitative judgments rooted in institutionally developed standards for performance. Assessing service work across the professoriate is seen as a formidable challenge, yet, unlike the challenge of teaching evaluation, it has yet to benefit from a significant body of research and experimentation.

As the notion of what constitutes legitimate scholarship is broadened, and as increased attention is being paid to the outcomes of faculty teaching, academe is pressed to redefine measures of quality. Institutions, and the profession as a whole, are facing new questions, such as: How is student learning, as evidence of quality teaching, best documented? What processes associated with the transmission of knowledge are to be considered legitimate evidence of faculty performance? What constitutes appropriate evidence of application of a disciplinary knowledge base into service activities? How is integration of one's area of expertise into a multidisciplinary intellectual context best evidenced?

In 1997 Glassick and colleagues offered a solid framework for institutions considering modifications of faculty assessment systems. This work has received considerable attention by scholars of higher education and, as of the beginning of the twenty-first century, it has presented the most well-defined approach to assessing faculty work. In this book, the authors examined methods for judging scholarly performance by colleges and universities, granting agencies, journal editors, and university presses. Applying Boyer's four domains of scholarship (discovery, teaching, application, and integration) to define faculty work, they identified six standards commonly associated with high-quality scholarly work: clear goals, adequate preparation, appropriate methods, significant results, effective presentation, and reflective critique. These criteria for determining quality are noteworthy in that they offer not only a framework for evaluation, but also aid in understanding the kinds of intellectual activities and outputs that are legitimate components of the work that faculty perform.

Assessment Tools

There are numerous tools used for collecting evidence of faculty performance quality. Braskamp and Ory cite the following assessment tools: evaluations by students and peers, as well as self-evaluations, of teaching; the evaluative conference; evaluative letters from colleagues and experts in the field; and portfolios that explicate professional accomplishments. Evaluators may vary depending on institution type. For faculty in research-oriented institutions, considerable weight is placed on the judgments of peers, including scholars from other institutions. In many liberal arts colleges and community colleges, where the faculty role is more oriented toward teaching, there may be greater reliance on the judgments of local faculty who are most familiar with institutional teaching and learning philosophies and standards for teaching performance.

Teaching performance has historically been accomplished by student evaluations at the end of each course, with the weight afforded such evaluations varying from institution to institution and, often, from department to department. In liberal arts colleges and other teaching-focused institutions, a combination of student evaluations and peer review is often used. Peer review involves a review of one's pedagogical skills by his or her faculty colleagues. Toward the end of the twentieth century, peer review of teaching became more popular in research-focused institutions, with a goal of providing a similar level of support, consultation, and evaluation typically provided for research. Some institutions have begun to enhance rewards for teaching performance, largely in response to societal concerns about the quality of undergraduate education. Such rewards have been in the form of cash awards, special stipends for teaching improvement, and endowed chairs devoted to teaching excellence.

While research performance is commonly measured in terms of the number of published articles, books, and other research products, there is a qualitative aspect to such measures. The type of publication, and whether or not selection for publication involved peer review, also determines research quality. Publications subjected to a peer-review process are typically considered to be of higher quality in that they are thought to have undergone a more rigorous critique than published work that is not reviewed.

The Portfolio as an Assessment Tool

As definitions of faculty work are broadened, ideas about what constitutes adequate evidence of such work must be broadened as well. The notion of the professional portfolio has received wide attention as a comprehensive way to represent the work of a faculty member. The professional portfolio is a collection of artifacts and materials gathered and presented by each faculty member. It represents the requirements for one's work, as defined by expectations specified at the time of appointment or at the start of the review period. Building on Peter Seldin's notion of the teaching portfolio, Robert Froh and his associates in 1993 specified a set of principles to guide the development of portfolios and their evaluation. While the concept of the portfolio does incorporate elements of the traditional dossier compiled by faculty to document their performance, the portfolio takes a broader view of scholarship by integrating the values of the faculty member with those of the departmental and institutional community. This might be accomplished through a reflective essay and work samples that uniquely represent such an integration of values. Thus, the more standardized representation of work that is characteristic of the traditional dossier must be transformed to reflect the work of each individual and the unique contributions he or she has made in relevant areas of scholarship.

Assessment Challenges

Assessment of faculty work has become an important consideration for both faculty and administrators, particularly as the pressure for measuring outcomes of faculty work over and above the easily quantifiable numbers of publications has escalated. Policymakers are calling for increased accountability of faculty members' time, and for evidence that the time spent produces acceptable outcomes, specifically with respect to undergraduate education. State governing bodies and Congress have become more involved with academic issues associated with student access, equality, and research integrity, all of which influence how faculty conduct their work and how institutions monitor and assess it. Such external pressures, combined with increasing trends that emphasize the importance of faculty professional development have led to increased attention on faculty assessment systems. Reevaluation of these systems is not without its challenges, however.

First, definitions of scholarship vary from discipline to discipline, and these differences have significant implications for how assessment systems are structured and implemented. Also, the importance of communication of discipline-specific performance factors across disciplines must be addressed if faculty on promotion and tenure review committees are to effectively and reliably judge the work of their colleagues. Second, effective evaluation systems depend on the communication of standards upon which judgments of quality will be based and acceptable mechanisms for documenting faculty work. Third, any changes to traditional assessment systems must maintain the historic autonomy of faculty as sole judges of quality scholarship. As members of a profession, faculty reserve the right to be the sole judges of the quality of the work performance of those claiming membership among their ranks.

Judgments of quality are based on standards set by the profession as a whole, the specific mission of the institution within which a faculty member's work is conducted, and standards circumscribed by the discipline and the institution. As definitions of scholarship are broadened, indicators of quality must be well considered in order to preserve the status of the profession and simultaneously satisfy new calls for accountability. Lastly, assessment systems must be flexible enough to consider the unique work profiles and needs of a growing segment of the professoriate–the full-time, non-tenure-track faculty. This is the environment within which effective evaluation systems must be developed so as to serve the changing roles of faculty in higher education.