Assessing Priority Competencies for Evaluation Capacity Building in Extension
Keywords
Evaluation Capacity Building
Extension
Performance
Program Planning
Evaluation
Impact
How to Cite
Abstract
Program planning and evaluation activities play a critical role in the quality of Extension programs. The literature consistently identifies program planning and evaluation as core Extension competencies. With an increased focus on accountability in Extension, there is a need to ensure Extension professionals are equipped with the most relevant competencies to show program impact. Guided by a competency model, this study sought to assess priority competencies for inclusion in an Evaluation Capacity Building (ECB) program at Utah State University (USU) Extension. Data were gathered from a sample of Extension professionals at USU Extension (n = 87) and analyzed using descriptive statistics, principal component analysis, paired t-tests, and Cohen’s d. While results indicated a need for professional development in most competency items of program planning and evaluation, the principal component analysis revealed two professional development themes for program planning, and three for program evaluation. Extracted professional development themes were Program Ideas and Schedules, Program Design, Evaluation Plans, Evaluation Data and Results, and Monitoring and Impact. This study recommends the development of professional development themes to organize and prioritize efforts for ECB in Extension. It also highlighted a need to continuously assess professional capacity for accountability in Extension.
References
Agbaje, K. A. A., Martin, R. A., & Williams, D. L. (2001). Impact of sustainable agriculture on secondary school agricultural education teachers and programs in the north central region. Journal of Agricultural Education, 42(2), 38–45. https://doi.org/10.5032/jae.2001.02038
Ary, D., Jacobs, L. C., Sorenson, C., & Walker, D. A. (2014). Introduction to research in education (9th ed.). Wadsworth.
Boone, E. J., Safrit, R. D., & Jones, J. (2002). Developing programs in adult education: A conceptual programming model (2nd ed.). Waveland Press.
Boyle. P. G. (1981). Planning better programs. McGraw-Hill.
Brown, A. S., Gibson, J. D., & Stewart, D. L. (2008). An assessment of Virginia Cooperative Extension’s New Extension Agent Training program. Journal of Extension, 46(4), Article 4FEA7. http://www.joe.org/joe/2008august/a7.php
Cooper, A. W., & Graham, D. L. (2001). Competencies needed to be successful county agents and county supervisors. Journal of Extension, 39(1), Article 1IRB3. https://www.joe.org/joe/2001february/rb3.php
Dewey, J. D., Montrosse, B. E., Schröter, D. E., Sullins, C. D., & Mattox, J. R. (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29(3), 268–287. https://doi.org/10.1177/1098214008321152
Diaz, J., Gusto, C., & Diehl, D. (2017). Developing a program plan based on the program planning cycle. University of Florida/IFAS Extension. http://edis.ifas.ufl.edu/wc290
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail and mixed-mode surveys: The tailored design method (4th ed.). John Wiley & Sons.
Dyer, J. E., & Osborne, E. W. (1999). The influence of science applications in agriculture courses on attitudes of Illinois guidance counselors at model student-teaching centers. Journal of Agricultural Education, 40(4), 57–66. https://doi.org/10.5032/jae.1999.04057
Field, A. (2013). Discovering statistics using IBM SPSS statistics. Sage.
Forest, L. B., McKenna, C., & Donovan, J. (1986). Program development and evaluation. In University of Wisconsin-Madison Cooperative Extension (Ed.), Connections: A” how-to” handbook for developing Extension long-range plans and major programs. University of Wisconsin-Madison.
Fox, J. E., & LaChenaye, J. M. (2015). Cultural core competencies: Perceptions of 4-H youth development professionals. Journal of Human Sciences and Extension, 3(3), 65–78. https://www.jhseonline.com/article/view/745/646
Franklin, E. A. (2011). Greenhouse facility management experts’ identification of competencies and teaching methods to support secondary agricultural education instructors: A modified Delphi study. Journal of Agricultural Education, 52(4), 150–161. https://doi.org/10.5032/jae.2011.04150
Franz, N. (2011). Advancing the public value movement: Sustaining Extension during tough times. Journal of Extension, 49(2), Article v49-2comm2. http://www.joe.org/joe/2011april/comm2.php
Franz, N., Arnold, M., & Baughman, S. (2014). The role of evaluation in determining the public value of Extension. Journal of Extension, 52(4), Article v52-4comm3. https://www.joe.org/joe/2014august/comm3.php
Franz, N., & Archibald, T. (2018). Four approaches to building Extension program evaluation capacity. Journal of Extension, 56(4), Article v56-4tt5. https://joe.org/joe/2018august/tt5.php
Frechtling, J. (2010). The 2010 user-friendly handbook for project evaluation. Directorate for Education and Human Resources. https://www.purdue.edu/research/docs/pdf/2010NSFuser-friendlyhandbookforprojectevaluation.pdf
Grinnell, R. M., Gabor, P. A., & Unrau, Y. A. (2019). Program evaluation for social workers (8th ed.). Oxford.
Harder, A. (2015). Priority competencies needed by UF/IFAS Extension county faculty. University of Florida/IFAS Extension. https://edis.ifas.ufl.edu/wc236
Harder, A., & Narine, L. K. (2019). Interpersonal leadership competencies of extension agents in Florida. Journal of Agricultural Education, 60(1), 224–233. https://doi.org/10.5032/jae.2019.01224
Harder, A., Place, N. T., & Scheer, S. D. (2010). Towards a competency-based Extension education curriculum: A Delphi Study. Journal of Agricultural Education, 51(3), 44–52. https://doi.org/10.5032/jae2010.03044
Henning, J., Buchholz, D., Steele, D., & Ramaswamy, S. (2014). Milestones and the future for Cooperative Extension. Journal of Extension, 52(6), Article v52-6comm1. https://www.joe.org/joe/2014december/comm1.php
Hetherington, C., Eschbach, C., & Cuthbertson, C. (2019). How evaluation capacity building grows credible and actionable evidence for Cooperative Extension programs [Special issue]. Journal of Human Sciences and Extension, 7(2), 175–188. https://www.jhseonline.com/article/view/825/718
Karpen, S. C. (2019). The social psychology of biased self-assessment. American Journal of Pharmaceutical Education, 82(5), Article 6299. https://doi.org/10.5688/ajpe6299
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler.
Lakai, D., Jayaratne, K. S. U., Moore, G. E., & Kistler, M. J. (2012). Barriers and effective educational strategies to develop Extension agents’ professional competencies. Journal of Extension, 50(4), Article v50-4rb1. https://joe.org/joe/2012august/rb1.php
Lakai, D., Jayaratne, K. S. U., Moore, G. E., & Kistler, M. J. (2014). Identification of current proficiency level of Extension competencies and the competencies needed for Extension agents to be successful in the 21st century. Journal of Human Sciences and Extension, 2(1), 71–89. https://www.jhseonline.com/article/view/584/513
Lamm, A. J., & Israel, G. D. (2013). A national examination of Extension professionals’ use of evaluation: Does intended use improve effort? Journal of Human Sciences and Extension, 1(1), 49–62. https://www.jhseonline.com/article/view/657/568
Lamm, A. J., & Lamm, K. W. (2018). Considering an appreciative approach to international Extension evaluation. Journal of International Agricultural and Extension Education, 25 [Special issue], 74–82. https://doi.org/10.5191/jiaee.2018.25406
Liles, R. T., Mustian, R. D. (2004). Core competencies: A systems approach for training and organizational development in Extension. Journal of Agricultural Education and Extension, 10(2), 77–82. https://doi.org/10.1080/13892240485300121
Lindner, J. R. (2001). Competency assessment and human resource management performance of county Extension chairs in Ohio. Journal of Agricultural Education, 42(4), 21–31. https://doi.org/10.5032/jae.2001.04021
Lindner, J. R., Dooley, K. E., & Wingenbach, G. J. (2010). A cross-national study of agricultural and Extension education competencies. Journal of International Agricultural and Extension Education, 10(1), 51–59. https://doi.org/10.5191/jiaee.2003.10107
Maddy, D. J., Niemann, K., Lindquist, J., & Bateman, K. (2002). Core competencies for the Cooperative Extension System. https://apps.msuextension.org/jobs/forms/Core_Competencies.pdf
McClelland, D. (1973). Testing for competence rather than “intelligence.” American Psychologist, 28(1), 1–14. https://doi.org/10.1037/h0034092
McClure, M., Fuhrman., & Morgan, A. (2012). Program evaluation competencies of Extension professionals: Implications for continuing professional development. Journal of Agricultural Education, 53(4), 85–97. https://doi.org/10.5032/jae.2012
McCormick, R. W. (1959). An analysis of training needs of county Extension agents in Ohio. [Unpublished doctoral dissertation]. University of Wisconsin, Madison.
Mertens, D. M., & Wilson, A. T. (2019). Program evaluation theory and practice (2nd ed.). Guilford.
Narine, L. K., & Harder, A. (2019a). Beyond Borich: Testing a contemporary method to assess professional development needs [Oral presentation]. Proceedings of the National Association of Extension Program and Staff Development Professionals, 37–39. https://naepsdp.org/resources/Documents/NAEPSDP%20Conference%20Proceedings%202019.pdf
Narine, L. K., Harder, A., & Zelaya, P. (2019b). A qualitative analysis of the challenges and threats facing Cooperative Extension at the county level. Journal of Southern Agricultural Education Research, 69. http://www.jsaer.org/pdf/Vol69/2019_008%20final%20print.pdf
Preskill, H., & Catsambas, T. T. (2006). Reframing evaluation through appreciative inquiry. Sage.
Rockwell, K., & Bennett, C. (2004). Targeting outcomes of programs: A hierarchy for targeting outcomes and evaluating their achievement. Faculty publications: Agricultural Leadership, Education & Communication Department. http://digitalcommons.unl.edu/aglecfacpub/48/
Rosenman, R., Tennekoon, V., & Hill, L. G. (2011). Measuring bias in self-reported data. International Journal of Behavioural & Healthcare Research, 2(4), 320–332. https://doi.org/10.1504/IJBHR.2011.043414
Rossi, P., Lipsey, M., & Freeman, H. (2004). Evaluation: A systematic approach (7th ed.). Sage.
Scales, J., Terry, R., Torres, R. M. (2009). Are teachers ready to integrate science concepts into secondary agriculture programs? Journal of Agricultural Education, 50(2), 100–111. https://doi.org/10.5032/jae.2009.02100
Scheer, S. D., Cochran, G. R., Harder, A., & Place, N. T. (2011). Competency modeling in Extension education: Integrating an academic Extension education model with an Extension human resource management model. Journal of Agricultural Education, 52(3), 64–74. https://doi.org/10.5032/jae.2011.03064
Scheer, S. D., Ferrari, T. M., Earnest, G. W., & Connors, J. J. (2006). Preparing Extension professionals: The Ohio State University’s model of Extension education. Journal of Extension, 44(4), Article 4FEA1. https://www.joe.org/joe/2006august/a1.php
Schwarz, M. H., & Gibson, J. (2010). A needs assessment of aquaculture Extension agents, specialists, and program administrators in Extension programming. Journal of Extension, 48(2), Article v48-2a6. http://www.joe.org/joe/2010april/a6.php
Suvedi, M., & Kaplowitz, M. (2016). What every extension worker should know: Core competency handbook. US Agency for International Development. https://agrilinks.org/sites/default/files/resource/files/MEAS%20(2016)%20Extension%20Handbook%20%20Suvedi%20Kaplowitz%20-%202016_02_15.pdf
Taylor-Powell, E., & Boyd, H. (2008). Evaluation capacity building in complex organizations. In M. T. Braverman, M. Engle, M. E. Arnold, & R. A. Rennekamp (Eds.), Program evaluation in a complex organizational system: Lessons from Cooperative Extension (pp. 55–69). New Directions in Evaluation, 120. Jossey-Bass.
Trede, L. D., & Whitaker, B. S. (2000). Educational needs and perceptions of Iowa beginning farmers toward their education. Journal of Agricultural Education,41(1), 39–48. https://doi.org/10.5032/jae.2000.01039
Warner, R. M. (2012). Applied statistics: From bivariate through multivariate techniques (2nd ed.). Sage.
Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessments: A practical guide. Sage.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.