Stakeholders’ Feedback on a Personalized, Online Platform for STEM Teacher Learning

Framework

The development of this vision and plan is grounded in a theory of practice-based teacher education. This theory purports that teachers’ learning is prompted by opportunities for them to engage in, unpack, and reflect on the instructional practices that comprise the work of teaching (Ball & Forzani 2009; Grossman, Compton, et al., 2009; Grossman, Hammerness, & McDonald, 2009; McDonald et al., 2014; Zeichner, 2012). Practice-based teacher education – both within university courses and in PD settings - emphasizes preparing teachers to enact specific instructional practices that occur as part of the day-to-day activities in which teachers and students engage (Borko et al., 2011; Fishman et al., 2017; Forzani, 2014; Pella, 2015). High-quality practices, such as facilitating discussions that engage students in argumentation, are key in teacher education curricula as they can create a milieu with rich and multiple interactions with social and cognitive experiences that support student thinking and learning (Martin & Dismuke, 2018). While there are varied pedagogies of practice used within practice-based teacher education, one pedagogy – approximations of practice – has been used to provide these practice teaching opportunities. As noted by Grossman, Compton, et al., (2009), approximations are “opportunities for novices to engage in practices that are more or less proximal to the practices of a profession” (p. 2058). In teacher education, approximations typically involve: (a) face-to-face rehearsals where one or more teachers act as K-12 students during a practice teaching interaction (Benedict-Chambers et al., 2020; Ghousseini, 2017; Lampert et al., 2013) or (b) online simulated teaching experiences where the teacher interacts with one or more digitally animated student avatars who are operated on the back end by a person called a simulation specialist (Author, 2020; Dotger et al., 2015). Overall, providing opportunities for teachers to engage in approximations of practice can support teachers in learning how to enact high-leverage or ‘core’ teaching practices in classrooms (e.g., Ball & Forzani, 2009; Franke et al., 2006; Kazemi et al., 2007; Kloser, 2014; Sleep et al., 2007).

Methods

Prior research has found that collaboratively studying co-design could help identify design principles and processes that can support the learning of other research-practice teams seeking to plan for and develop tools to address student and community priorities.

The study uses a mixed-method methodology including the collection and analysis of quantitative survey data and qualitative data from surveys, focus groups, and document reviews (Kyza & Agesilaou, 2022; Plano Clark & Ivankova, 2016). Reviews of documents include reviews and analysis of meeting notes and artifacts on partnership development, such as the guiding document developed at the outset of the co-design process. All partner team members will participate in reflective discussions around the strengths and areas for improvement of the partnership development process. Reflections will also focus on the extent to which the vision and plan for future professional development was successful in incorporating potential ways to address the initiatives, priorities, and concerns of the partner district teachers and guiding frameworks. A survey will also be used to gather individual feedback from team members on the partnership development process and on the professional development vision and plan developed through the partnership.

Feedback on the vision and plan will also be gathered via focus groups, co-facilitated by educators and researchers, with relevant external stakeholder groups representing a range of school types and student needs. Potential stakeholder participants include the STEM teaching professional developers from the global STEM ecosystem community of practice supporting teachers across the U.S. Focus group questions will focus on the usability of the vision and plan for the digital, performance-based PD platform. All data collected will be content analyzed using thematic coding approaches, and all partnership team members will be invited to participate in analyses. We will use this stakeholder feedback to refine and improve the vision and plan to enable development and testing of the platform.

Results

As noted above, one of the goals of this project is to document and share lessons learned from the partnership development process. Prior research has found that collaboratively studying co-design could help identify design principles and processes that can support the learning of other research-practice teams seeking to plan for and develop tools to address student and community priorities. Our partnership team researchers and educators will collaborate on the plan for studying the partnership development process; if educators without research experience are interested in co-conducting the participatory research to document lessons learned, they will be provided opportunities for knowledge and skill building in research. Thus, it is expected that the study results will include lessons learned on co-design between educators and researchers, as well as a highly feasible, educator-informed plan for personalized, online professional learning for in-service teachers.

Importance

This vision and plan focus on performance tasks and automated feedback for reimaging elementary mathematics and science teachers’ PD learning experiences for several reasons. First, interactive performance tasks enable teachers to gain practice and build skills related to teaching in specific disciplines. The use of performance tasks directly addresses an ongoing and critical need for teacher education to go beyond only developing teachers’ knowledge and beliefs, but orienting teachers’ learning around developing their instructional practice (Author, 2021; Author, 2022a; Benedict et al., 2016; Chung, 2008; Hauser & Kavanagh, 2019). Prior research provides strong support for the importance of building teachers’ abilities to engage in core instructional practices to impact student learning outcomes (Matsumoto-Royo & Ramirez-Montoya, 2021; Paolini, 2015; Zuo et al., 2023) in an online format (Chandran et al., 2021), which highlights a clear research need for understanding how performance tasks can be used to re-envision the professional learning opportunities currently available to support elementary mathematics and science teacher learning.

Second, we need to consider the feasibility and scalability of these performance-based teacher learning approaches from the start, and automated feedback is critical to addressing the challenges of how best to scale digital performance tasks so that they are an accessible and cost-effective tool to support personalized, on-demand teacher learning. This is especially critical for teachers in under-resourced school districts and those who teach in settings where their opportunities to engage in and receive feedback on their instructional practice is limited. To date, providing formative feedback at scale, especially in combination with performance tasks, has been challenging to achieve. However, the use of natural language processing and machine learning to solve perennial challenges in education has been accelerating at a rapid pace during the last few years (Author, 2023a; Author, 2023c; Demszky et al., 2023; Jacobs et al., 2022; Jensen et al., 2020; Suresh et al., 2021). Thus, the decision to focus on developing a vision and plan for automated feedback when deploying interactive, performance tasks to support teacher learning is important, strategic, and timely, as it allows us to ensure that the solutions we co-design and study will be able to be used at scale to support teacher learning worldwide.

This work is important because of its potential long-term impacts on student math and science learning, an area in need of improvement if our nation is to achieve goals related to getting more U.S. students, especially those from historically underrepresented backgrounds, into the STEM careers pipeline (National Science Board, 2021). It is also important because teacher self-efficacy has been linked to teacher retention (Boyd et al. 2005; Hanushek et al., 2004), thus improvements in elementary teachers’ self-efficacy in math and science teaching may have ripple effects on their satisfaction with teaching and their likelihood of remaining in the teaching field. This is critical at a time of widespread teacher shortages, especially in harder to fill subjects such as mathematics and science (Podolsky, 2016).

References

Ade‐Ojo, G. O., Markowski, M., Essex, R., Stiell, M., & Jameson, J. (2022). A systematic scoping review and textual narrative synthesis of physical and mixed‐reality simulation in pre‐service teacher training. Journal of Computer Assisted Learning, 38(3), 861-874. 

Author. (2021).

Author. (2022a).

Author. (2022b).

Author. (2023a).

Author. (2023b).

Author. (2023c).

Ball, D. L., & Forzani, F. M. (2009). The work of teaching and the challenge for teacher education. Journal of Teacher Education, 60(5), 497-511. https://doi.org/10.1177/0022487109348479

Baxter, J. A., Ruzicka, A., Beghetto, R. A., & Livelybrooks, D. (2014). Professional development strategically connecting mathematics and science: The impact on teachers' confidence and practice. School Science and Mathematics, 114(3), 102-113.  

Benedict, A., Holdheide, L., Brownell, M., & Foley, A. M. (2016). Learning to Teach: Practice-Based Preparation in Teacher Education. Special Issues Brief. Center on Great Teachers and Leaders. 

Benedict-Chambers, A., Fick, S. J., & Arias, A. M. (2020). Preservice teachers' noticing of instances for revision during rehearsals: A comparison across three university contexts. Journal of Science Teacher Education, 31(4), 435-459.

Bondie, R., Mancenido, Z., & Dede, C. (2021). Interaction principles for digital puppeteering to promote teacher learning. Journal of Research on Technology in Education, 53(1), 107-123. 

Bondurant, L., & Amidon, J. (2021). Virtual field experiences as an opportunity to develop preservice teachers’ efficacy and equitable teaching practice. Online Learning in Mathematics Education, 317-334. 

Borko, H., Koellner, K., Jacobs, J., & Seago, N. (2011). Using video representations of teaching in practice-based professional development programs. Zdm, 43(1), 175-187. https://doi.org/10.1007/s11858-010-0302-5

Boyd, D., Lankford, H., Loeb, S., & Wyckoff, J. (2005). Explaining the short careers of high-achieving teachers in schools with low-performing students. American Economic Review, 95(2), 166-171. http://www.aeaweb.org/articles.php?doi=10.1257/000282805774669628 

Chandran, K. B., Haynie, K. C., Tawbush, R., & Wyss, J. M. (2021). Effectively adapting and implementing in-person teacher professional development to a virtual format. Journal of STEM outreach, 4(3), 10.15695/jstem/v4i3.12. https://doi.org/10.15695/jstem/v4i3.12 

Chung, R. R. (2008). Beyond assessment: Performance assessments in teacher education. Teacher Education Quarterly, 35(1), 7-28. 

Davis, E. A., & Haverly, C. (2022). Well-started beginners: Preparing elementary teachers for rigorous, consequential, just, and equitable science teaching. In Handbook of Research on Science Teacher Education (pp. 83-96). Routledge.  

Demszky, D., Liu, J., Hill, H. C., Jurafsky, D., & Piech, C. (2023). Can automated feedback improve teachers’ uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course. Educational Evaluation and Policy Analysis, 01623737231169270. 

Dieker, L. A., Straub, C., Hynes, M., Hughes, C. E., Bukathy, C., Bousfield, T., & Mrstik, S. (2019). Using virtual rehearsal in a simulator to impact the performance of science teachers. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 11(4), 1-20. 

Dotger, B., Masingila, J., Bearkland, M., & Dotger, S. (2015). Exploring iconic interpretation and mathematics teacher development through clinical simulations. Journal of Mathematics Teacher Education, 18(6), 577-601. https://doi.org/10.1007/s10857-014-9290-7

Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., ... & McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform. Teaching and Teacher Education, 72, 107-121. 

Fishman, E. J., Borko, H., Osborne, J., Gomez, F., Rafanelli, S., Reigh, E., Tseng, A., Million, S., & Berson, E. (2017). A practice-based professional development program to support scientific argumentation from evidence in the elementary classroom. Journal of Science Teacher Education, 28(3), 222-249. https://doi.org/10.1080/1046560X.2017.1302727

Forzani, F. M. (2014). Understanding “core practices” and “practice-based” teacher education: Learning from the past. Journal of Teacher Education, 65(4), 357-368. https://doi.org/10.1177/0022487114533800 

Francis, A. T., Olson, M., Weinberg, P. J., & Sterns-Pfeiffer, A. (2018). Not just for novices: The programmatic impact of practice-based teacher education. Action in Teacher Education, 40(2), 119-132. https://doi.org/10.1080/01626620.2018.1424053 

Franke, M., Grossman, P., Hatch, T., Richert, A. E., & Schultz, K. (2006, April). Using Representations of Practice in Teacher Education. In annual meeting of the American Educational Research Association, San Francisco, CA.

Ghousseini, H. (2017). Rehearsals of teaching and opportunities to learn mathematical knowledge for teaching. Cognition and Instruction, 35(3), 188–211. https://doi.org/10.1080/07370008.2017.1323903

Goodson, B., Caswell, L., Dynarski, M., Price, C., Litwok, D., Crowe, E., Meyer, R., & Rice, A. (2019). Teacher preparation experiences and early teaching effectiveness: Executive summary (NCEE 2019-4010). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. 

Grossman, P. (Ed.). (2021). Teaching core practices in teacher education. Harvard Education Press. 

Grossman, P., Compton, C., Igra, D., Ronfeldt, M., Shahan, E., & Williamson, P. W. (2009). Teaching practice: A cross-professional perspective. Teachers College Record, 111(9), 2055-2100. https://doi.org/10.1177/016146810911100905

Grossman, P., Hammerness, K., & McDonald, M. (2009). Redefining teaching, re-imagining teacher education. Teachers and Teaching: Theory and Practice, 15(2), 273-289. https://doi.org/10.1080/13540600902875340

Hammerness, K., McDonald, S., Matsko, K. K., & Stroupe, D. (2020). How do teachers learn to teach science in ambitious and equitable ways? In D. Stroupe, K. Hammerness, & S. McDonald (Eds)., Preparing science teachers through practice-based teacher education (pp. 13-28). Harvard Education Press. 

Hanushek, E., Kain, J. F., & Rivkin, S. G. (2004) Why public schools lose teachers. Journal of Human Resources 39(2), 326-354. https://hanushek.stanford.edu/publications/why-public-schools-lose-teachers  

Hauser, M., & Kavanagh, S. S. (2019). Practice-based teacher education. In Oxford research encyclopedia of education. 

Jacobs, J., Scornavacco, K., Harty, C., Suresh, A., Lai, V., & Sumner, T. (2022). Promoting rich discussions in mathematics classrooms: Using personalized, automated feedback to support reflection and instructional change. Teaching and Teacher Education, 112, 103631. 

Jensen, E., Dale, M., Donnelly, P. J., Stone, C., Kelly, S., Godley, A., & D'Mello, S. K. (2020, April). Toward automated feedback on teacher discourse to enhance teacher learning. In Proceedings of the 2020 Chi Conference on Human Factors in Computing Systems (pp. 1-13). 

Kang, H., & Zinger, D. (2019). What do core practices offer in preparing novice science teachers for equitable instruction? Science Education, 103(4), 823-853. 

Kaufman, D., & Ireland, A. (2016). Enhancing teacher education with simulations. TechTrends, 60, 260-267. 

Kazemi, E., Lampert, M., & Ghousseini, H. (2007). Conceptualizing and using routines of practice in mathematics teaching to advance professional education. Report to the Spencer Foundation, Chicago.

Kloser, M. (2014). Identifying a core set of science teaching practices: A delphi expert panel approach. Journal of Research in Science Teaching, 51(9), 1185-1218. https://doi.org/10.1002/tea.21171

Lampert M., Franke M. L., Kazemi E., Ghousseini H., Turrou A. C., Beasley H., Cunard, A., & Crowe K. (2013). Keeping it complex: Using rehearsals to support novice teacher learning of ambitious teaching. Journal of Teacher Education, 64(3), 226-243. https://doi.org/10.1177/0022487112473837

Lee, C. W., Lee, T. D., Castles, R., Dickerson, D., Fales, H., & Wilson, C. M. (2018). Implementation of immersive classroom simulation activities in a mathematics methods course and a life and environmental science course. Journal of Interdisciplinary Teacher Leadership, 2(1). 

Martin, S. D., & Dismuke, S. (2018). Investigating differences in teacher practices through a complexity theory lens: The influence of teacher education. Journal of Teacher Education, 69(1), 22-39. https://doi.org/10.1177/0022487117702573

Matsumoto-Royo, K., & Ramírez-Montoya, M. S. (2021). Core practices in practice-based teacher education: A systematic literature review of its teaching and assessment process. Studies in Educational Evaluation, 70, 101047. 

McDonald, M., Kazemi, E., & Kavanagh, S. (20134). Core practices and pedagogies of teacher education: A call for a common language and collective activity. Journal of Teacher Education, 64(5), 378-386. https://doi.org/10.1177/0022487113493807

National Science Board. (2021). Elementary and Secondary STEM Education. Science and Engineering Indicators 2022. NSB-2021-1. Alexandria, VA. Available at https://ncses.nsf.gov/pubs/nsb20211/. 

Paolini, A. (2015). Enhancing teaching effectiveness and student learning Outcomes. Journal of Effective Teaching, 15(1), 20-33. 

Pella, S. (2015). Pedagogical reasoning and action: Affordances of practice-based teacher professional development. Teacher Education Quarterly, 42(3), 81-101. https://www.jstor.org/stable/teaceducquar.42.3.81

Phelps, G., Wylie, C.E., Omodolapo, S. (2023, April). Using short, situated performance tasks and peer feedback to support preservice teacher learning. American Association of Educational Research Annual Conference, Chicago, Il.  

Podolsky, A., Kini, T., Bishop, J., & Darling-Hammond, L. (2016). Solving the teacher shortage: How to attract and retain excellent educators. Learning Policy Institute. 

Powell, C. G., & Bodur, Y. (2019). Teachers’ perceptions of an online professional development experience: Implications for a design and implementation framework. Teaching and Teacher Education, 77, 19-30. 

Saderholm, J., Ronau, R. N., Rakes, C. R., Bush, S. B., & Mohr-Schroeder, M. (2017). The critical role of a well-articulated, coherent design in professional development: an evaluation of a state-wide two-week program for mathematics and science teachers. Professional Development in Education, 43(5), 789-818. doi:10.1080/19415257.2016.1251485 

Sleep, L., Boerst, T., & Ball, D. (2007, March). Learning to do the work of teaching in a practice-based methods course. In Presentation made at the research pre session of the annual meeting of the National Council of Teachers of Mathematics, Atlanta, GA.

Suresh, A., Sumner, T., Jacobs, J., Foland, B., & Ward, W. (2019, July). Automating analysis and feedback to improve mathematics teachers’ classroom discourse. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, No. 01, pp. 9721-9728). 

Whitcomb, J., Borko, H., & Liston, D. (2009). Growing talent: Promising professional development models and practices. Journal of Teacher Education, 60(3), 207-212. 

Zeichner, K. (2010). Rethinking the connections between campus courses and field experiences in college-and university-based teacher education. Journal of Teacher Education, 61(1-2), 89-99. https://doi.org/10.1177/0022487109347671

Zuo, G., Sy D., & Kaufman, J. H. (2023). How Do Teachers Spend Professional Learning Time, and Does It Connect to Classroom Practice? Findings from the 2022 American Instructional Resources Survey. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/research_reports/RRA134-18.html.