What Can Behavioral Science Do For Your Virtual Skills-Based Program?

The perception that online is inferior to in-person likely stems from poorly developed virtual pedagogy, antipathy born during COVID-19 lockdowns, or a general lack of familiarity with digital methods. The effectiveness of traditional schools depends on the strong social presence of teachers and students, dynamic interactions, physical activity, and natural engagement. Technology ostensibly lags in facilitating the same social presence. After all, a person in real life is more than just a video of their face. Physical appearance, gestures, authority, charisma, timbre, a brief look of derision, or a funny expression are all subtleties that could explain why students become captivated by a teacher. Recognizing this strength in physical teaching is important, but it cannot be the sole determinant as to whether online education is effective or not. Instead, the CATALYST program by Winchester College leverages three techniques that are fundamental for both in-person and online learning: rich peer interactions, well-designed activities, and engaging and applied tasks. 

Peer-to-Peer Interactions

Good online teaching often shifts responsibility away from the classroom and instead onto the systems in place, addressing the common lament of poor learning experiences due to unengaging instructors. Rather than targeting teachers, addressing student interactions is an effective strategy for creating a successful online learning experience. Within skills-based learning, peer-to-peer interactions are the linchpin for developing communication, negotiation, and problem-solving abilities. In line with findings of how students’ arguments become more comparative and reasoned following organized conversations among peers, we observed at CATALYST that structured peer work (discussions, group tasks, challenges, academic puzzles) is a great training ground for building the above-mentioned skills and learning how to collaborate empathetically.

To be truly stimulating and engaging online, the chosen online learning platform must extend beyond basic video calling and breakout rooms – such as by incorporating elements like virtual movement and proximity chat for organic interactions. In online settings, student-centric learning is vital with a higher need for facilitating student learning rather than delivering knowledge. The online educator’s role, while still important for explaining key concepts, better serves learners when it emphasizes facilitating interaction. In such environments, student agency often deepens their inherent curiosity for knowledge..

Gamification Goals

Gamification is the application of game-like elements and principles in non-game contexts to engage and motivate people. It can be instrumental in creating educational moments for skill expression. The iterative nature of skill-building necessitates dynamic and stimulating environments to keep learners engaged (much like sports, learning an instrument, or immersive language courses abroad). More ludic and explorative learning activities are common in skills-based or online learning. Beyond the conventional use of PBL (points, badges, leaderboards), simulations of real-world scenarios, such as Model United Nations, mock trials, or role-playing, provide engaging learning experiences. These activities, coupled with competition and artificial conflict, are highly effective for skill development.

One of CATALYST’s favorites is the “unlock” mechanic, where content is gated behind task completion, ensuring equal learning opportunities and combating the passivity that can arise in online settings. This approach demands content that is stimulating and encourages active participation. In this way, well-executed online programs can be particularly effective for skills development, offering advantages like ease of meeting, resource access, and feedback. 

Gamified learning should require minimal facilitator involvement, so that learning becomes student-centric and experiential, although success is dependent on student skill. Playful activities are central to online learning, fostering engagement and social presence. 

The Pros of Project-based Learning

Intrinsic motivation in learning (learning because it is inherently satisfying and enjoyable, rather than being driven by external rewards or pressures) is often tied to a sense of relatedness, agency, or individual responsibility. While the causal relationship between intrinsic motivation and increased learning outcomes has been debated, it is widely understood to contribute to students' development of important exploratory skills that are crucial to the learning process. Project-based learning, particularly with real-world implications, can be highly effective. For instance, CATALYST’s Change Project encourages students to find solutions to tangible issues such as pollution, wastefulness, or poverty. Identifying a local issue offers practical problem-solving experience and more of a direct impact. Implementing solutions provides valuable hands-on experience. 

Students have to learn how to evaluate the consequences of their plans, points of failure, as well as possible counterarguments to and feasibility of their solutions. It is therefore good to guide students in identifying a genuine problem in a local community, something relatable to their own experience. For instance, tackling global warming is too ambitious to gain practical experience in problem-solving, but applying a solution to reduce food waste in a local school provides a perspective for actionable ideas, access to stakeholders, and small-scale but observable impact. Putting a solution into action suddenly makes an idea real.

Traditional means of educational assessment have often prioritized evaluating knowledge while completely abandoning skill development.² Therefore, the new emerging frameworks for assessing skills are founded on different constructions of learning by centering skills as the purpose of education. This has the significant benefit of ensuring that skills-based programs and their students’ learning are assessed more accurately. 

There is a growing understanding of skills that youth should learn (e.g. collaboration, communication, and critical thinking); however, existing skill assessment tools often assume in-person instruction. This has important implications for how activities around these conceptualizations of skills are designed and the opportunities to assess students across the skill categories both within a classroom setting and a real-world context. Although in-person instruction provides opportunities for educators to have greater access to their students, advanced virtual platforms enhance our ability to collect data in unique and less interruptive ways.

TDL’s partnership with CATALYST by Winchester College aimed to rethink digital skill assessment tools. The CATALYST program offers both synchronous and asynchronous virtual learning to youth globally, over a 1-2 week period via Gather (a web-conferencing software). Gather provides a virtual room with kinaesthetic features that allow for movement via an avatar around multiple custom-made classrooms and hallways as well as proximity chat (hearing others more clearly when their avatar is nearby). The CATALYST program focuses on project-based learning and offers its students university-level topics taught in a way that aims to develop their skills across six competency areas: 1) negotiation and resolution, 2) narrative defense, 3) effective communication, 4) quantitative fluency, 5) empathetic collaboration, and 6) critical analysis and problem-solving. 

The goal of the partnership was to account for the short and dense nature of the CATALYST curriculum and design a student assessment toolkit that is an easy lift, protecting the bandwidth of CATALYST facilitators and students alike. For this reason, it needed to comprise innovative approaches to the delivery, the data inputs that are assessed, and the individuals involved in the assessment process to arrive at a triangulated story of student progress that leverages the various online platforms that CATALYST uses.

Together, TDL and the CATALYST team developed an assessment tool that is based on the drivers and barriers of student engagement and learning as assessed via existing literature, CATALYST staff, and CATALYST alumni. The assessment process was designed to span the entire program timeline, incorporating a final check-in six months post-program to evaluate practical application of learnings.

Employing diffused delivery, students responded to brief questions throughout their day, promoting continuous engagement. The approach empowered students through self-assessment and reflection, fostering ownership of their learning. Timely feedback, facilitated by this method during the two-week program, provided opportunities for improvement while the facilitators were still present. The assessment relied on a multifaceted rubric, incorporating self-assessments, observations of project work, and submitted outputs of that work. Leveraging existing platforms, including AI for rubric translation, enhanced the efficiency and effectiveness of the assessment process. 

These methods effectively capture the research reality and can inform future improvements in teaching methods. Removing barriers to engagement with assessments and framing them in a motivating manner resulted in nearly 90% completion rates for even optional self-assessments. The use of triangulated assessments not only enhanced student engagement but also instilled confidence in facilitators regarding student transcripts, underscoring the significance of behaviorally informed initiatives for the entire learning community.

So how can you ensure that your applications of technology in your virtual classroom are evidence-based? How can you leverage behavioral science to enhance your online delivery? As you navigate the ever-evolving landscape of digital education, we leave you with these few words to better harness the potential of skill-building online:

• Lead with peer-to-peer learning: take time to understand your target learners and their realities, then unlock collaborative insights in a way that is most relevant to them to deepen their subject understanding.
• Take time to craft evolved and well-researched project-based learning experiences that push your students to bring their physical experiences into the digital classroom. After all, hands-on experiences ignite creativity and the practical application of acquired skills.
• Advocate for triangulated formative assessments: providing a holistic view of learners' progress is always important and particularly so with skill-building, where the learning objective is very applied.
• Seize unique opportunities that technology creates: we are living in a time where technology affords experiences that in-person learning cannot (e.g. accessibility for differently abled learners) so be sure to take full advantage!