Friday SLO Talks

In this episode of Friday SLO Talks, a team from the University of California, Berkeley Center for Teaching and Learning discusses how rubrics can be used to clarify expectations, support student learning, and improve the consistency of assessment in higher education classrooms.The presenters begin by explaining that rubrics are often misunderstood as simple grading tools. In reality, well-designed rubrics can serve a much broader instructional purpose. When used thoughtfully, rubrics communicate what quality work looks like, help students understand performance expectations, and guide instructors in providing more consistent and transparent feedback.The Berkeley team describes how rubrics function as a bridge between learning outcomes, assignments, and evaluation. By clearly defining the criteria for performance and describing levels of achievement, instructors make expectations visible to students. This transparency can help students better prepare their work and understand how their performance will be evaluated.A key theme of the presentation is that rubrics are most effective when they are integrated into the learning process rather than used only at the end of an assignment. The presenters encourage instructors to share rubrics with students early, discuss the criteria in class, and use them as tools for reflection, peer review, and revision. In this way, rubrics can support formative feedback and help students develop stronger work over time.The discussion also addresses common challenges faculty encounter when creating rubrics. Designing clear criteria and meaningful performance levels requires careful thought about what instructors truly value in student work. The presenters emphasize that effective rubrics focus on observable aspects of performance rather than vague qualities such as “good understanding” or “effort.”Another important issue raised in the talk is consistency in evaluation. When multiple instructors or teaching assistants assess student work, rubrics can help align expectations and reduce variability in grading. Calibration conversations among instructors can further improve reliability and ensure that evaluators interpret rubric criteria in similar ways.The presenters also highlight the importance of flexibility. Rubrics should not be seen as rigid scoring instruments but as evolving tools that instructors refine over time. By reviewing how rubrics function in practice and gathering feedback from students and colleagues, instructors can continually improve how they define and evaluate learning.Throughout the conversation, the Berkeley team emphasizes that rubrics ultimately support a larger goal: helping students understand what successful performance looks like and how they can improve their work. When used effectively, rubrics promote clearer communication between instructors and students and strengthen the connection between assignments and course learning outcomes.Although the session focuses on practices developed at UC Berkeley, the ideas discussed apply broadly across disciplines and institutions. The presentation offers practical insights for instructors, assessment coordinators, and educational leaders seeking to design assessment approaches that are transparent, meaningful, and supportive of student learning.

In this episode of Friday SLO Talk, we dive deep into the complexities of evaluating student performance in real-world clinical settings. Guests John Moore and Phil Reeves from the National Board of Medical Examiners (NBME) join us to share insights from an extensive research project involving five medical schools and over two million lines of assessment data.The Challenge of StandardizationThe study highlights a massive divide in how medical schools design their Workplace-Based Assessments (WBAs). From two-point "pass/fail" scales to complex ten-point rubrics, the lack of standardization across institutions—and even between departments within the same school—makes comparing student competency a significant hurdle.Key Research FindingsDespite the structural differences in how schools grade, the data revealed a remarkably consistent (and concerning) trend:The "Ceiling Effect": Over 92% of all ratings were positive, with more than 60% hitting the highest possible score.Personality vs. Performance: Qualitative feedback often drifted away from clinical skills (like reasoning or diagnosis) toward personality traits, praising students for being "friendly" or "punctual" rather than offering actionable medical critiques.Administrative Friction: The "time tax" on supervising clinicians often turns evaluations into a "check-the-box" exercise rather than a meaningful coaching moment."The assessment process sometimes becomes a procedural requirement rather than a meaningful learning tool."Why This Matters Beyond MedicineWhile the data comes from hospitals and clinics, the implications reach into any field involving hands-on performance—from the arts to career technical education. Moore and Reeves challenge educators to look at their own data and ask:Are we measuring meaningful growth or just generating reassuring numbers?How do we reduce the cognitive load for the evaluators?Are we distinguishing between minimum competency and true excellence?Tune in to learn how we can move beyond "uninformative data" to create assessment systems that actually help students improve.

In this Friday SLO Talk, J.D. Mosley-Matchett, Senior Assessment Developer at Western Governors University, examines how higher education is responding to artificial intelligence and the broader technological changes affecting teaching and learning. Drawing on more than three decades of experience in higher education as a professor, dean, and administrator, Mosley-Matchett frames the current moment through three competing narratives about the future of universities: “buggy whips,” “rocket ships,” and “total eclipse.”The “buggy whip” narrative reflects the fear that traditional academic practices may become obsolete as knowledge becomes instantly accessible through AI and digital technologies. However, Mosley-Matchett argues that institutions rarely disappear; instead, they adapt and redefine their roles.The “rocket ship” narrative views higher education as a pathway to economic mobility, but this model faces growing pressure as the cost of college rises and questions emerge about grade inflation, credential value, and whether degrees reliably signal competence to employers.The “total eclipse” narrative suggests that AI could replace universities entirely. Mosley-Matchett rejects this view, emphasizing that colleges serve broader purposes beyond information delivery, including collaboration, social learning, and professional networking.Throughout the discussion, participants explore how AI should be incorporated into teaching rather than resisted. Mosley-Matchett argues that institutions have a responsibility to train faculty to use AI effectively and to move away from assessments that reward merely producing the “right answer.” Instead, education should focus on skills, competencies, and the ability to search for and evaluate information.The conversation concludes with reflections on curiosity, student agency, competency-based education, and the evolving role of educators in an AI-rich environment. Rather than replacing higher education, AI is likely to force institutions to reconsider how learning is defined, assessed, and supported.

This Friday SLO Talk with Drs. Ronald and Nancy Martella challenges common misconceptions about behaviorism and reintroduces it as a precise, ethical, and evidence-based framework for understanding learning and motivation. The discussion emphasizes a simple but powerful truth: behavior is shaped by the environment, not by invisible mental states. When teachers design conditions that promote success, learning becomes predictable, measurable, and replicable.Behaviorism views learners as active participants whose actions are selected by consequences. The three-term contingency—Stimulus → Response → Stimulus (S-R-S)—explains how antecedent cues prompt behavior and how reinforcing outcomes make that behavior more likely to reoccur. Reinforcement increases behavior; punishment decreases it. “Positive” means adding a stimulus; “negative” means removing one. What matters is not intent but effect on future behavior.The Martellas dismantled persistent myths:Myth 1: Behaviorism is simplistic “S-R psychology.” Operant conditioning studies voluntary behavior shaped by consequences, far beyond reflexive responses.Myth 2: Behaviorism relies on punishment. Ethical practice requires reinforcement first and limits punishment to rare, last-resort use. Skinner’s work sought humane alternatives.Myth 3: Skinner’s “daughter in a box.” False. The “air-crib” was a safe, climate-controlled baby bed; his daughter later described a happy childhood.Myth 4: Behaviorism ignores motivation. Motivation is explained through environmental variables such as the Premack Principle, Response Deprivation, and Motivating Operations, which alter the value of reinforcers and the probability of behavior.Myth 5: Behaviorism only applies to animals. In reality, it underlies effective human interventions—from autism therapy and literacy instruction to Positive Behavior Supports and MTSS.Myth 6: Behaviorism dismisses the mind. Internal events are acknowledged as behaviors influenced by environment, not mysterious causes. This perspective leads to concrete instructional fixes instead of speculation.The Martellas contrasted this model with cognitive and developmental theories that often rely on labels and circular logic (“She can’t read because she has a reading disability—she has a reading disability because she can’t read”). Behaviorism avoids such traps by identifying functional relationships between environment and action and then changing those conditions to improve performance.They noted three forces shaping behavior—physiology, culture, and environment—and stressed that only the immediate environment is under a teacher’s control. Thus, effective education depends on designing reinforcement systems and clear contingencies that support desired academic and social behaviors.Finally, they linked behaviorism to Skinner’s concept of selection by consequences, analogous to natural selection. Just as adaptive traits survive through reinforcement, effective behaviors are “selected” and strengthened. When reinforcement is consistent, students build repertoires of successful behavior; when it is absent or inconsistent, learning stalls.The message for educators is clear: learning is observable change, not an internal mystery. By focusing on measurable performance, continuous feedback, and well-designed environments, teachers can replace blame with accountability and speculation with evidence. Behaviorism, properly understood, is not mechanical—it is humane, pragmatic, and relentlessly focused on helping every student succeed.

In this episode, educators and innovators explore how generative AI can transform classrooms into continuous experiments in growth. Drawing from the October 10, 2025 session Empowering Inquiry: AI and the Scientific Method in Practice, the discussion reveals how AI can support every stage of scientific inquiry—from background research and hypothesis formation to data analysis and communication.Adult educators Lizelena Iglesias, Vi Hawes, Dr. Jarek Janio, and Enrique Jauregui unpack how AI tools can boost problem-solving, data interpretation, and critical thinking while helping teachers model evidence-based teaching. The conversation reframes prompt engineering as a new literacy—an experimental skill that mirrors how scientists refine hypotheses.Yet, the panel also wrestles with a vital question: how do we preserve independent thought in the age of intelligent tools? Their answer—treat AI as a partner, not a crutch. Listeners will hear practical classroom strategies for balancing AI assistance with authentic student inquiry and learn how the scientific method can guide not only student learning but also teaching itself.

In this session, Dr. Pavithra Suresh and Dr. Sabrina Altema of Howard University share how the Doctor of Physical Therapy (DPT) program has built a student-centered, equity-driven model of assessment that goes far beyond the traditional transcript. Their approach focuses on preparing graduates who are not only clinically competent but also culturally sensitive and deeply committed to serving under-resourced communities.The presentation highlights Howard’s university-wide framework, the Howard Annual Assessment Process (HAP), and its six guiding pillars: centering students and equity, honoring community expertise, prioritizing quality over compliance, fostering collaboration, ensuring transparency, and cultivating lifelong learning.The DPT program illustrates these principles in action. With a 94% licensure pass rate in 2023–24, the program emphasizes training underrepresented physical therapists and embedding community service into the student experience. Assessment is designed “with the end in mind,” developing confident and competent practitioners through scaffolded practical exams, formative feedback, Bloom’s Taxonomy made transparent to students, and authentic clinical experiences supported by standardized evaluation tools and trained preceptors.Evaluation is holistic, capturing cognitive knowledge, psychomotor skills, and affective growth, while also addressing the “hidden curriculum” of professional norms and communication. Faculty monitor progress at both individual and program levels through weekly meetings, developmental teams, comprehensive exams, and curricular mapping aligned with evolving accreditation standards.Key takeaways include the importance of faculty and student buy-in, the value of empowering learners with self-assessment tools, and the role of transparency in deepening engagement. The Howard DPT program demonstrates how assessment can drive both student success and continuous program improvement, ensuring graduates leave with the competence, confidence, and commitment to serve where they are most needed.

Dr. Brian Beatty from San Francisco State University discussing "Addressing the Challenges of Assessment in HyFlex Courses Using Custom GPTs." Dr. Beatty, a professor of instructional design, introduces the concept of HyFlex learning, which blends face-to-face, synchronous, and asynchronous online instruction to offer students flexible participation choices, initially developed to address enrollment issues. A significant portion of the discussion focuses on how he leverages generative AI—specifically custom GPTs built on platforms like ChatGPT—to support both student self-assessment and faculty course design in these complex, multi-modal environments. He details various student-facing GPT tools he created for engagement and formative assessment, such as "QuizMe" and "Breakout for 3," and explains the simple process of building these custom tools. The presentation also addresses assessment challenges in HyFlex, emphasizing the need for equivalent learning outcomes across all modes and the importance of authentic, flexible, and self-assessment strategies.

In this Friday SLO Talk, Jarek Janio from Santa Ana College and Enrique Jauregui from Fresno City College host a dynamic session highlighting an innovative nursing course development project from the University of Texas at Arlington (UTA). Dr. Leslie Jennings, Missina Minter, Megan Zara, and Dr. Stacy Greathouse, an interdisciplinary team nicknamed the "Motley Crew"—share their collaboration model for building a high-impact, competency-based perioperative nursing course.The panel describes how the course originated in response to the national nursing shortage, aiming to prepare students for perioperative roles in operating rooms. Dr. Jennings, a perioperative nurse with over 30 years of experience, led the design effort with the support of an instructional designer, librarians, and an OER specialist. Together, they developed an accelerated "Maymester" course, compressing a full semester’s clinical and didactic content into an intensive two-and-a-half-week experience.Stacy Greathouse introduces the "Dark Classroom" and "Motley Crew" frameworks, emphasizing collaboration, transparency, and the breakdown of traditional course design silos. Each member of the team fulfilled clearly defined roles—content specialist, learning architect, instructional technologist, accessibility specialist, and OER librarian—to ensure smooth workflow, full accessibility, and rigorous alignment with student learning outcomes.A key focus was meticulous alignment: the team employed a master course map, “Holy Hail” language (ensuring consistent terminology across outcomes, content, and assessments), and Quality Matters (QM) standards. They also integrated Open Educational Resources (OER) and accessibility reviews, while using Rice University’s Workload Estimator to help minimize hidden time constraints for students.Jennings and Greathouse explain how an international travel theme unified the course design. Students "traveled" through different stages of perioperative care, earning "passport stamps" as they progressed, with every assignment explicitly tied to course outcomes.The discussion highlights the importance of formative assessments and structured "pause points" to help students reflect and maintain mental wellness during the compressed course. Flipped classroom strategies shifted the responsibility for preparation onto students, promoting agency and ownership of learning.Attendees then engaged in breakout rooms with the "Motley Crew" members to dive deeper into key practices:OER sourcing and licensingAccessibility auditing and integrationInstructional design strategies for accelerated learningBuilding student-centered activities with tools like H5PThe session concluded with a reflection on the cultural changes needed in higher education to support collaborative course development and the critical role of advocating for institutional resources to make sustainable, high-quality programs possible. Attendees left inspired by the team's commitment to authentic assessment, transparency, and student-centered design.Key Takeaways:High-stakes, accelerated nursing education requires intentional design, transparency, and teamwork.True accessibility and OER integration enhance quality and equitable access for all students.A "Motley Crew" team structure—built on mutual accountability and clear roles—produces stronger, more resilient learning environments.Flipped classrooms and competency-based education support student agency while reducing faculty workload.Special Thanks: Thank you to Dr. Leslie Jennings, Missina Minter, Megan Zara, and Stacy Greathouse for sharing their remarkable journey, and to the Friday SLO Talks community for supporting these groundbreaking conversations on student learning outcomes.

Podcast Episode: Assessment of Student Learning – The Case for Observable Behavior with Dr. Jarek JanioIn this episode of Friday SLO Talks, organized by the California Outcomes Assessment Hub (COACHES), Dr. Jarek Janio from Santa Ana College led a compelling discussion on the role of observable behavior in assessing student learning. The session challenged traditional metrics of success such as grades, graduation rates, and course completion by questioning whether they truly reflect learning or simply institutional priorities.The Gap Between Institutional and Student Success MetricsDr. Janio identified a disconnect between what students seek practical skills and competencies and how institutions evaluate success. He argued that while colleges track graduation and employment rates, these indicators do not necessarily capture learning outcomes. He invited participants to consider alternative assessment strategies that focus on what students can do rather than just what they remember or complete.The Role of Learning Theories in AssessmentThe discussion transitioned to educational theories, including behaviorism, constructivism, and transformative learning. Dr. Janio highlighted that these frameworks influence how faculty teach and assess learning, yet they often fail to translate into meaningful, skill-based assessment practices. He proposed flipping Bloom’s Taxonomy on its head starting with application and problem-solving before introducing memorization so that students first engage with real-world challenges and later refine their foundational knowledge in context.Observable Behavior as the Key to Effective AssessmentDr. Janio argued that the only valid measure of learning is observable behavior. He criticized traditional assessments like standardized tests and essays, noting that they often fail to capture real competencies. Instead, he advocated for:Performance-Based Assessments: Evaluating students based on their ability to demonstrate skills in real-world scenarios.Portfolios and Projects: Allowing students to compile evidence of their learning over time.Self-Assessment and Peer Feedback: Encouraging students to reflect on their own progress and engage in meaningful discussions about their learning.AI-Assisted Assessment: Leveraging generative AI tools to analyze student responses, create adaptive learning paths, and enhance formative assessment strategies.The Role of AI in AssessmentDr. Janio explored how artificial intelligence can support assessment by automating rubric development, generating feedback, and helping faculty analyze student progress over time. However, he cautioned against over-reliance on AI-generated responses, emphasizing that faculty must take responsibility for interpreting and applying AI insights in a meaningful way.Redefining Student Engagement and SuccessA significant portion of the discussion focused on student engagement. Dr. Janio questioned whether traditional engagement metrics such as class participation, assignment completion, and time spent in tutoring centers actually indicate learning. He proposed that engagement should instead be measured by students’ ability to articulate and apply what they have learned in meaningful ways.The episode concluded with a call to action: Educators must rethink assessment practices to ensure they capture real learning, rather than relying on outdated proxies like grades and course completion. By shifting toward skill- and competency-based assessment, faculty can provide students with a more meaningful and equitable learning experience.

Here’s a more detailed description of the episode:Podcast Episode: GenAI – Your Supercharged Assessment Assistant with Dr. Gavin HenningIn this episode of Friday SLO Talks, organized by the California Outcomes Assessment Hub (COACHES), Dr. Gavin Henning, professor of higher education at New England College, provided an in-depth look at how Generative AI (GenAI) is revolutionizing assessment in higher education. With over 25 years of experience in student learning assessment and institutional research, Dr. Henning explored the practical applications of AI in streamlining assessment processes, reducing faculty workload, and improving the quality of learning outcomes.Moderated by Dr. Jarek Janio from Santa Ana College and Enrique Jauregui from Fresno City College, the session opened with a discussion on the rapid evolution of AI, highlighting how tools such as ChatGPT, NotebookLM, and Gamma are reshaping assessment strategies. Dr. Henning acknowledged the common anxieties faculty have about AI—particularly concerns about academic integrity and student cheating—but emphasized the immense potential of AI to enhance, rather than undermine, meaningful assessment.Throughout the presentation, Dr. Henning demonstrated multiple AI-driven applications for assessment, including:Developing and Refining Learning Outcomes – AI can generate student learning outcomes based on course descriptions, revise them for clarity, and align them with different learning taxonomies such as Bloom’s Taxonomy or Fink’s Significant Learning.Rubric Creation and Evaluation – AI tools can draft assessment rubrics, refine them based on institutional needs, and help faculty evaluate the quality of their rubrics to ensure clear, measurable learning criteria.Survey and Interview Protocols – AI can assist in designing assessment instruments, such as survey questions and focus group protocols, reducing the time needed to create structured assessment tools.Qualitative and Quantitative Data Analysis – While Dr. Henning cautioned against relying too heavily on AI for statistical analyses, he demonstrated how AI can identify themes in qualitative data, synthesize feedback, and even generate reports summarizing key findings.Program Review and Accreditation Support – AI can streamline the program review process by helping faculty organize assessment data, generate accreditation reports, and compare institutional assessment strategies.Dr. Henning also showcased how AI-generated reports and summaries can be transformed into interactive learning materials, including AI-generated podcasts. He demonstrated Google’s NotebookLM, a tool that allows users to create AI-powered podcasts from assessment reports, complete with interactive question-and-answer capabilities. This innovative application of AI, he suggested, could make assessment findings more engaging and accessible to broader audiences.The discussion also touched on the ethical implications of AI in assessment, including concerns about bias, data privacy, and the environmental cost of AI technology. Dr. Henning encouraged faculty to approach AI as a tool for efficiency and equity while remaining vigilant about its limitations. He also highlighted the importance of maintaining faculty agency in assessment design, ensuring that AI complements, rather than replaces, human expertise.Dr. Henning’s message was clear: AI is moving at the speed of the Road Runner, and while it presents challenges, it also offers unprecedented opportunities to improve assessment, streamline faculty workload, and enhance student learning outcomes.

1. The Evolving Role of Assessment Professionals: As AI automates routine tasks, assessment professionals must focus on higher-order skills like identifying key data, designing effective data collection strategies, and interpreting AI-generated insights. "These tools are going to allow us to collect a lot more data about students… giving us deeper insights into what’s working and what’s not in our programs."2. Agent-Based AI Models: The transition from dialogue-based tools (e.g., ChatGPT) to autonomous agent-based models introduces new considerations for task delegation and ethical use. "When we have models that we can delegate tasks to, it becomes critical to be thoughtful about what tasks are delegated."3. Thoughtful AI Adoption: Implementing AI in education requires careful consideration of student needs, data privacy, and equity to ensure ethical and effective use.Most Important IdeasAutomation of Routine Tasks: AI streamlines assessment, freeing professionals for strategic roles. For example, Walton’s tool analyzes student artifacts automatically using pre-defined rubrics.Prompt Quality Matters: The "garbage in, garbage out" principle applies. Poorly crafted prompts lead to suboptimal AI outputs. "If you give it a dumb prompt, you’ll access the dumb part of that model’s brain."Equity Concerns: Wealthier institutions investing heavily in AI may widen gaps between privileged and underserved students. "This is the single biggest equity topic…wealthier schools are throwing money at AI for their students."Human Connection is Essential: AI should complement, not replace, human tutors, allowing them to focus on personalized support and emotional needs.Communicating AI Benefits: Institutions must clearly convey the advantages of AI-powered tools to students and parents. "How are we going to package this…in a way that helps students understand the benefit?"Key QuotesDevan Walton: "We’re shifting to agent-based tools…thoughtfulness about delegated tasks is crucial."Peter Shea: "Wealthier schools are already investing heavily in AI, creating an equity gap."Anne Converse Willkomm: "We need to help students understand AI’s benefits."Ruth Slotnick: "This is a once-in-a-generation technology…bigger than the Internet."Call to ActionEngage with AI: Assessment professionals should develop a deep understanding of AI’s capabilities and limitations.Create Ethical Frameworks: Institutions must establish guidelines for responsible AI use in education.Foster Collaboration: Open dialogue between educators, students, and technology experts is essential to ensure AI serves all stakeholders effectively.AI offers transformative potential for higher education assessment, but ethical, equitable, and strategic adoption is critical to harness its benefits fully.

AI and Instructional Design in Higher Education1. A New Pedagogical Model: AI offers an opportunity to transition from traditional teaching methods to "generativism," a novel approach combining evidence-based learning principles with AI technology. This shift is essential for maximizing AI's potential. "We are largely trying to use AI to drag along an older pedagogical model... rather than build an entirely new model using the unique affordances of AI."2. Institutional AI Adoption: Colleges and universities vary in their approach to AI adoption, characterized as:Ostriches: Ignoring AI.Crows: Slowly exploring policies.Falcons: Actively experimenting with AI in teaching and learning. "Falcons tend to be those schools... exploring AI for tutoring chatbots, experiential learning, and simulations."3. Transformative Instructional Design: AI revolutionizes instructional design by enhancing productivity, creating interactive learning tools, and enabling performance-based assessment. "The ability to create learning tools that observe students while they perform tasks is one of the most exciting possibilities."4. Equity and Accessibility: AI bridges equity gaps by providing consistent, high-quality assessment and feedback. It also supports continuous learning through accessible ecosystems beyond the classroom. "A well-designed AI, guided by a rubric, can generate high-quality feedback quickly. For me as a writing instructor, it's revolutionized student support."Most Important IdeasGenerativism: A pedagogical approach leveraging AI's unique capabilities to reimagine teaching and learning.Instructional Design Productivity: AI streamlines material creation, like simulations, freeing instructional designers for strategic roles.Performance-Based Assessment: AI enables real-time tracking and assessment of student performance, improving personalization and outcomes.AI-Powered Ecosystems: AI fosters engaging, continuous learning environments beyond the classroom, enhancing knowledge retention.Key TakeawaysInstitutions must embrace new models like generativism to harness AI's transformative potential.AI enhances instructional design efficiency, personalizes learning, and bridges equity gaps in feedback and assessment.Developing AI-powered ecosystems is vital for lifelong learning and extending support beyond traditional classrooms.RecommendationsAdopt Generativism: Incorporate generativism principles into instructional design.Invest in AI Tools: Focus on tools that create interactive and engaging materials.Implement Performance-Based Assessment: Integrate real-time AI-driven feedback into learning experiences.Collaborate: Build partnerships among instructional designers, faculty, and technology experts for effective AI solutions.Professional Development: Stay informed about evolving AI trends in education.AI offers transformative potential for higher education, requiring institutions to rethink pedagogy, embrace innovation, and prioritize accessibility and equity.

Cautionary Points and Considerations Around the Adoption and Use of AI in Higher Education1. AccessibilityAvailability & Communication: Institutions must clarify which AI tools are accessible to students, faculty, and staff, balancing enterprise versus free versions and effectively communicating available resources.Cost & Equity: Free versions often have security risks and limited features, while enterprise versions raise concerns about equitable access for all students.Faculty Attitudes: Faculty views on AI range from banning its use to fully embracing it. Those embracing AI must educate students on responsible and ethical usage.2. Dependence on Generative AITime Investment & Efficiency: Crafting prompts for AI can be time-consuming. Institutions must find a balance between exploration and efficiency to avoid wasted effort.Over-Reliance & Cognitive Skills: Overuse of AI may harm critical thinking, creativity, and ethical writing. A 2024 Smart Learning Environments study noted AI can boost writing proficiency but risks originality and ethical standards.3. Environmental CostsElectronic Waste & Resource Use: AI data centers generate significant e-waste and consume large amounts of raw materials.Energy & Power Demands: These centers require high energy consumption, mostly from fossil fuels, contributing to greenhouse gas emissions. For example, "A ChatGPT request uses 10 times the electricity of a Google search."Water Usage: Cooling and constructing AI systems require substantial water resources, posing environmental concerns.4. Misinformation & DisinformationBias: Algorithmic and user biases can produce misleading information. Confirmation bias, where users shape prompts to affirm beliefs, is particularly problematic.Inaccurate Information: AI-generated "hallucinations" (false outputs) pose a significant challenge.5. Privacy & Consent IssuesCyber Threats: AI facilitates cyber risks like malware, deep fakes, phishing, and impersonation.Non-Consensual Data Use: Many AI systems lack clear mechanisms for opting out of data collection, raising ethical concerns about personal data use.Input Privacy: The security of data submitted to AI remains a major issue.Looking AheadEnterprise AI & Critical Thinking: The growing adoption of enterprise AI may hinder long-term critical thinking, echoing social media's effects on judgment.Environmental Impact & Regulation: While AI offers environmental solutions, stricter policies are needed to mitigate its negative impacts. Promising initiatives from companies like Google and Microsoft to pair data centers with renewable energy provide hope for sustainability.Misinformation & Disinformation: Addressing these issues requires greater regulation at all levels.Job Displacement: Planning is essential to address AI-driven job disruptions.ConclusionAI offers significant potential for higher education, but addressing its risks is essential. Institutions must prioritize awareness, education, and proactive research to ensure ethical and responsible AI integration within academia.

AI in Higher Education: Policy, Impact, and EthicsDrawing from her experience as a faculty member and policy developer at Drexel University, Willkomm identifies critical focus areas as institutions adapt to the fast-evolving landscape of AI.I. Policy Considerations:Willkomm highlights the urgency of developing AI policies that address the needs of all stakeholders: faculty, students, and staff.Faculty: Institutions must ensure faculty understand AI's capabilities, limitations, and ethical implications. Providing clear guidelines on responsible AI use in classrooms is vital for both educators and students. "We created documents to help faculty understand AI—what it truly is and how it can be used. Some are familiar with it; others avoid it altogether."Students: Willkomm warns against over-reliance on AI detection tools due to their limitations and biases. Instead, she advocates for an educational approach that promotes ethical and responsible AI use over punitive measures. "We need to focus on education rather than punishment when dealing with students and AI."Staff: Often overlooked in AI policy discussions, staff need clear guidelines on AI usage, especially for handling sensitive data. "Staff are frequently excluded from these conversations, which is problematic."II. Potential Impacts:Willkomm underscores the consequences of unclear or inconsistent AI practices.Faculty: A lack of guidance leads to confusion and inequity in how AI is applied, creating disparities for students. "Inconsistent application of AI tools by faculty promotes inequity."Staff: Ambiguity around AI usage can result in misuse, including unintentional breaches of data privacy. "Without clear boundaries, even unintentional misuse of AI could lead to serious issues."Students: Navigating inconsistent AI practices across courses hinders students' learning experiences. "Students face challenges when AI usage varies from one class to another."III. Ethical Considerations:Willkomm strongly advocates embedding ethics into all aspects of AI integration.Faculty: Educators should discuss AI ethics with students, model responsible use, and treat misuse as teachable moments. "We must talk to students about ethical AI usage and model it ourselves."Staff: Establishing ethical guidelines for staff is essential, especially for work involving sensitive data. "Universities need clear rules on when staff can ethically use AI for tasks like assessment or reporting."Students: Teaching students about AI ethics now prepares them for responsible use in their careers. "If we don't address AI ethics consistently, what are we truly teaching them for their future in industry?"Willkomm's insights emphasize the importance of policies, equity, and ethics as institutions incorporate AI into education and operations.

The Impact of AI on Assessment ProfessionalsSource: Excerpts from "Bobbijo Grillo Pinnelli.txt"(transcript of the presentation)Main Themes:The rapid evolution of AI and its impact on the roles of assessment professionals.The need for assessment professionals to adapt and acquire new skills to remain relevant in the age of AI.A predicted shift from traditional assessment roles towards more strategic, coaching, and leadership positions.Key Ideas and Facts:1. The Changing Landscape:The speaker, Bobbijo Pinnelli, reflects on the initial anxiety and uncertainty surrounding AI's impact on assessment roles, similar to the feelings experienced during the rise of instructional design tools.Quote: "I couldn't keep up with AI. And what I know now is that we'll never be able to keep up with the pace of AI, and that's okay?"It's acknowledged that AI is rapidly evolving and will continue to do so, requiring constant adaptation from professionals.2. AI's Efficiency and Potential Obsolescence:Pinnelli acknowledges the potential for AI to replace some assessment functions, particularly those heavily reliant on processing and basic tasks.Quote: "Will some of the roles in the assessment become completely obsolete due to the efficiency the AI offers."However, roles involving leadership, creativity, coaching, and mentoring are deemed essential and likely to remain in demand.3. The Need for Skill Development and Adaptation:Assessment professionals are urged to identify their current skill sets and areas where AI can be leveraged.Quote: "I need to identify where I stand right like, what do I already know how to do around AI."Acquiring new skills, applying them in practice, and collaborating with others are crucial steps in remaining relevant.4. A Shift in Roles and Focus:Pinnelli predicts a significant shift from traditional assessment roles to more strategic positions, such as "experienced architects," coaches, and strategists.Quote: "We are going to see an entirely new position in the field. For some...some will develop the skills and capacities needed for that experience architect role, but not everybody will get there."This shift necessitates a greater emphasis on interpersonal skills and human oversight in AI-driven assessment processes.5. Call to Action and Career Development:Assessment professionals are urged to engage in self-reflection and analyze their current roles and responsibilities.Quote: "Think about in your current position...how much time? What is your responsibility? What percentage of responsibility is spent processing, what percentage is reporting, what percentage is consuming data, creating leading others, and really be very honest, and sit and look at that."This introspection helps identify areas where skills development is needed to stay ahead of the curve and ensure continued relevance in the evolving field of assessment.Overall, the excerpt highlights the critical need for assessment professionals to proactively embrace the opportunities and challenges presented by AI. By focusing on developing new skills, adapting to changing roles, and prioritizing human oversight, professionals can remain relevant and effectively support student learning in the age of AI.

AI in Higher Education AssessmentMain Themes:Urgency of AI adoption in assessment: Assessment professionals need to proactively engage with AI tools and their implications for student learning despite feelings of overload or a wait-and-see approach.Leadership role of assessment professionals: Professionals must lead AI adoption in their institutions, building policies and practices, and collaborating within and across institutions.Human-AI partnership: While AI offers opportunities to enhance assessment practices, human oversight remains crucial for ethical considerations and ensuring the validity of insights.Shifting roles in assessment: The traditional role of the assessment director is predicted to evolve into an “experience architect,” focusing on personalized learning and leveraging AI to predict student success.Key Facts and Ideas:The Java Jams initiative, a year-long community-driven effort, focused on exploring the intersection of AI and higher education assessment.A survey of 264 assessment professionals revealed:50% believe AI will play a major or moderate role in assessment work.70% are not currently using AI in assessment.55% feel prepared for AI integration despite limited practical experience.Two main reasons for hesitation: technology overload and a wait-and-see attitude for AI to mature.Successful AI integration in assessment requires:Experimentation with AI tools to understand their capabilities and limitations.Collaborative efforts across departments and institutions to share knowledge and best practices.Proactive leadership in developing policies and guidelines for ethical and effective AI use.The future of assessment points towards a more personalized learning experience, with AI playing a key role in predicting student success and shaping learning pathways.Important Quotes:On the need for leadership: "If you're an assessment professional in AI, you got to get out there and lead. If you have no idea what you're talking about in AI... you gotta experiment with the tools." - Ruth SlotnickOn human involvement: "The human always stays in the loop... for insight and oversight, for validity, and just ethically." - Ruth SlotnickOn the future of assessment: "We will be shapeshifting from that traditional assessment director to something else... moving more to what I call the experience architect." - Ruth SlotnickOn hesitancy to adopt AI: "One person said that they have, since the pandemic, technology overload... and the second one is, 'I'm not concerned at all... I just want others to fine tune it.'" - Ruth SlotnickCall to Action:The Java Jams Grand Finale serves as a call to action for assessment professionals to actively engage with AI and lead its integration into higher education assessment practices. By embracing experimentation, collaboration, and a forward-thinking approach, professionals can harness the power of AI to enhance student learning and shape the future of assessment.

Demystifying Student Learning Outcomes (SLOs) in Higher EducationMain Themes:Defining and differentiating Student Learning Outcomes (SLOs) from other outcome measures in higher education.Understanding the relationship between SLOs and student achievement.Exploring the interconnected levels of learning outcomes: Institutional (ILOs), Program (PLOs), and Student (SLOs).Addressing the challenge of aligning outcomes across these levels to ensure meaningful learning and achievement.Key Ideas and Facts:1. What are SLOs?SLOs are "observable and measurable demonstrations of what students can do as a result of instruction." They represent the specific skills, knowledge, abilities, and attitudes students should acquire during a course, program, or activity.SLOs vs. Student Achievement:While both relate to student progress, they differ in focus:SLOs emphasize the learning process and application of acquired skills. Examples include writing a persuasive essay, solving practical math problems, or demonstrating software proficiency.Student achievement focuses on measurable milestones like grades, test scores, and completion rates. These are aggregated at an institutional level and don't always reflect specific skills learned.Relationship: SLOs are the building blocks of student achievement. Aligning instruction with well-defined SLOs ensures that achievement reflects genuine learning.2. Levels of Learning Outcomes:ILOs (Institutional Learning Outcomes): Broadest level, aligning with the institution's mission and vision. They represent overarching competencies all students should develop, like critical thinking, communication, and problem-solving.PLOs (Program Learning Outcomes): Describe what students should achieve by completing a program. They often align with external guidelines and standards relevant to the discipline.SLOs (Student Learning Outcomes): Most specific level, tied to individual courses. They specify the knowledge, skills, and competencies students should demonstrate after completing the course.Alignment: These levels are interconnected, creating a cohesive learning experience:SLOs (course-level) build foundational skills.PLOs (program-level) apply and synthesize these skills, leading to program-specific competencies.ILOs (institutional-level) translate these competencies into broader skills aligned with the institution's goals.Key Distinction:All student learning outcomes are assessed at the course level through observable behavior, not just through surveys or evaluations.Challenges:The term "SLOs" is not widely recognized outside academia, leading to communication challenges.The term "outcomes" is often used ambiguously in institutional accountability, encompassing both learning and achievement, which can obscure the focus on genuine learning.

Centering Learning and Serving StudentsMain Themes:Student-centric approach: The current higher education system often prioritizes administrative processes and metrics over student needs and learning outcomes.Accountability through SLO assessment: Student Learning Outcomes (SLO) assessment is crucial for ensuring students gain real-world skills and knowledge, yet it remains underfunded and underutilized.Empowering faculty: Faculty are key drivers of student success but require adequate resources and institutional support to focus on learning.Redefining success: Institutions need to shift their focus from attendance and completion to actual learning outcomes as the primary measure of success.Key Ideas & Facts:The current system can be a barrier for non-traditional students, with complicated processes, long wait times, and financial hurdles."Imagine this scenario: a student loses a job right before the holidays and turns to a community college for help, only to discover they can’t meet with a counselor until mid-January."Institutions often measure attendance, persistence, and completion, but these metrics don't necessarily reflect the quality of education or student learning."Yet, while institutions measure attendance, persistence, and completion, they rarely measure what truly matters: learning."SLO assessment is crucial for accountability and ensuring students develop valuable skills, yet it lacks sufficient funding."If we want to transform this system, funding for SLO assessment is non-negotiable."Faculty play a critical role in student learning but face constraints due to limited resources and institutional support."Faculty must be empowered—not just through funding but through institutional commitment—to make learning the central focus of everything we do."Institutions need to prioritize learning outcomes as the primary measure of success, reflecting the true purpose of higher education."Students don’t attend college to sit in a classroom; they come to acquire skills, gain knowledge, and change their lives."Call to Action:The briefing document advocates for a paradigm shift in higher education, calling for:Increased funding for SLO assessment to ensure accountability and focus on learning outcomes.Stronger institutional support for faculty, providing them with the resources and autonomy to prioritize student learning.A fundamental shift in how success is measured, moving beyond attendance and completion to focus on demonstrable learning outcomes.By implementing these changes, higher education can rebuild trust and ensure that every student's journey leads to meaningful success. The focus must be on serving the students and empowering them with the knowledge and skills they need to thrive.

Making Student Learning Outcomes (SLO) Assessment Meaningful and ActionableThis briefing doc reviews key takeaways and advice from Friday SLO Talks, hosted by California Outcomes Assessment Coordinators’ Hub (COACHES). It explores tools, training, and frameworks to enhance SLO assessment and provides guidance for those new to the process.Main ThemesMoving beyond compliance to focus on student learning. SLO assessment should be viewed as a tool to enhance student learning and skill acquisition rather than a bureaucratic box to tick.Collaboration and leveraging resources. Connecting with colleagues, utilizing available technology, and engaging with professional development opportunities are crucial to successful SLO implementation.Starting small and celebrating progress. Beginning with manageable projects and recognizing successes can build momentum and foster a culture of continuous improvement.Key Ideas and FactsCOACHES provides a wealth of resources:Curated website with resources for coordinators and facultyAI-driven tools for communication including podcasts, YouTube channels, and social mediaRecorded Friday SLO Talks sessionsAnnual SLO Symposium attracting over 1,000 attendees nationwideBeyond COACHES, valuable tools and frameworks exist:Bloom’s Taxonomy: Helps design measurable learning outcomes progressing from basic to higher-order thinking skills.AAC&U's VALUE Rubrics: Offers detailed criteria for assessing essential learning outcomes.Learning Management Systems (LMS): Built-in tools for data collection, analysis, and reporting.National Institute for Learning Outcomes Assessment (NILOA): Provides research, tools, and best practices for assessment improvement.Advice for newcomers to the SLO journey:Start Small: Begin with a single course or program and focus on defining clear, measurable outcomes.Collaborate: Engage with colleagues within your institution and through networks like COACHES.Leverage Technology: Use AI and LMS tools to simplify data collection and analysis.Focus on Learning: Approach SLOs as a way to enhance student learning.Seek Professional Development: Attend workshops, webinars, and conferences.Celebrate Progress: Recognize and share successes to build momentum.Quotes"Focus on Learning, Not Compliance: Approach SLOs as a way to enhance student learning, not as a bureaucratic requirement.""Collaboration can reduce the feeling of isolation that often comes with SLO work.""SLO assessment is a continuous journey."ConclusionSLO assessment is an ongoing process that requires commitment, collaboration, and the utilization of available resources. By focusing on student learning and embracing a culture of continuous improvement, educators can ensure that SLOs serve their intended purpose: to equip students with the skills and knowledge they need to succeed.

Harnessing Generative AI for Enhanced LearningSource: Transcript of Dr. Curt Bonk's presentation. Friday SLO Talk: "How faculty can harness generative AI for enhanced learning. Part one", hosted by the Coaches Community College network.Main Themes:The educational landscape is rapidly changing: AI is the hot topic in education today, similar to MOOCs a decade ago.Generative AI presents both opportunities and challenges: While students are already utilizing AI for learning, educators are navigating awareness, resistance, and implementation stages.AI can enhance learning by fostering deeper engagement and critical thinking: Dr. Bonk emphasizes going beyond knowledge acquisition towards higher-order thinking skills.Two frameworks for incorporating AI in pedagogy: Dr. Bonk introduces his "Tech Variety" model and the "R2D2" (Read, Reflect, Display, Do) model.Important Ideas/Facts:Student adoption of AI is growing: Studies show significant increases in student use of AI for learning and its perceived impact on grades.AI skills are in high demand: Graduates with AI experience on their resumes are securing better job opportunities and salaries.Numerous resources are available for educators: Dr. Bonk highlights several books, articles, and websites offering practical examples and guidance on AI pedagogy.Ethical considerations surrounding AI in education must be addressed: Discussions on bias in AI-generated content and the implications of students training AI are crucial.Key Quotes:"Data is emerging that indicates students may be more likely to be considered for high paying jobs if they have experience with AI in their job applications." - Ray Schroeder, University of Illinois, Springfield."My job is not to police everybody in here, nor I really don't to rank them, but to find ways to inspire students to get them to join me in the learning process." - Professor at Middlebury College."By leveraging AI, we're able to author not only comprehensive case studies, but generate relevant dialogue feedback with branches that take the learner on a rich journey." - Instructional Technologist, Open RN Project."I cannot do this alone. I need all your help." - Dr. Curt BonkModels for AI Integration:Tech Variety Model: Based on 10 principles of motivation, it encourages educators to incorporate AI tools in diverse ways to enhance learner engagement.R2D2 Model: Focuses on leveraging AI to facilitate reading comprehension, critical reflection, visual representation, and hands-on activities.Call to Action:Dr. Bonk urges educators to embrace AI, explore available resources, and collaborate to develop innovative pedagogical approaches that harness the potential of AI for enhanced learning experiences. He emphasizes that individual efforts are not enough and collaborative action is needed to navigate this transformative period in education.

Main Themes:The Higher Education Social ContractUniversities must certify students' knowledge and abilities to meet societal expectations.Generative AI (Gen AI) and Academic IntegrityGen AI introduces challenges and opportunities for education, affecting teaching and learning integrity.Shifting from Performance to MasteryTransitioning from performance-based assessments to mastery-oriented learning can reduce cheating and align with the realities of Gen AI.Most Important Ideas/Facts:The Social Contract and Gen AIHigher education's promise to certify students' abilities is threatened by Gen AI misuse.Gen AI enables students to misrepresent their knowledge and skills, undermining trust.Understanding Student Behavior and Gen AIStudents use Gen AI primarily to improve grades, showing extrinsic motivation.Concerns include learning quality, job readiness, and fairness in academic competition.Crafting a Gen AI PolicyClear guidelines on Gen AI use are essential, with options including banning, conditional use, or full integration.Secure assessments are critical to preserving academic integrity, despite ethical concerns around surveillance.Shifting from Performance to MasteryFocus on mastery by:Updating learning outcomes for Gen AI contexts.Personalizing and real-world-connected assessments.Prioritizing learning processes and scaffolding.Using active learning and transparent writing strategies.Adjusting grading rubrics to reflect mastery.AI Detection ToolsUse AI detection tools responsibly, following institutional guidelines, and avoid sole reliance on them for identifying misconduct.Key Takeaways:Gen AI prompts the need for revised teaching and assessment practices in higher education.Open communication, clear policies, and mastery-oriented learning are crucial to maintaining academic integrity in the age of AI.

Driving Change and Co-Creation in Higher EducationFriday SLO Talk: Janet De Wilde, Emily Salines, and Stephanie Fuller of Queen Mary University of London. The presentation explores the challenges and opportunities of driving change in large, legacy institutions, particularly in the context of evolving student needs, economic pressures, and technological advancements.Co-creation as a Mechanism for Change: The speakers advocate for co-creation as a collaborative approach to implementing change, involving students, staff, and various stakeholders in the process.Leadership, Scholarship, and Recognition: The presentation highlights the importance of effective leadership, scholarship, and recognition systems to support change initiatives and empower individuals within the institution.Driving Forces for Change:New Regulations and Socioeconomic Pressures: Universities face increasingly specific and challenging regulations in the UK, coupled with socioeconomic pressures impacting the sector, institutions, and students alike.Diverse Student Needs and Employability: The need to address the diverse requirements of growing student cohorts and the emphasis on employability drive efforts to meet skills gaps and prepare students for the workforce.Challenges to Change:Legacy Systems and Resistance: "People like and feel reassured and comfortable. And and that's our challenge... we need a shared understanding of the need for change."Inconsistent Understanding and Practice: "We had inconsistency of practice, and we had inconsistency of educational change. So that was our challenge is, you know, it's very spread out. And it's very different understandings."Strategies for Implementing Change:Alignment and Career Pathways: Establishing alignment between the strategy, institutional frameworks, and career pathways was crucial for effective implementation.Role-Based Leadership Programs: Specific educational leadership programs focused on role-based leadership, such as empowering module organizers to lead effectively.Focus on Scholarship and Evidence-Informed Practice: Significant investment in leadership and training helped staff develop evidence-based approaches to evaluating and implementing changes in their practices.Utilizing Co-Creation: Co-creation was embraced as a key mechanism to drive change and ensure widespread engagement and collaboration.Co-Creation in Practice:Shared Understanding and Distributed Leadership: The 2030 strategy faced challenges because many assumed it was solely the university's responsibility. To succeed, individuals needed to understand their accountability and feel empowered to act with agency.Importance of Trust and Recognition: Trust develops through effective frameworks, rewards, and recognition, which foster a sense of reliability and collaboration.Student Involvement and Recognition: Students played a key role in co-creating the University's graduate attributes, leading to the creation of the Seed Award to honor student-enhanced engagement.Key Quotes:"Strategy can drive change. But the whole thing needs scaffolding so that Staff can engage. We can't just have a strategy.""Co-creation at Queen Mary is central to our strategy. We will deliver outstanding inclusive world class education co-created with our diverse student body."

Exploring Assessment Choice to Enhance Practice and Inclusivity in Higher EducationFriday SLO Talks: Stephanie Marshall, Janet De Wilde, Emily Salines, and Stephanie Fuller of Queen Mary University of London's Queen Mary Academy.Main Themes:Inclusive Assessment: Queen Mary University prioritizes inclusive assessment practices, recognizing that assessment drives learning and significantly impacts student outcomes and well-being. The presentation explores how offering assessment choices can enhance inclusivity and support students from diverse backgrounds.Assessment Choice as a Driver for Change: The presentation highlights the benefits of assessment choice, such as promoting self-regulation, self-efficacy, engagement, and performance, aligning with principles of Universal Design for Learning. It showcases a small-scale research project exploring the impact of offering choice in a postgraduate academic practice program.Addressing Barriers to Change: The presentation acknowledges challenges in implementing assessment changes within a large institution, including potential resistance to moving away from traditional approaches, concerns about workload, and navigating regulatory frameworks.The Role of Educational Development: The presentation emphasizes the role of educational development units like the Queen Mary Academy in driving assessment change. It positions educational development programs as catalysts for fostering a community of innovative educators and facilitating a shift toward more inclusive practices.Burkana Institute's Two-Loop Change Model: The presentation draws inspiration from the Burkana Institute's two-loop change model, highlighting the importance of identifying and supporting pioneers of change, convening networks of innovators, and strategically phasing out outdated practices while respecting their value.Key Ideas and Facts:Queen Mary University prides itself on being an inclusive Russell Group University, committed to transforming lives and opening doors of opportunity for its diverse student body."You can survive bad teaching, but you can't survive bad assessment." This quote highlights the lasting negative impact of poor assessment practices on student learning and outcomes.A national student survey in the UK consistently reveals student dissatisfaction with assessment and feedback, underscoring the need for reform."Rethinking assessment really means rethinking long-held beliefs, and that can be difficult and painful at times." This quote acknowledges the emotional dimension of challenging ingrained assessment practices.A research project on introducing assessment choice in a postgraduate program showed that most participants found the experience positive, inclusive, and less stressful. 90% of respondents believed that offering assessment choice would be beneficial for their own students. 60% were planning to implement it in their practice.The Burkana Institute’s model encourages identifying "pioneers" of change, "convening" them into communities of practice, and utilizing "storytellers" to disseminate innovative approaches.Quotes:"Assessment drives students' learning.""[Assessment is] a key element in supporting social justice in education." Having the choice was surprisingly empowering."

Learning Principles and Motivation Source: Excerpts from a Friday SLO Talk webinar featuring Dr. Marie Norman, co-author of How Learning Works: Eight Research-Based Principles for Smart Teaching. Main Themes:Teaching vs. Learning: Content expertise alone doesn’t ensure effective teaching. For instance, Isaac Newton, despite his brilliance, struggled to convey his knowledge through lectures."If you know a lot about something... you are then automatically qualified to teach. But... teaching has its own very specific set of skills and knowledge."Importance of Prior Knowledge: Students bring varied knowledge, beliefs, and misconceptions to the classroom. Effective learning relies on activating, building upon, and addressing existing knowledge gaps and inaccuracies."Prior knowledge... is the single most important component in new learning." Challenges include:Inactive knowledge: Difficulty recognizing or applying prior knowledge.Gaps in knowledge: Missing foundational information.Inappropriate prior knowledge: Misapplying correct knowledge.Inaccurate prior knowledge: Holding misconceptions.Dr. Norman’s card test illustrates this: students struggled with an abstract problem but succeeded when it was framed in a familiar context (drinking age rules).Expert Blind Spot: Experienced instructors may overestimate students' understanding. For example, in a research methods course, a professor assumed students could apply statistical tests, overlooking the actual complexity for novice learners."It simply requires that students take a data set, select and apply... things which are actually quite complex."Motivation as a Key Driver of Learning: Motivation fuels student effort, with two crucial aspects:Value: Students’ perception of relevance.Expectancy: Belief in their ability to succeed and in the fairness of the environment."Students' motivation generates, directs, and sustains what they do to learn." Strategies to foster motivation include connecting content to student interests, promoting competence and autonomy, and providing clear expectations and feedback.Recommended Resources:How Learning Works by Ambrose et al.Make It Stick: The Science of Successful Learning by Brown, Roediger, & McDanielTeach Like a Champion by LemovConclusion: Dr. Norman underscores the value of understanding and applying learning principles. Addressing issues such as prior knowledge and motivation allows instructors to create more engaging and effective learning experiences.

This podcast explores the pressing need for a transformation in higher education assessment practices, particularly given the impact of generative AI. Professor Elkington argues that traditional, high-stakes assessments fail to prepare students adequately for modern careers, advocating instead for a learning-centered approach that values process, feedback, and evaluative skills.Key Themes and Insights:Assessment’s Evolving Role: Current assessments often impose high stakes with limited relevance to future career demands. The narrow range of formats limits student expression. Generative AI challenges traditional assessments, prompting a reassessment of academic integrity and a renewed focus on distinctly human skills.“The high-risk nature of many of our established assessment practices. We’re weighting assessments at 100, so students have a single opportunity to demonstrate their learning. That’s high-risk and high-stress.”Three Purposes of Assessment:Assessment of Learning: Summative, focused on final grades.Assessment for Learning: Formative, guiding through feedback.Assessment as Learning: Engaging students with real-world issues and metacognitive skill-building.“Assessment as learning focuses on authentic problems, issues, challenges, and projects.”Principles for Effective Assessment:Alignment with Learning Outcomes: Ensuring assessments reflect key skills.Engaging Tasks: Encouraging active, relevant learning.Timely Feedback: Providing actionable insights for immediate and future improvement.Building Evaluative Expertise: Guiding students to recognize quality work.“When it comes to assessment, you get what you model for.”Flexible Assessment at Teesside University: Teesside emphasizes inclusivity and transparency in its assessment design, offering students choice and promoting autonomy.“Flexible assessment is inclusive, learning-focused, and transparent.”Broadening the Assessment Lens: Recognizing students’ entire learning journey fosters coherence. Elkington calls for “assessment practice with an S” (processes) alongside “assessment practice with a C” (outcomes) to enable a more holistic and progressive approach.“We need to embed assessment practice with an S [processes] and assessment practice with a C [outcomes].”Assessment in an AI-Driven Era: Embracing flexible, multimodal assessment formats is crucial. Offering diverse submission options and setting negotiated criteria encourages authentic, skill-based learning for graduates in an AI-influenced world.Practical Strategies for Modern Assessment:Employer Engagement: Align assessments with authentic, industry-related tasks.Student Agency: Promote self-regulation and ownership.Collaboration: Prepare students for teamwork with collaborative projects.Adaptability: Shift towards flexible, personalized assessments.Analogies:Swiss Cheese Paradox: Layering multiple assessment types to minimize learning gaps.Mass-Produced vs. Crafted Chairs: Choosing between standardization and personalized, creative skill demonstration.

In this episode, Dr. Gianina Baker, Associate Director of the Office of Community College Research and Leadership and Acting Director of the National Institute for Learning Outcomes Assessment (NILOA), explores how assessment practices in higher education can shift from a compliance-driven model to one focused on student empowerment and continuous improvement.Key themes include:Evolving Assessment Practices: Baker outlines how assessment began with an accreditation-driven approach but has increasingly moved towards a model that values student-centered and improvement-focused assessment.Transparency in Communication: The NILOA Transparency Framework is highlighted as a guide for institutions to effectively communicate assessment practices and learning outcomes to both internal and external stakeholders. Transparency goes beyond availability—it requires clarity and context.Using Data for Improvement: Emphasizing action over mere data collection, Baker calls for using assessment evidence to inform decisions, support improvement initiatives, and communicate the value of student learning.Equity in Assessment: Baker stresses the importance of analyzing data by student demographics to address disparities and support all learners effectively.Notable points include:The NILOA Transparency Framework: Comprised of six components, this framework supports institutions in making learning evidence clear, accessible, and actionable.Student-Centered Assessment: Moving beyond accreditation, the presentation emphasizes putting students at the heart of assessment through agency, feedback loops, and a holistic view of learning experiences.Quotes and Insights: Quotes from thought leaders like George Kuh reinforce the importance of patience and practice in assessment, with Baker noting that institutions must actively use assessment data to earn distinctions like the “Excellence in Assessment” designation.Takeaways:Apply the NILOA Transparency Framework to your institution’s assessment practices.Enhance communication around assessment data, ensuring it’s accessible to diverse audiences.Prioritize using assessment data to drive improvement and demonstrate educational value.Involve students as active participants, emphasizing their feedback and impact on assessment outcomes.This presentation provides a powerful call for institutions to rethink assessment as a tool for empowerment, transparency, and true educational impact.

In this episode, Kristen DiCerbo, Chief Learning Officer at Khan Academy, provides a deep dive into Khan Academy’s research-backed approach to improving student learning outcomes. From understanding the importance of usage in K-12 learning to unveiling the potential of AI-powered tools, DiCerbo shares insights on the methodologies driving Khan Academy’s commitment to educational efficacy and equity.Key themes include:Efficacy in Action: DiCerbo details how Khan Academy measures impact, emphasizing that even 30 minutes of weekly engagement can lead to significant learning gains. She explains that these benefits span across demographics, including gender, ethnicity, and socioeconomic status, though there are opportunities to enhance support for English language learners.Mastery Learning: Khan Academy champions deep learning over surface-level coverage, using the “Skills to Proficient” metric to track skill mastery, which closely correlates with external assessments. DiCerbo argues for focusing on proficiency in fewer skills rather than broad, superficial coverage to reinforce long-term retention.AI-Powered Learning with Conmigo: Khan Academy’s AI tutor, Conmigo, helps students in math, science, and writing by guiding them through problem-solving steps without giving direct answers, fostering active learning and “productive struggle.” Available in multiple languages, Conmigo also supports English language learners and includes transparency features for teachers to track student-AI interactions.Supporting Student Agency and Ethics in AI: DiCerbo discusses how AI tools can empower students to become active learners. She also emphasizes the ethical considerations of AI in education and advocates for community-driven conversations around AI literacy to responsibly integrate these technologies in learning environments.Looking ahead, Khan Academy is refining Conmigo’s design to better balance productive struggle with student frustration, ensuring learning remains both challenging and supportive. This episode highlights Khan Academy’s forward-thinking use of AI and its dedication to creating data-informed, ethical solutions for students and teachers. Join us to explore how technology is reshaping education and the future of learning.

In this episode, Christina Supe examines how AI is reshaping writing, literacy, and authorship. She opens by highlighting literacy’s longstanding link to power and social mobility, emphasizing that AI tools might disrupt this by enabling “cognitive outsourcing.” This trend could risk intellectual stagnation and perpetuate existing power imbalances by reducing opportunities for critical thinking.The discussion challenges traditional definitions of a "writer," exploring AI as an agent that creates text and forces us to rethink what qualifies as writing and authorship. Supe raises ethical concerns, particularly about when AI assistance is appropriate. She emphasizes the need for clear distinctions between tasks requiring authentic human input and those that may reasonably involve AI. Copyright laws are evolving to address AI’s role in content creation, with recent changes from the US Copyright Office now allowing copyright for works with minimal AI input, signaling the need for defined policies.Key quotes from Supe underline literacy as a means to access power, warning against the dangers of “outsourcing” cognitive skills to machines. She discusses the value of "reasonable reluctance" toward AI, as voiced by educators, who fear AI could undermine critical thinking and erode essential skills in close reading and analysis. AI, she argues, poses risks as a "shortcut" that may diminish learning and creativity if used inappropriately.For educational institutions, the discussion underscores the need to:Establish Clear AI Policies: Clearly outline when and how AI can be ethically used in student work to guard against plagiarism and intellectual shortcuts.Foster Critical Thinking: Encourage original thought and close engagement with materials to help students retain control over their intellectual growth.Educate on AI’s Role: Help students understand AI’s limitations and ethical considerations in writing.Reevaluate Assessments: Develop assessments that prioritize originality and deeper engagement, ensuring AI use doesn’t overshadow learning objectives.Through Supe’s insights, this episode encourages proactive strategies to balance AI’s potential with a commitment to authentic learning and intellectual development.

In this podcast, Ryan Smith, Director of University Assessment at Illinois State University, discusses how data storytelling transforms overwhelming data in higher education into actionable insights that inspire meaningful change. He addresses the challenge of institutions collecting vast amounts of data but lacking the capacity to utilize it effectively. Ryan advocates shifting from viewing data as purely objective to embracing the storyteller's role in crafting impactful narratives.Main Themes:Data Overload vs. Utilization: Institutions are inundated with data they struggle to use effectively. Data storytelling offers a solution by turning complex datasets into clear, compelling narratives.The Role of the Storyteller: Acknowledging the storyteller's influence is essential for impactful assessment, moving beyond a strictly positivistic approach.Building Trust and Understanding Stakeholders: Successful data storytelling requires understanding diverse stakeholder needs, engaging non-technical audiences, and building trust through transparency and empathy.Key Points:Data Storytelling Process:Identify the Problem: Focus on the specific issue the data story addresses. ("Ask yourself what is the point...and if you can't answer that, don't do it." – Ryan Smith)Know Your Audience: Understand their background and data perception. ("Prioritize the top 3 and don't make them up." – Ann Emory)Define the Purpose: Decide if the visualization is for exploration, explanation, or both.Craft the Narrative: Use storytelling elements like setting, characters, conflict, and resolution.Types of Visualizations:Exploratory: Reader-driven dashboards for interactive insight discovery.Explanatory: Author-driven visualizations conveying specific narratives.Key Considerations:Build Trust: Be transparent about data limitations and engage stakeholders. ("People know us...they know what we're doing. So it's important to be confident and tell that story...and of course, to be genuine, too." – Ryan Smith)Emphasize Utility: Focus on stories highlighting areas for improvement. ("If you're not using assessment, why are you doing it?" – Ryan Smith)Address Challenges: Recognize different stakeholder interpretations and use storytelling to bridge perspectives.Key Quotes:"It's seeing something that doesn't exist already. It's taking a giant data set...and then making it into a decision and making it into a story." – Ryan Smith"The amount of data we have far exceeds our capacity to use it." – Ryan Smith"When you know information that can result in more effective student learning, and you don't do anything with it...shame on us!" – Daphne BernardActionable Takeaways:Initiate Stakeholder Conversations: Understand data needs and preferred storytelling methods.Start Small: Focus on a single, impactful data story addressing a key priority.Experiment with Visualizations: Use tools like Power BI and Tableau.Continuous Learning: Develop skills through resources and communities.Advocate for Cultural Shift: Move from data collection to data utilization via storytelling.

In this podcast, Dr. Daphne Bernard and Glenn Phillips explain how Howard University enhanced its assessment process by incorporating storytelling to improve student learning outcomes (SLOs) and competency attainment.Context:Howard University, a historically Black institution with a unique mission, faced an accreditation review. Traditional, compliance-focused assessment methods didn't fully capture its rich educational experiences. The COVID-19 pandemic added further challenges.Integrating Storytelling:They adopted Banks-Wallace's framework:Story: Providing meaningful context to events.Storying: Weaving events into coherent narratives.Storytelling: Engaging stakeholders by sharing these narratives.Implementation:Faculty Engagement: Faculty shared stories of curricular improvements and student successes.Student Involvement: An essay contest invited students to reflect on their learning experiences.Pandemic Adaptation: Documented remote learning challenges and adaptations.Outcomes:Enhanced Understanding: Combined quantitative data with personal narratives for a fuller picture of student learning.Increased Engagement: Personal stories fostered deeper investment from faculty and students.Mission Alignment: The approach resonated with Howard's cultural heritage and mission.Continuous Improvement: Stories informed enhancements in programs and services.Key Insights:Storytelling uncovers deeper insights into student learning.Engaging stakeholders as storytellers fosters ownership.Tailoring assessment to the institution's context enhances effectiveness.Balancing narratives with data leads to comprehensive understanding.Conclusion:By integrating storytelling into assessment, Howard University enriched its evaluation of student outcomes and aligned the process with its mission. This approach offers valuable insights for other institutions seeking to enhance their assessment practices in meaningful and culturally responsive ways.

"Leveling the Learning Landscape: An Outcomes-Driven Approach to Equity-Focused Curriculum Change"In this podcast, Stanford University's Center for Teaching and Learning discusses their "Leveling the Learning Landscape" project—a five-year initiative aimed at transforming the undergraduate curriculum to enhance equity, accessibility, and student learning outcomes.Key Themes:SMART Goals for Curriculum Transformation:Utilizing Specific, Measurable, Attainable, Relevant, Time-bound goals to ensure clear and effective curriculum changes that enhance skills and competencies.Equity-Focused Curriculum:Addressing challenges of equity and belonging by redesigning courses, creating new learning pathways, and fostering inclusive learning environments.Collaborative Process:Engaging faculty, staff, and students in interdisciplinary teams.Teams participate in the Curriculum Transformation Institute (CTI) to develop proposals using evidence-based practices and receive peer feedback.Continuous Assessment and Evaluation:Implementing strategies like placement exams, student feedback, and data analysis to measure the impact of curriculum changes and tailor them to student needs.Examples of Initiatives:Center for Comparative Studies in Race and Ethnicity (CCSRE):Developing a new introductory curriculum that emphasizes student feedback and fosters a sense of belonging.Statistics Team:Creating new courses with better pacing and providing a free online textbook to enhance accessibility.Physics Team:Using placement diagnostic exams to assess prior knowledge and promote equity by guiding students to appropriate courses.Inclusive Field Education Team:Enhancing inclusive practices through class observations and providing feedback to instructors.Challenges and Solutions:Faculty Workload and Resources:Addressed by offering seed grants, potential course releases, and support from data coaches to sustain faculty engagement.Navigating Institutional Processes:The CTI assists teams in understanding and navigating complex approval procedures for curricular changes.Data Access and Privacy:Balancing the need for data to understand student experiences with privacy concerns through collaboration with institutional research and adherence to policies.Key Insights:Effectiveness of SMART Goals:Clear and measurable objectives lead to more effective curriculum changes and improved competency attainment.Student Partnerships Enhance Equity:Involving students in the assessment process aligns the curriculum with their needs and promotes a sense of belonging.Continuous Assessment Drives Improvement:Ongoing evaluation allows for curriculum adjustments that better meet student needs.

Utilizing Course Outcomes for Continuous ImprovementSource: Excerpts from a presentation by Dr. Will Miller, Associate VP for Continuous Improvement at Embry-Riddle Aeronautical University, on leveraging course-level outcomes for student success.Main Themes:The Unit Disconnect: Miller addresses the disconnect between program-level and faculty-level assessment views, resulting in fragmented learning for students. Misaligned courses and inconsistent objectives hinder students from synthesizing knowledge effectively.Quote: "The unit disconnect concerns me... It leads to fragmented learning."Holistic Assessment: Advocates for assessing the entire student experience, including academics and co-curriculars. Emphasizes aligning objectives and recognizing experiential learning.Quote: "We want assessment at Riddle to be holistic... look at the entire student experience."Individual Student Tracking: Highlights the need for tracking student learning at the course level to enable personalized learning, targeted remediation, and proactive support.Quote: "We need faculty on board with tracking individual student learning outcome achievement."Data-Driven Decision Making: Emphasizes using data to improve areas like curriculum mapping, prerequisites, instructor effectiveness, and course scheduling. Outcome data helps identify gaps and redundancies.Quote: "It should lead to better decisions."Curriculum Mapping: Encourages a dynamic, data-driven approach to curriculum mapping to gain real-time insights into student learning.Quote: "We're moving towards a much more dynamic, data-driven process."Importance of Culture and Collaboration: Stresses the need for institutional commitment, data literacy, and collaboration among faculty, advisors, and administrators.Quote: "We need the data literacy. We need collaboration."Key Ideas/Facts:Granular Data Matters: Analyzing individual outcomes reveals important patterns and correlations.Predicting Success: Understanding crucial outcomes can help better prepare students for future courses.Data Improves Teaching: Outcome data identifies effective teaching practices and supports personalized learning.Ethical Use of Data: Emphasizes privacy, transparency, and avoiding punitive use of data.Change Requires Commitment: A data-driven approach needs institutional buy-in and faculty development.Selected Quotes:On the Disconnect Between Grades and Learning: "Success isn't measured by hoops or clock hours. We have to really pick apart these subjects."On Data to Inform Decisions: "I love giving advisors access to data because parents and students respond when shown the trends."On Transparency: "Please tell students the course outcomes. If it helps them take it seriously, it's a win for everyone."Overall: Miller advocates for shifting from a grade-centric approach to a holistic, data-driven model prioritizing individual learning outcomes.

Key Takeaways1. Experiential Learning (EL) as an Ecosystem:EL is a dynamic ecosystem involving students, faculty, administrators, employers, and community members. Each stakeholder has unique perspectives and expectations, creating a complex environment where definitions and approaches to EL vary. 2. The Multifaceted Nature of EL:EL encompasses diverse practices, from internships to cross-disciplinary projects. Given this diversity, acknowledging the varied approaches and epistemologies of EL across disciplines and institutional levels is essential for developing inclusive and effective assessment methods.3. Assessing the Full Cycle of EL:Effective EL assessment requires going beyond traditional grading to evaluate the full experiential learning cycle, including engagement, reflection, skill development, and application. 4. Data as a Powerful Tool:Thoughtful data collection and analysis are critical for understanding EL’s impact on student learning outcomes and institutional effectiveness. Detailed ReviewUnderstanding the EL Ecosystem:The EL ecosystem is complex, involving various stakeholders with distinct perspectives on EL. Heinrich emphasizes the importance of recognizing these differences for effective communication and assessment, as different stakeholders may interpret "experiential learning" in unique ways.Institutions are increasingly expected to demonstrate the value of their programs to students, families, employers, and legislators. EL is seen as a driver of student success and institutional accountability, aligning with the goals of higher education.Exploring EL Pedagogy:EL is grounded in theories like Kolb’s Experiential Learning Cycle, which emphasizes four stages: concrete experience, reflective observation, abstract conceptualization, and active experimentation.EL engages multiple parts of the brain, enhancing retention and learning. This approach aligns with brain-based design principles, which support active and reflective learning.The session includes examples from JMU X-Labs, an innovative, cross-disciplinary learning environment, where students’ reflections provide insights into their learning experiences, offering valuable qualitative data for assessment.Creating a Data Model for EL:The presenters introduce a data model linking Kolb’s cycle stages to specific student outcomes:Concrete Experience: Fosters student engagement, belonging, and retention.Reflective Observation: Develops self-awareness, empathy, and holistic growth.Abstract Conceptualization: Leads to knowledge acquisition and academic success.Active Experimentation: Builds career readiness and applied skills.Addressing Practical Challenges in EL Assessment:The session addresses the challenges of managing extensive data generated by EL assessments. Heinrich notes the importance of aligning EL assessment with disciplinary standards and institutional priorities. Effective EL assessment requires collaboration and communication across departments. Heinrich encourages institutions to consider their “ecosystem” and find new stakeholders who can benefit from EL data.Call to ActionEmbrace the complexity of the EL ecosystem by considering stakeholder perspectives.Develop comprehensive assessment strategies to evaluate EL practices.

Most Important Ideas:Questioning the Credit Hour: Dr. Blondin argues that while the credit hour is convenient for administrative purposes, it fails to measure genuine learning. She urges institutions to explore alternative methods that better reflect student competencies. "I've been associated with that battle for over 30 years… The credit hour drives financial aid, it drives transfer… but are we truly seeing results? Are we bearing fruit?"Prior Learning Assessment: Recognizing and valuing the knowledge students bring from previous experiences—whether through formal education, work, or life—is essential for equity and student success. "I urge you to help develop and expand policies that acknowledge prior learning. I'm constantly amazed by what our students already know."Stakeholder Involvement: Faculty, students, and employers should collaborate in defining and assessing student learning outcomes, ensuring relevance to the workforce and gaining buy-in from all involved. "What if this advisory group helped develop outcomes? Or, dare I say, students helped shape the outcomes for a program or course."Seamless Transition Between Credit and Non-Credit: Dr. Blondin advocates for removing barriers between credit and non-credit programs, emphasizing that learning occurs in multiple forms and settings. "I envision a permeable boundary between credit and non-credit, allowing everything to flow together seamlessly. That’s my dream, and I have a year and a half to make it happen."Flexibility in Learning: Acknowledging that students learn at different paces and in varied ways, institutions must offer flexibility in how, when, and where learning occurs. This includes embracing online learning, competency-based education, and other innovative approaches. "Flexibility in where, when, and how learning happens is a mindset shift for many—administrators, faculty, and staff alike."Data-Driven Decision Making: Analyzing data from student learning assessments can reveal areas for improvement, inform program development, and guide resource allocation to support student success. "We've held data summits to share and learn, and we use this data to inform our budget. For instance, if students aren’t achieving outcomes in developmental math, we ask, ‘What can we do to help?’"Key Quotes:"It's not just our students who are stakeholders… some employers have lost confidence in our ability and are deciding to train in-house instead.""No one ever, on the first day of any job, handed me a blue book.""I've been in a lifelong battle with the credit hour and the Carnegie unit..."Conclusion:This discussion emphasizes the need for a paradigm shift in higher education assessment, moving from outdated, rigid models to more flexible, learner-centered approaches. By prioritizing student intent, accommodating diverse learning styles, and collaborating with key stakeholders, institutions can create a more equitable and effective learning environment that better prepares students for success in both the workforce and life.