AI Governance Public Sector Challenges

Discussion with AI governance experts analyzed Bill 194’s public sector implications with the need for explainable AI and skill development. Bill 194 and Public Sector Analysis of Bill 194 indicated that the public sector accepted the regulations but needs resources to adhere to them, emphasizing the elevated need for explainable AI. The legislation places pressure on teams to understand model operations and manage shadow AI sprawl within public sector operations. Addressing Skill Gaps and Governance The massive skill shortage, especially in the public sector, requires shifting AI from an IT issue to a governance concern that affects insurance and council matters. Small municipalities should map existing processes to identify automation opportunities and consider collaboration with other small municipalities to sustain AI models. Policy Integration and Hyperscaler Influence Integrating AI elements into existing conduct and privacy policies is more beneficial than creating standalone AI policies, supported by investing in AI drills and tabletop exercises. Concern exists that large hyperscalers are developing the AI constitution, which may dictate future government AI policies to support their products and economic interests. Details ● Introduction and Context Setting: Daphne Thomson and Ali Abbas began the meeting with an informal check-in, noting that Ali made time for the meeting despite travel plans [i]. Daphne explained their work involves implementing practical AI for local governments, focusing on the practical aspects and high-level training [i]. They intended to generate additional content and information for these governmental bodies through their discussion [i]. ● Discussion Agenda and Scope: Daphne outlined the agenda, which included discussing Bill 194 and its differences and implications for Ontario, Quebec, and other provinces [i, j]. Other topics included looking at privacy legislation changes in Alberta, the importance of policy, and the ongoing conversation surrounding an AI constitution [i, j]. The plan was to have a 20-minute, informal chat about what is happening in AI within the governance and security domains [i, j]. ● Speaker Introductions: Ali Abbas confirmed that they prefer to be called Ali Herji and suggested their role be kept specific to teaching masters-level AI courses at UFRE [i]. Daphne introduced Ali as a former professor from their MBA program in artificial intelligence, emphasizing Ali's expertise and the positive impact of their classes [i, j]. Ali expressed appreciation for the introduction, noting that the engagement with Daphne in class was a pleasure and that they learned a lot from their submissions [i]. ● Analysis of Bill 194 and its Impact: Ali explained that provincial legislation regarding AI, such as Bill 194, is not unique to Ontario or Quebec, as every place touched by digitalization is discussing policies for AI control [i, j]. They noted that the bill is specific to the public sector, and the sector's response was not resistance, but rather a request for the necessary resources to adhere to the regulations [i, j]. They advised that the need for human and specialist resources in AI will increase as entities recognize the need for positions like a Chief AI Officer [i]. ● The Pressure of Explainable AI and Sprawl: Bill 194 elevates the need for explainable AI, requiring a strong grip on how models operate and transparency regarding their generative outputs and decisions [i]. This legislation places pressure on executive teams and engineers to understand their models and may lead to a rethink of some AI deployments, moving away from viewing AI as a "black box" [i]. Furthermore, organizations must develop a clear declaration of where AI is being used in public sector operations, anticipating the growing challenge of controlling "shadow AI" and AI sprawl [i, j]. ● Challenges of Skill Shortage and Governance in Public Sector AI: Daphne observed that while new expertise is needed to implement governance frameworks, there is currently a massive skill shortage, particularly in the public sector where IT staff often focus only on cyber security and hardware [i, j]. AI is becoming a governance issue that extends beyond the IT department to council, affecting aspects like insurance requirements for good governance [i, j]. The practical challenge lies in creating attainable, tiered governance frameworks so that small local governments can benefit from AI without being punished by overly complex requirements from large consulting firms [i]. ● Addressing Skill Gaps and Practical Implementation in Smaller Municipalities: Ali countered the idea of an AI professional shortage by stating there is a shortage of AI professionals who can articulate the return on investment and innovation, explaining that many can deploy "amazing things" but struggle to integrate them into workflows [i, j]. Ali advised small municipalities to start by completely reviewing and mapping out all existing processes, data flow diagrams, and standard operating procedures to identify automation opportunities [i, j]. It is critical to determine the true costs—including hosting and token costs—to ensure that AI deployment is financially sustainable, as simply using AI "for AI's sake" may not be worthwhile or cost-effective [i, j]. ● Strategy of Collaboration for Smaller Municipalities: Ali noted that a viable strategy for smaller municipalities facing financial constraints is collaboration with associations of other small municipalities, citing past examples of shared services like cyber insurance [i, j]. If smaller municipalities operate as "lone islands," their AI models may not be sustained in the long run unless there is substantial growth in their tax base [i, j]. Daphne agreed that immediate, small wins are possible within existing systems, such as the Microsoft environment, and caution must be used to avoid disastrous results from implementing AI without considering adoption and usage [i, j]. ● Integrating AI Policy into Existing Governance: Daphne suggested that instead of creating a standalone AI policy, it would be more beneficial to integrate AI elements into existing policies, such as those governing conduct, IT, and privacy [i, j]. This approach would help address issues like employees not taking responsibility for AI-generated outcomes (e.g., using ChatGPT or Co-Pilot without editing) and embedding AI responsibilities into established organizational workflows [i, j]. ● Importance of Practical Training and AI Drills: Ali agreed with the need for integrated policy and emphasized that organizations must invest in tabletop exercises and AI drills, similar to fire drills, to help teams understand the repercussions of unchecked AI output [i, j]. The goal of these engagement strategies is to immerse people in scenarios, making the training engaging and gamified, rather than relying solely on policy to prevent teams from bypassing security measures [i, j]. ● The Role of the AI Constitution and Hyperscalers: The discussion shifted to the AI constitution, which private sector companies like Perplexity and OpenAI are developing without government involvement [i]. Ali expressed concern that the influence of large hypersc...