Building The Case For AI-Powered Parliaments & The Necessary Guardrails: Houses Of The Oireachtas
Author: Ciarán Doyle, Assistant Secretary and CIO of the Oireachtas of Ireland. Written on September, 2024
This publication is part of the book “Artificial Intelligence in Legislative Services: Principles for Effective Implementation”. To download the entire book, use the button below:
Introduction (Building The Case)
This paper provides a comprehensive summary of the ICT teams work to date on assessing the possibilities of AI in Parliaments, specifically within the Houses of the Oireachtas. It outlines their journey from initial research and familiarisation with Generative AI to the development of use cases and the formulation of a responsible AI framework and overarching AI strategy. This document aims to build the case for AI-powered Parliaments while emphasising the need to develop and establish the necessary guardrails to ensure responsible and effective implementation.
Since the last quarter of 2022 to present day (Q3, 2024) the Houses of the Oireachtas has been engaged in a multi-step process of assessing and evaluating the potential use of Generative AI. From Q4 2022 to end Q1 2023 that process involved early familiarisation and self-driven research in the area, as it became a prominent topic globally. Dedicated members of the Houses of the Oireachtas ICT team engaged with Generative AI content across all communication mediums such as research publications, news articles, lectures and talks on content streaming platforms and Government publications. This early emersion in the topic allowed team members to be well informed on the topic and key aspects within and across it to be able to assess it and its potential relative to our data and processes.
Informed by this self-driven research the Houses of the Oireachtas ICT team, across late Q1 and Q2 2023, then began to ideate use cases themselves while also engaging directly with ICT partners such as Microsoft and Accenture. With Microsoft the team looked at possible early use cases for natural language processing search and content summarisation on public data sets such as Parliamentary Questions. They also spent time understanding the defining the differences between services such as Azure OpenAI and the emerging Copilot which would become generally available in Q4 2023. With Accenture the team took the internal ideation that had been progressed around 30 possible use cases to develop them out more fully with assistance from the Accenture AI team and broadly categorise them.
Just as the self-driven research stage had developed the team’s ability to engage at a technical and use case level with ICT partners, the partner engagement then led the team to be able to begin to develop a briefing and engagement plan with key stakeholders. Across Q3 2023 meetings and workshops were held off-site with key staff working in wider digital transformation areas and the management board of the Houses of the Oireachtas. The purpose of the workshops was to distil all the information gathered and potential applicability of Generative AI into understandable concepts and relatable use cases for the organisation. To do that the team prepared a briefing and prototype use case demonstration session covering a wide range of topics.
Stakeholders were introduced to foundational knowledge such as understanding artificial intelligence’s spectrum from Narrow/Weak commonly understood ubiquitous forms to newer emergent Generative forms. The focus then turned to Ireland’s national landscape by examining how various Government departments and agencies are examining and taking early steps towards potentially adopting this general-purpose technology. From there the briefing then moved towards peers and what international counterparts in other Parliaments were doing or planning to do, to gain a broader perspective. Key national publications such as “AI Here For Good: National Artificial Intelligence Strategy For Ireland” were reviewed alongside critical cybersecurity guidance “Cyber Security Guidance on Generative AI for Public Sector Bodies” tailored for public sector bodies from Ireland’s National Cyber Security Centre.
Furthermore, the group were then briefed on the key framework components of the coming EU AIA (Artificial Intelligence Act), which would later be published in Q1 2024, and its implications for organisations looking to adopt and integrate Generative AI into work processes and service delivery. These were important discussions to prompt and engage in questions and answers around with the group before moving onto demonstrations of and discussions on use cases or proofs of concept. This attention to detail on guidance and proving of a firm grasp on coming regulation clearly illustrated that the team progressing the case for Generative AI knew that balance between embracing innovation and meticulously considering governance & risk management were both crucial for advancement.
With foundational concepts and the importance of regulation and governance covered it was then possible to brief stakeholders on practical Generative AI examples. The team started in the area of unintended consequences and the risks posed to content and information (video) generated by Parliaments which could be manipulated by increasingly real deepfakes that have emerged from Generative AI technologies. From there however the practical side of Generative AI was equally emphasised, through demonstrations from the private sector, showcasing Generative AI applications in the area of call centre assistant agents and content creation tools already in use by advertising teams of global consumer brands to exemplify real-world utility.
This set the stage to turn to early internal prototype demonstrations, which had been developed in consultation with Microsoft and Accenture to showcase how Generative AI could transform internal processes. Specifically, it was demonstrated how Generative AI could dramatically change user experience and engagement with Parliamentary Question data through contextual search. Additionally, the team demonstrated how the manual process of generating highly detailed summaries from Bills & Bill Memos in the form of Bills Digests could be recreated by training Generative AI.
The comprehensive nature of the briefing and workshop demonstrations enabled the team to then propose documenting the research and exploration to date into a next steps paper for the management board which would propose an actionable path forward. The paper was developed across Q4 2023 and presented and approved in Q1 2024. In it 6 of the original 30 use cases which had been developed were outlined in more detail with a proposal to advance 2 put forward. Estimates and a timeline to end Q2 2024 were agreed to. In July 2024 an update and further steps paper was then presented to management board. This paper outlined the work of the team involved in developing the use cases, the architecture that was designed and a recommendation of how the work should continue. Before outlining the next steps, the Houses of the Oireachtas will take across the remainder of 2024 a summary of these 2 use cases follows:
Use Case 1 – Contextual Search & Summarisation
Use Case 1 was built for the Library and Research Services (L&RS) to assist them in carrying out their research activities. The application has two separate uses:
Contextual Search
The user uploads any number of documents, which are generally research papers in PDF format relating to a particular topic of research. The user can then use a chat interface (similar to ChatGPT) to ask questions about the research topic. The application provides a response to the query based solely on the information within the uploaded documents, along with in-text citations referencing the source(s) of the response. The application is a method for the researcher to search across multiple documents with one natural language query, receiving a comprehensive response which synthesises information from multiple sources to give an answer. This saves the user the time that would otherwise be required to manually search through the documents for content relating to their query and compile an answer.
Summarisation
The user can use this tool to generate a summary of an uploaded document, such as a research paper that the individual has written. The summary generated can be in one of three style options: Blog Post, Executive Summary, or Other (user specifies their requirements using text). The application leverages GenAI to provide a comprehensive summary of the document. The user can also input additional instructions to the application if they want to further refine the summary to their preferences.
Iterative feedback and development process for use case 1
For use case 1, user feedback was received in the form of a template. This allowed the SMEs to submit feedback on any response generated by the model and facilitated iterative improvements to the application over the proof-of-concept timeline. The users were asked to provide scores for correctness, completeness, conciseness, and an overall score for each response, along with a free text response to explain the reasoning for the scores provided. Once feedback was gathered and stored, a feedback session was coordinated to talk through the feedback. The feedback was then used to form actionable changes included in a change log. Changes were prioritised and the developers implemented a feasible number of changes before generating new outputs and starting the feedback process again. Any changes that could not be implemented given the time constraints were included in a backlog.
The application delivered to the testers also included the option to provide feedback directly for each response. This has meant that ongoing feedback can continue to be gathered, which in future can be used to make further improvements to the application. In addition, this use case application was shared with the Office of the Parliamentary Legal Advisor to determine whether it may be useful to researchers in other units and the response to date has been positive.
The application has been welcomed by the business team who have been working on this project. In particular, they see a benefit in being able to summarise research papers that they have written themselves. Suggestions have been made as to how the application could be improved further and this can be scheduled in parallel with the other steps to be taken on Generative AI.
Use Case 2 – Automated Report Generation
Use Case 2 was built for the Digital Team to assist them in proving the concept of automatically creating the first draft of a Parliamentary journal. It should be noted that the scope of this use case did not include business items which are not taken in the House and coming from the Clerk Sheet.
Iterative feedback and development process for use case 2
A combination of Traditional AI (OCR - Optical Character Recognition) along with Generative AI was used to solve this business problem. The focus was building automated Parliamentary journals. These are very text-heavy reports that require data from multiple sources, including both digital and non-digital (handwritten) documents. The first input file was handwritten Clerk Sheet, in a mix of Irish and English. Multiple people write on the same Clerk Sheet throughout the day, so there is a high variance in the handwriting. The sheet contains information that is not available elsewhere, such as the start and end times for each topic that was discussed in the Dáil on a particular day.
The first step of the solution leveraged the OCR technology to create a digital, comma separated value (CSV) version of the handwritten sheet - which was the first output for the user. The second step utilised this OCR’d Clerk Sheet, along with blank templates of the journals and completed journals (to use as reference material) which were provided by the Digital Team. The AI then filled in the blank journal template using the data from the OCR’d Clerk Sheet by leveraging the completed journals as a reference.
The third step involved the AI application connecting to the internet (www.oireachtas.ie) to fetch the debate transcripts related to the relevant sitting day so that the information could be included in the Clerk sheet. As the verbatim transcripts can be very long, spanning many pages of text, the AI revises its initial draft report from the previous step, augmenting information into each of the sections where required (again, utilizing completed journals as reference). Essentially the AI takes relevant information from the Debate, where required, based on the reference journals. This augmented information includes resolutions on topics discussed on that date (which are crucial to these Parliamentary journals).
For the final step a layer of formatting is introduced into the process. After the AI application has processed and collated the corresponding data from the handwritten Clerk Sheet (via OCR) and official report (from Oireachtas.ie), using historical journals as reference guides to fill in the blank template, it then goes through a two-stage formatting process. This is because the first output of the draft journal is in plain text, without any formatting, as large language models (LLMs) only work in simple text outputs. To address this and deliver the formatted output required, HTML (Hypertext Markup Language) coded versions of the templates, which include HTML <tags> that define the bolding, text sizing, positioning etc., were developed. The HTML code could then be rendered to produce a formatted version of the report as a .html document, and finally converted to a Word document.
The digital team has engaged with the GenAI developers on this application as the Journal Office had flagged that they would not have sufficient availability to engage. The output shows good potential in terms of how the output looks and the ability to work from a scanned handwritten Clerk Sheet. This is a complex output in that different business items require different elements from the debate to be inserted into the Journal. Arising from this, a number of further issues to be tackled and addressed have been identified and work will continue on this use case to generate the best possible draft Journal for checking. When this work is completed, it is intended that the Journal Office and Seanad Office will be shown the output and further assessment can be conducted.
Next Steps (The Necessary Guardrails)
Having constructed the case for the potential use of AI in Parliament and then receiving approval to demonstrate that potential through early proof of concept use cases it is now vital to focus on developing the necessary guardrails in the form of a responsible AI framework and overall AI strategy that will be required before this technology can be used to transform any processes or services. This was the key next step recommendation made by the ICT team to the management board of the Houses of the Oireachtas in July 2024.
Responsible AI
The ICT team stressed that in the use cases to date, the applications had been assessed using Accenture’s Responsible AI framework until the Oireachtas could conduct the work to build out an internal Responsible AI Framework. It was recommended and approved that across Q’s 3 & 4 of 2024 the ICT team will hold a series of workshops to develop a Responsible AI Framework which can be utilised for all applications developed for live use within the Oireachtas. A number of key stakeholders across the Oireachtas will be involved in this process, including the Data Protection Officer, Chief Risk Officer and Office of the Parliamentary Legal Advisor. It will also be essential to ensure that the EU Artificial Intelligence Act and the associated guidance is a foundational element of the Responsible AI Framework for the Oireachtas.
Early planning work on the Responsible AI framework workshops has commenced and across the remaining 4 months of 2024 the ICT team and stakeholders will work to assess and develop a framework around these 8 key risk dimensions.
Soundness: Ensuring that AI systems are reliable and perform as intended, minimising errors and inaccuracies.
Transparency: Making AI processes and decisions understandable and accessible to stakeholders, fostering trust and accountability.
Privacy: Protecting personal data and ensuring that AI applications comply with data protection regulations.
Sustainability: Developing AI solutions that are environmentally sustainable and resource-efficient.
Fairness: Ensuring that AI systems are unbiased and equitable, preventing discrimination and promoting inclusivity.
Robustness: Building AI systems that are resilient to adversarial attacks and can maintain performance under varying conditions.
Accountability: Establishing clear lines of responsibility for AI decisions and actions, ensuring that there is oversight and governance.
Liability & Compliance: Ensuring that AI applications adhere to legal standards and that there are mechanisms in place for addressing any legal issues that arise.
Overarching Strategy
Sitting above the Responsible AI Framework will be the overall AI Strategy. The AI Strategy work will be coupled along with the engagements around the Responsible AI Framework. The full set of actions across both (research, cross-directorate workshops, skills gap analysis, AI maturity assessment, development of value assessments and metrics)are designed to generate the required outputs (vision statement, principles, values, roadmap, resource plan, training and change management plan, governance operating model and budget estimate) which will then be brought to the Management Board for review, feedback, hopefully approval, and then long term implementation from Q1 2025 onwards.
To conclude, for now, it is the ICT teams hope that upon the successful completion of the Responsible AI Framework and the overarching AI Strategy, approval will be granted for the development of a comprehensive three-year programme of AI development work. This programme’s aim will be to further integrate AI technologies into our Parliaments processes, while ensuring that innovation is balanced with robust governance and risk management.