The government's use of machine learning to draft NDIS participant plans has sparked debate, with experts weighing in on its benefits and potential pitfalls. Documents reveal that the National Disability Insurance Agency (NDIA) has been utilizing Microsoft's Copilot AI in a trial, alongside traditional machine learning algorithms, to streamline the plan-making process. While the agency emphasizes that AI is not used for final decisions, the use of machine learning in budget planning has raised concerns about the balance between efficiency and individual needs.
The NDIA's AI policy document from April 2024 states that AI tools must not access participant records without explicit authorization. However, the briefing document for Senate estimates 2023-24 highlights the use of machine learning in draft budgets, where algorithms provide recommendations based on participant profiles, but final decisions are made by human delegates. This approach aims to speed up the initial analysis and improve service delivery.
The trial's report shows a 20% reduction in task completion times and a 90% satisfaction rating among staff, including those with hearing impairments who benefited from live transcription during meetings. However, concerns have been raised about the potential for automation bias, where planners might rely too heavily on AI recommendations, potentially neglecting individual needs and judgment.
Dr. Georgia Van Toorn, an expert in algorithmic decision-making, warns that machine learning may struggle with complexity and nuance. She emphasizes the importance of human discretion and judgment, especially in cases where individuals have unique needs that don't fit neatly into predefined categories. The 'black box' nature of machine learning, where data points and biases are not always transparent, adds to the complexity.
Dr. Stevie Lang Howson, an NDIS participant and advocate, expresses concern about whether staff have the training and time to create plans that truly reflect individual needs. The impact of these plans on people's daily lives, from bathroom access to wheelchair suitability, demands careful consideration and transparency.
Despite the benefits, the NDIA's use of AI in NDIS participant plans has sparked a broader conversation about the role of technology in public services. The federal government's recent AI plan aims to provide public servants with access to generative AI tools, but it also underscores the need for responsible and ethical implementation, ensuring that human discretion remains a crucial component of decision-making processes.
