link.springer.com
Design versus reality: assessing the results and compliance of algorithmic impact assessments - Digital Society
Algorithmic impact assessments (AIAs) have become a dominant regulatory instrument in governing artificial intelligence (AI). While there are noteworthy examples across the global north, Canada’s AIA is considered to be the best practice worldwide. When AIAs are studied, evaluations have been based on the assessment of the instrument and not the examination of their answers. We examine Canada’s published AIAs. We report five findings: (1) Uneven compliance is observed in the completion of AIAs; (2) Reasons for automation legitimize efficiency and innovation narratives; (3) Impacts and trade-offs are framed as non-existent, positive and undermine harms; (4) Civil society organizations are non-existent in AIAs; and (5) Accountability is framed as processual mitigation of AI impacts. Despite the promise of AIAs for accountability of AI systems, our results reveal a “design-reality” gap between literature and practice. We observed that any negative impacts were framed positively; input was not elicited from the public; and an over-emphasis of self-regulation conformed to organizational procedures instead of investigating outcomes. Although submission is mandatory, its processual accountability failed to ensure compliance. We recommend strengthening accountability to include civil society, formalizing harms instead of emphasizing impacts or risks and blending processual accountability with outcomes of AI systems.
