Artificial intelligence (AI) models are being applied more often across a range of biomedical domains to support clinical decision-making and therapeutic strategies. This Review examines the transparency of medical AI systems, highlighting key approaches to increasing transparency in model design, operation and outcomes.