Health systems evaluating AI must look beyond datasets and algorithms to understand the assumptions and priorities embedded within the applications themselves.