There is also the issue of Language models replicating the biases and issues of our current society, which should disqualify it as a tool to further societal progress.
Neural networks can optimize and itterate (to a degree), but not invent completely new things.
(Sorry for the rant) #NoAI