Imagine they built chess models, and accused a rival of stealing their model's "chess reasoning" by studying its output.
That would suggest the model was trained on a large number of chess games and is just predicting which move's most likely to come next. Which, BTW, is exactly how LLMs play chess