back at the end of 2020, before AI took over the world, i laboriously finetuned a GPT-2 model. it was the first public LLM, essentially useless but able to emit curiously coherent text compared to previous methods.
i used tweets by several people whose posting i admired, plus a couple other things