Reposted by Michael Carbin
What’s the most effective way to add new domain knowledge into an open LLM? A new blog post from my team covers experiments we did at the beginning of the year to start answering this question. It starts, unsurprisingly, with sweeping your learning rate… www.databricks.com/blog/charact...
databricks.com
Characterizing Datasets and Building Better Models with Continued Pre-Training
