> For the complete documentation index, see [llms.txt](https://www.raoulharris.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.raoulharris.com/technical-courses/deeplearning.ai-short-courses.md).

# DeepLearning.AI short courses

## Improving the accuracy of LLM applications

Practical examples in this course were based on using Llama 3 with the Lamini library.

* Start with rigorous evaluation and iterating the prompt
* If adjusting the prompt isn't enough then try fine-tuning
* Fine-tuning often doesn't require much data
* Parameter-efficient fine-tuning can be very cheap
* Memory tuning allows you to embed specific facts directly into the model
  * I haven't independently researched whether this is actually useful or just Lamini marketing
* Evaluation dataset
  * Start small
  * Quality > Quantity
  * Focus on the areas that it does poorly on
  * Try to find the easiest examples that still fail
  * Try to break the process to identify issues
  * Set an accuracy target
  * Iterate the dataset as performance improves
* Scoring using LLMs
  * You would ideally use a deterministic approach instead if practical
  * Ask for a numerical score
  * Use a structured output to enforce this
  * Can you provide a reference answer for it to score against?
* Rolling your own fine-tuning can be hard, so consider managed fine-tuning
  * Inefficient implementations
  * Idle compute due to not being able to parallelize efficiently
  * Crashes
* Consider using LLMs to help create fine-tuning datasets