Getting Started with Local LLMs
Chapter 1: Introduction to Large Language Models
What Is a Large Language Model (LLM)?
A Simple View of How LLMs Work
Understanding Tokens and Text Generation
Why Run LLMs Locally?
Local vs. Cloud-Based LLMs
Quiz for Chapter 1
Chapter 2: Preparing Your Local Environment
Hardware Considerations: CPU
Hardware Considerations: RAM
Hardware Considerations: GPU and VRAM
Checking Your System Specifications
Operating System Compatibility
Installing Python (Optional but Recommended)
Introduction to the Command Line / Terminal
Quiz for Chapter 2
Chapter 3: Finding and Selecting Local LLMs
Where to Find LLM Models: Hugging Face Hub
Understanding Model Sizes and Parameters
Model Formats: GGUF and Others
Quantization: Making Models Smaller
Reading Model Cards for Information
Model Licenses and Usage Restrictions
Choosing Your First Model
Quiz for Chapter 3
Chapter 4: Running Your First Local LLM
Introduction to Local LLM Runners
Setting up Ollama
Downloading a Model with Ollama
Running a Model with Ollama (Command Line)
Setting up LM Studio
Finding and Downloading Models in LM Studio
Loading and Chatting with a Model in LM Studio
Introduction to llama.cpp (Concept)
Hands-on Practical: Running a Model
Quiz for Chapter 4
Chapter 5: Basic Interaction and Prompting
What is a Prompt?
Your First Prompt: Simple Questions
Giving Instructions
Understanding Context Window
Basic Prompt Formatting Tips
Temperature and Creativity
Common Interaction Patterns
Practice: Simple Prompting Techniques
Quiz for Chapter 5
Reading Model Cards for Information
Was this section helpful?
- Model Cards for Model Reporting, Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru, 2019 Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*) (Association for Computing Machinery) DOI: 10.1145/3287560.3287596 - Introduces the concept of model cards as a standardized reporting framework to promote transparency and responsible AI development.
- Model cards, Hugging Face, 2024 - Official documentation explaining what model cards are on the Hugging Face Hub and how to create and interpret them.
- HELM: Holistic Evaluation of Language Models, Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda, 2023 Transactions on Machine Learning Research (TMLR) DOI: 10.48550/arXiv.2211.09110 - Proposes a framework for comprehensive evaluation of language models across a broad range of metrics, offering insights beyond simple benchmark scores mentioned in model cards.