Practical Deep Learning
Homepage for the Practical Deep Learning workshop at Cornell Tech in Spring 2024.
Instructor: Jack Morris
Meetings: MONDAY 5:55pm-7:10pm (Bloomberg 091)
Office hours: By appointment only (for now)
Course description
Students will learn deep neural network fundamentals and practical skills for training and evaluating these networks. We will practice using a variety of tools in real-world scenarios to prepare students for practical deep learning problems seen in both research and industry.
Schedule
Lecture
| Week | Date | Title | Slides | Puzzle |
|---|---|---|---|---|
| 1 | 1/22 | Introduction: debugging neural networks | 📊 | 🧩 |
| 2 | 1/29 | Frameworks and Transfer Learning | 📊 | 🧩 |
| 3 | 2/5 | Language modeling / How chatGPT works | 🧩 | |
| 4 | 2/12 | Attention | 📊 | 🧩 |
| 5 | 2/19 | Multimodal models | ||
| 6 | 3/4 | Information retrieval, vector databases, and inversion | 🧩 | |
| 7 | 3/11 | LLM Quantization & LoRA | ||
| 8 | 3/18 | Advanced Prompting |
Project
All students will complete a small project at the end of the semester to test their new deep learning skills. The goal is to build something that uses contemporary technology to do something interesting and exciting and to showcase your skills as an engineer.
For more information on the project, see the Project page.
Project working sessions
| Date | Â | Â |
|---|---|---|
| 3/25 | Â | Â |
| 4/8 | Â | Â |
| 4/15 | Â | Â |
| 4/22 | Â | Â |
Resources
- Recipe for Training Neural Networks - Andrej Karpathy
- Frequently Asked Questions – PyTorch Team (actually a very good resource for beginners!!)
- GPT in 60 Lines of NumPy
- Intro to Text Embeddings (Stack Overflow) - a decent introduction to text embeddings