Excited to share our production guide for building RAG-based LLM applications where we bridge the gap between OSS and closed-source LLMs.
- 💻 Develop a retrieval augmented generation (RAG) based LLM application from scratch.
- 🚀 Scale the major workloads (load, chunk, embed,
🔥 Putting ML in Production! We're going to publicly develop
@madewithml
's first ML service. Here is the broad curriculum:
- 📦 Product
- 🔢 Data
- 🤖 Modeling
- 📝 Scripting
- 🛠 API
- 🚀 Production
More details (lessons, task, etc.) here:
Thread 👇
Excited to release practicalAI -
- 📚 Notebooks on topics from basic Python to advanced deep learning techniques w/
@PyTorch
- 🖥️ Run everything using
@GoogleColab
or
@ProjectJupyter
- 📦 Learn object-oriented ML to code for products, not just tutorials
Excited to launch Made With ML - (
@madewithml
)!
🔍 Discover ML projects (with code/blog posts) on interesting topics.
🛠 Build projects of your own and share it with the community.
💻 Showcase your profile on your resume or apply directly to ML managers.
I updated the
@madewithml
pipeline lesson to include recent work on continual learning systems (algorithmic & MLOps workflow perspective).
🧵 Thread below on common themes, recent papers & critical decisions that prevent continual machine learning systems from being continuous.
Had a great time presenting on my approach (+ industry trends) for offline evaluation in
@chipro
's amazing CS329s Machine Learning Systems Design course
-
@MadeWithML
lesson (+ code):
- CS329s course:
🧵 Highlights thread below 👇
It's been nice to see small jumps in output quality in our RAG applications from chunking experiments, contextual preprocessing, prompt engineering, fine-tuned embeddings, lexical search, reranking, etc. but we just added Mixtral-8x7B-Instruct to the mix and we're seeing a 🤯
⚡️ Testing ML Systems: Code, Data & Models ⚡️
- Test types, coverage, best practices
- Pytest fixtures, markers, parametrize
- Test data via
@expectgreatdata
- Test models via slicing
@SnorkelAI
- Behavioral tests
- Testing vs. monitoring
Highlights 👇
🔥 Excited to release practicalAI 2.0 -
- 📚 Illustrative ML lessons in
@TensorFlow
2.0 + Keras
- ⚒️ Build robust models using the functional API w/ custom components
- 📦 Train using simple yet highly customizable loops to build products fast
Just released the API lesson for
@madewithml
's Applied ML course!
- API design from first principles using FastAPI
- Schemas & custom validation via pydantic
- Using decorators for validation & consistency
- Project template with DBs, auth, etc.
@tiangolo
Beyond excited to share the biggest update yet to
@MadeWithML
-- based on 8+ years of helping machine learning teams get to production:
- 📈 Scaling ML (+ LLMs)
- 🔗 MLOps integrations
- 🚀 Dev → Prod (fast)
- ✅ open-source
🧵 Detailed thread below 👇
New
@MadeWithML
lessons on data engineering for machine learning to construct a data stack for ELT workflows & orchestrate them for quality data.
- 📚 Data stack:
- ⎈ Orchestration:
- 💻 Repo:
🧵 Thread👇
In the latest
@madewithml
MLOps lesson, we use
@GitHub
Actions for CI/CD workflows:
- Workflow components (events, runners, jobs)
- Testing Actions locally using Act
- Best practices (ex. caching)
- ML Actions (
@expectgreatdata
checkpoints,
@DVCorg
CML)
The first few lessons of
@madewithml
’s Applied ML in Production course are out!
📦 Product
- Objective
- Solution
- Evaluation
- Iteration
🔢 Data
- Annotation
+ more soon!
📄 Details:
💻 Code:
🎥 Videos:
Just released the reproducibility lessons for
@madewithml
's MLOps course!
- Git basics via workflows (dev, collab, inspect, etc.)
- Pre-commit hooks (+ custom local)
- Versioning code + config + data = models via
@DVCorg
- Containerization via
@Docker
We’ve become the
#1
NLP repository on GitHub - - with the POC and now I want to make this content even more interactive and practical for even more people. And oh btw, we’re switching from
@PyTorch
to
@TensorFlow
2.0 + Keras 🙃
Added some new components (fine-tuning embeddings, lexical search, reranking, etc.) to our production guide for building RAG-based LLM applications. Combination of these yielded significant retrieval and quality score boosts (evals included).
Blog:
Excited to share our production guide for building RAG-based LLM applications where we bridge the gap between OSS and closed-source LLMs.
- 💻 Develop a retrieval augmented generation (RAG) based LLM application from scratch.
- 🚀 Scale the major workloads (load, chunk, embed,
🦜 The
@raydistributed
team released Project Aviary today and besides the really fun way to explore LLM outputs (alongside latency, cost, etc.), I was more interested in how they’re serving this LLM model registry (+ evaluation, batch inference, etc.) 🤯 It took me ~30 minutes to
The Experiment Tracking & Optimization lessons are out!
- Tracking, viewing and loading w/
@MLflow
- HP optimization (sampling + pruning) w/
@OptunaAutoML
- Advantages of using
@weights_biases
,
@Cometml
etc.
• Lessons:
• Colab:
⚕️ Deep Learning with Electronic Health Record (EHR) Systems - - Wrote this last yr. before jumping back into clinical ML but never got around to sharing it. Added some 2019 updates (clinical BERT and approach for industry applications).
📊 The Exploratory Data Analysis (EDA) lesson is out for
@madewithml
's Applied ML in Production course!
EDA is:
- not about doing every possible visualization
- used to determine if dataset is apt for task
- intent-driven to answer Qs
- revisited often
📜 The Documentation lesson is out for
@madewithml
's Applied ML course!
- Typing and docstrings (w/ admonitions)
- Autogenerate beautiful docs using MkDocs
- Deploy via Github Actions and host via GitHub Pages
• Repo:
• Lesson:
📝 Just released the Logging and Styling (Black + isort + flake8) lessons for
@madewithml
's Applied ML course (>50% through). There's not really much more ML but now the real fun starts with everything we need to actually deliver value with ML!
Lessons:
😅 Ironically,
@MadeWithML
isn’t made with any ML yet (go manual before ML).
Now it’s time to apply ML and I was thinking of developing this publicly, from Product → ML → Production / MLOps, with open source tools.
Would people be interested in this?
📊 The Baselines lesson is out for
@madewithml
's Applied ML course.
- Why we need baselines
- Iteratively identifying limitations and motivating complexity
- Case study w/ code
- Weighing tradeoffs
- Lesson:
- Refresher on models:
✂️ The Splitting lesson is out for
@madewithml
's Applied ML in Production course! We cover:
- Why we need three splits
- Criteria for proper data splitting
- When to shuffle (and when not to)
- Iterative stratification for multi-label classification
A very comprehensive case study on fine-tuning Llama-2 across three different tasks👇
- code for distributed fine-tuning w/
@raydistributed
+
@huggingface
Accelerate +
@MSFTDeepSpeed
- data prep + eval + baselines
- when to & not to fine-tune
- using perplexity for checkpointing
🚀 Exploring Llama-2’s Quality: Can we replace generalist GPT-4 endpoints with specialized OSS models? Dive deep with our technical blogpost to understand the nuances and insights of fine-tuning OSS models.
🔗
🧵 Thread 1/N👇
What tool do you prefer for experiment tracking and WHY? I'm working on the next lesson for
@madewithml
's Applied ML course & I wanted to cover the contextual advantages of the options instead of just saying use "X".
@Cometml
@MLflow
@neptune_ai
@weights_biases
+ any others?
“ML is way easier to learn now with all the free resources” - yeah, but it’s also a lot more overwhelming (low signal/noise ratio)! I’ve followed these 3 principles to stay afloat:
🔎 Explore vs. exploit
⚒️ Build projects
🗃️ Triage
Details below 👇
Desperate Question: How to all the ML people out there keep track of the barrage of new information? Formal note keeping? Rely on whatever sticks? Some other system? It seems like each year I'm losing ground; would love tried and true tips.
(I'd love a RT for visibility. Thx!)
🤯 70%+ CPU RAM savings at ANY scale is a big deal but especially at 70B (Llama-2) scale! Quick guide for these massive savings w/
@huggingface
accelerate + 3 line
@raydistributed
config that enables heterogeneous clusters ... a must have with today's limited GPU availability 👇
Checkpointing a 70B model like Llama-2 during training is a tough task, particularly when doing full-parameter fine-tuning on limited resources (A10Gs). But today, I had a fascinating revelation with
#Ray
that enables this process. Let's get technical! 🧵👇 1/11
🔥 A lot happening today on
@madewithml
including research, articles, videos on text generation, pose estimation, self-supervised RL, BART for QA and even a cool podcast (
@underrated_ml
)!
@madewithml
has fundamentally changed how I keep up with ML. 👇
As promised, here is the first draft of the Colab notebook that shows how to use pretrained LMs and an Elasticsearch search engine on the
@elastic
cloud to build a simple text similarity search application using COVID-19 scholarly articles dataset. Enjoy!
The Preprocessing lesson is out for
@madewithml
's Applied ML in Production!
- methods, caveats, unique ways to encode
- Prepare: missing values, outliers, clean
- Transform: scale, encode, extract
📄:
🎥:
🔝 Made With ML Top Projects of the Week: Power normalization, SOLT, Self-supervision w/ fastai, JAX tutorials, Visual pre-training, NeRF, LM Decoder methods and more! I'm absolutely blown away by the ML projects that have been added by the community.
🔝Check out this week’s top Tutorials, Toolkits and Research from
@madewithml
: Torchlayers, Finetuning w/ Haiku, Illustrated graph NNs, Visualized debugging, NLP highlights, Text-to-image GAN with BERT and more!
Hey everyone, I need a bit more time to kick off these lessons. We'll start doing regular releases every Monday starting Oct 26.
And to be honest, it's lonely recording these lessons (I prefer live talks) so I've asked someone special to join me (hopefully they say yes).🤞
🔥 Putting ML in Production! We're going to publicly develop
@madewithml
's first ML service. Here is the broad curriculum:
- 📦 Product
- 🔢 Data
- 🤖 Modeling
- 📝 Scripting
- 🛠 API
- 🚀 Production
More details (lessons, task, etc.) here:
Thread 👇
We (
@pcmoritz
& I) have been productionizing LLM apps (more later) but at the heart are OSS LLMs served via
@anyscalecompute
Endpoints.
- ✅ Drop-in sub for OpenAI
- ☁️ Deploy on own cloud if needed
- 💸 < $1 / M tokens for Llama-2-70b
Try it for free 👉
👋 Twitter, it’s been a while! I'm grateful for good health again and excited to return back to teaching. Expect 💻 + 🎥 content soon at and as always, it'll be 100% open-source :) Thread below👇
Who is this course for?
- 💻 Software engineers / Data scientists looking to learn how to responsibly create ML systems.
- 🎓 College grads looking to learn the practical skills they'll need for the industry.
- 🚀 Product Managers who want to develop a technical foundation.
🔥 The
@practicalAI_me
curriculum and Lesson 01. 🐍 Python are out! - - Most of you know everything in the lesson but we do cover decorators and callbacks which we’ll heavily use with training scripts and when building production ML apps with Flask.
Couple of awesome posts with great tips for Python builds with GitHub Actions 👇 Had to try
@epwalsh10
’s tip right away and managed to decrease build time by ~40% (compared to previously just caching wheel files)!
Just finished reading this and if you’re overwhelmed with all the NLP research,
@omarsar0
’s NLP Highlights are a must read! He carefully summarizes major trends and their implications without the technical jargon - and it’s a great way to figure out what papers to read in-depth.
NLP Research Highlights — Issue
#1
ICYMI, in this article I highlight some NLP trends and topics with a focus on summarizing the what, how, and why of a selection of interesting and important NLP papers published in the last few months.
Even more exciting content coming later this year, so stay tuned!
- 🏆 Among top MLOps repos on GitHub:
- 🛠️ A highly recommended resource used by industry:
- ❤️ 30K+ community members:
What other factors should I consider when deciding b/w models? I started listing few for the Baseline lesson in
@madewithml
Applied ML course:
- performance
- latency
- size
- compute
- bias tests
- 🕓 to develop
- 🕓 to retrain
- maintenance overhead
What else should be here?
1/ Working on hype-free, practical AI for non-technical folks. First posts are on where to use AI and some basic intuition on data/models. Upcoming posts will be on designing AI products and building scalable engineering pipelines for AI. Check it out @
We start with lessons on the fundamentals of ML through intuitive explanations, clean code and visualizations.
📚 Foundations
- Python (variables, functions, classes, decorators)
- NumPy (numerical analysis)
- Pandas (data analysis)
- PyTorch (operations, gradients)
🏥 A friend asked if ML can cure COVID-19 as a joke. While ML may not directly cure it, here are some cool projects that may help us get there (code/blog/videos included) from visualization/APIs to identifying potential drugs that can inhibit the virus.
We’ve covered all the topics (complete w/ code) that I planned to cover, so that’s a wrap for now. But we’ve got even more amazing content coming later this summer!
🎓 Lessons:
🐙 Repository:
📬 Subscribe:
Be sure to checkout both of the new lessons for a lot more details and the step-by-step UI and code walkthroughs to set up and orchestrate these data workflows. And oh, we've got a few more updates on the website too :)
The research in CVPR this year has been producing quality results for tools to augment designers, storytellers, etc. So grateful to see this work on
@madewithml
and have it be discovered, tagged and curated for future learning/building!
🏆 Trending content of the day on
@madewithml
by Erik Härkönen (). This looks like so much fun to experiment with the layer-wise, interpretable controls and then save it and apply it to other images!
Project details:
With
@madewithml
, I’m discovering so many great resources that I’d otherwise miss on Twitter because I wasn’t following the right people or online at the right time. I also realized that researchers/devs share amazing content on Twitter that never sees the light of day (cont.) 👇
The next lesson involves setting up CI/CD pipelines with
@github
actions and then just three more lessons before a major update/release!
🎓 Lessons:
🐙 Repository:
📬 Subscribe:
First lesson releases next week (📦 Product) & subsequent lessons will follow a weekly cadence. Be sure to follow me or
@madewithml
for updates, discussions & feedback because I'll be creating the course content dynamically using the community's feedback.
I learned the basics of JAX today (to counter cabin fever) and I found these resources really helpful: and especially this recent one by
@RobertTLange
. Let me know if there are any other notable resources I should check out!
Puuuh. What are you up to these days? 💭 I try to stay sane, clean my place 🧹& write✍️. Todays edition - 'Getting started with
#JAX
'. Learn how to embrace the 'jit-grad-vmap' powers 💻 and code your own GRU-RNN in JAX. Stay safe & home. 🤗
It will be a hands-on, code-driven case study for the task we're trying to solve but also a guide for production ML / MLOps and clean software engineering.
Next steps: If there’s a lot of interest, I’ll detail the specific task and the curriculum in a few days and share it.
🚀 No doubt that we’re headed towards multi-modal LLMs (text, image, etc.) but scaling those models (10B+) for inference (200TB) is a different story.
👇 Here's a detailed read on how
@BytedanceTalk
(TikTok's parent org) uses
@raydistributed
to achieve this (+ cost reductions).
@BytedanceTalk
, the company behind TikTok, uses Ray for fast & cheap offline inference with multi-modal
#LLMs
. They generate embeddings for a staggering 200 TB of image and text data using a model with >10B parameters.
🧵 Thread below 👇
Then we dive into implementing basic ML algorithms 1⃣ from scratch then 2⃣ in PyTorch. Starting from simple models → complex models.
📈 Modeling
- Linear Regression
- Logistic Regression
- Neural Networks
- Data Quality (⚠️ very important)
- Utilities (for loading and training)
I like to build self-sustaining tools that enable others to learn. So far with
@madewithml
, we have the:
🗞 Newspaper: trending content of the day/week.
📚 Library: automatically up-to-date collection of the best resources by topic.
📚 Extremely excited to launch Made With ML Topics - a collection of the best ML tutorials, toolkits and research organized by topic. It’s all automatically organized, always up-to-date and curated by the community.
Details below 👇
We wrap up the fundamentals by implementing deep learning algorithms in PyTorch.
🤖 Deep Learning
- CNNs
- Embeddings
- RNNs
- Transformers
💡 We motivate the need for specific architectures and additional complexity as we implement each method.
@sudalairajkumar
@madewithml
Thank you Sudalai! It’s going to be updated every few months to reflect the best practices from industry with a continued focus on the fundamentals so that readers can adapt to the changing landscape.
We want to ensure that our work is entirely reproducible by anyone.
♻️ Reproducibility
- Git basics via workflows (dev, inspect, merge, etc.)
- Pre-commit hooks (+ custom local)
- Versioning code + config + data = models via DVC
- Containerization via Docker
Storage, compute and interface with notebooks! "the most popular tool at Netflix for working with data — across the entire data platform user base —is now notebooks."
They developed a self-supervised framework that de-occludes scenes:
1. Extract the order of the objects via pair-wise relations
2. Compose objects to completion (amodal and content)
3. Recompose the entire scenery (!!!)
For more by
@ccloy
et al.:
ML is not a separate industry, instead, it's a powerful way of thinking about data. The foundations we've laid out will continue to hold but the methods and avenues of application will evolve. So these lessons are by no means "complete" and we'll continue to keep them up-to-date.
Another extremely well written post by
@jeremyjordan
, this time on testing ML. He nails it with the image below. We'll be heavily focusing on a lot of what he touches on in
@madewithml
's upcoming release next month!
machine learning systems can be more difficult to test than traditional software systems since we're shoving most of the logic into parameters of a model. i wrote a blog post collecting my thoughts on the subject, let me know what you think!
Then, we wrap all of the CI/CD workflows we’ve created with
@GitHub
Actions:
🔄 CI/CD workflows
- Workflow components (events, runners, jobs)
- Testing Actions locally using Act
- Best practices (ex. caching)
- ML Actions (Great Expectations checkpoints, DVC CML)
Behavioral testing via NLP Checklist:
- invariance: change should not affect outputs
- directional: change should affect outputs
- minimum functionality: simple combination of inputs and expected outputs.
- Creating tests interactively via Checklist:
@bhutanisanyam1
@madewithml
I have non-CS background → founded a small (failed) startup → worked at a large company → worked at a small startup. Happy to share my journey, tips and challenges for those in research/industry and what I think really matters at the end of the day on your learning journey
A lot of people have been asking if their project is “good enough” to post on
@madewithml
. The answer is always YES because there’s always a crowd that cares about your niche project and they can now discover it via search or the Topics page. (cont.) 👇
The first MLOps lessons are on the Product development and iteration cycle.
📦 Product
- Identify the core objective.
- Design a solution with constraints.
- Evaluation strategies that avoid bias.
- Iterate via feedback and motivate adding complexity.
I’m recording the lesson on EDA and I’m curious how people work with notebooks and testing/formatting?
N =
@ProjectJupyter
or
@GoogleColab
notebooks
S = Python scripts
→ = transfer code
If you're doing
#3
, are you calling functions that are written in scripts into notebooks?
I currently work closely with early-stage & mid-sized companies in helping them deliver value with ML while diving into the best & bespoke practices of this rapidly evolving space. I want to share that knowledge with the rest of the world so we can accelerate overall progress.
I'll be updating many of the existing lessons with what I'm learning through my work but also from what I'm learning via collaborations w/ teams across many industries. It's an amazing time to witness how quickly research transitions to production-grade!
This thread was an abbreviated summary of the lessons in
@madewithml
’s MLOps course, so be sure to check that out for much more detailed and up-to-date content.
🎓 Lessons:
🐙 Repository:
📬 Subscribe:
Over the past 7 years, I've worked on ML and product at
@Apple
, health tech startups and ran my own venture in the rideshare space. I've worked with brilliant developers and managers and learned how to responsibly develop and iterate on ML systems across various industries.