Here's my
@PyDataLondon
talk on human-in-the-loop distillation:
⚗️ LLMs → distilled task-specific models
🌎 real-world case studies
📉 avoid prototype plateau
✊ no need to compromise on best practices & privacy
Slides:
Video:
Like many of you, I'm incredibly disappointed by DataCamp. I wanted to make a free version of my spaCy course so you don't have to sign up for their service – and ended up building my own interactive app. Powered by the awesome
@mybinderteam
&
@gatsbyjs
💖
TIL:
@Firefox
comes with a built-in JSON viewer, which automatically transforms JSON-formatted files and API responses into a collapsible, searchable and syntax-highlighted tree 😍
Someone asked me for tips in the comments, so I might as well make this a thread. Here are my personal recommendations for how to get started with front-end development, APIs, CSS, React, UI design and more 👇
.
@streamlit
is very cool 🔥✨
It took me hardly any time and around 100 lines of Python to build an interactive
@spacy_io
model visualizer, complete with dependencies, named entities, similarity and more.
📄 Code:
▶️ $ streamlit run streamlit_spacy.py
The video of my
@EuroPython
2018 keynote is now live: "How to Ignore Most Startup Advice and Build a Decent Software Business" ✨
For everyone who's been asking for more readings along the same lines, here are some classics (thread)
Looking to self-publish your own interactive online courses? 🎓🐍
It's now even easier, thanks to the amazing
@hayleyfboyce
, who contributed a super detailed guide for my little open-source framework. Plus a Dockerfile for faster setup!
📖 Read it here:
I'm trying something new today! 💥
Had a lot of fun giving my keynote talk at
@PyConColombia
yesterday, so here it is – in tweet form. One tweet per slide.
The future of NLP in Python (thread)
Full slide deck:
This came up the other day, so here's a quick guide to fast processing with spaCy!
Full pipeline:
🐌 docs = [nlp(text) for text in texts]
🔥 docs = list(nlp.pipe(texts))
Tokenizer only:
🐌 docs = [nlp.tokenizer(text) for text in texts]
🔥 docs = list(nlp.tokenizer.pipe(texts))
Built this wrapper for the new
@StanfordNLP
library so you can use their state-of-the-art Universal Dependencies models directly in your
@spacy_io
pipeline 💘
Today we also started open-sourcing some of our datasets & NLP example projects!
Includes 1k+ annotated examples each, train/eval scripts, results, data vizualizers & some powerful tok2vec weights trained on Reddit to initialize models.
💝 Repo:
We bootstrapped an NER model from scratch with sense2vec + . In ~2.5h we created 1735 annotations and trained a decent model. Let us know if you beat the score with a different implementation 🙌
📄 Code & data:
Here are the slides for my
@PyDataLondon
keynote on LLMs from prototype to production ✨
Including:
◾ visions for NLP in the age of LLMS
◾ a case for LLM pragmatism
◾ solutions for structured data
◾
@spacy_io
LLM +
I've started putting together flowcharts for solving various NLP problems with (and beyond). Obviously none of this is foolproof – it's just a summary of our usual advice. Here's the first one for Named Entity Recognition!
📥 PDF:
My first
@DataCamp
course on advanced NLP with
@spacy_io
is finally out! I put a lot of work into this, and I have a lot of thoughts. A short thread 🎉👇
New Course by
@_inesmontani
: Advanced NLP with spaCy! Learn how to use
@spacy_io
, a fast-growing industry standard library for
#NLP
in
#Python
, to build advanced natural language understanding systems, using both rule-based and
#MachineLearning
approaches.
The app and framework are 100% open-source and based on Markdown + custom elements. I built it for my content, but if you want to use it to publish your own DIY online course, it should be easy to adapt. In theory, it should even work for other languages.
Working on a new feature and needed to create lots of visualizers. So I built spacy-streamlit:
👑
@spacy_io
building blocks for
@streamlit
apps
📚 Parser, NER, text classification, similarity & tokens
📦 Embed out-of-the-box or use components
Repo:
Here are the slides from my
@PyConIndia
keynote ✨🐍 (The talk was recorded, so there should be a video up sometime soon.)
Thanks to everyone who came by 💖 I've had so many great conversations with so many people working on all kinds of different stuff.
Started building wheels for all our Python packages! Here's what this means:
⌛️ Super fast installation – install
@spacy_io
in <1 min!
🎛 No compilers needed locally (once all dependencies have wheels)
🐍 Releasing our scripts to help others do the same
I built another
@streamlit
app (~250 lines of code 🤯), this time for !
✨ View datasets, annotations & stats
✨ Visualize & search merged NER annotations
✨ Run quick training experiments for NER and text classification
📄 Code:
Something fun for the weekend: I built a shitty cat petting robot with a Raspberry Pi and Python 😹🐍
The process was a lot of fun so I documented it for those interested in embedded development. Bonus: cute cat pics ✨
Speaking of our team, I just want to note that
@spacy_io
has been a majority female-run open-source project for quite some time ✊ Core devs:
@OxyKodit
, Adriane (not on Twitter),
@honnibal
& me.
(Or put differently: we have more female NLP PhDs on our team than guys named Matt.)
What a year! We put together a review with some of the highlights 🎉 Includes: notable releases, talks, great podcasts,
#spaCyIRL
and 4 new team members. Thanks for all your support!
✨📺 NEW VIDEO! 📺✨
I'm using &
@spacy_io
to train an NER model from scratch with semi-automatic annotation & transfer learning. It's about food trends & we're also building a cool bar chart race visualization!
Watch it here:
Recorded a video version of the
@spacy_io
course! 👩🏫🎬
You can watch it as video lessons in the course app, or as one single video tutorial on YouTube, with breaks for exercises. More languages coming soon!
📝 Course:
📺 YouTube:
Treated myself to a new tattoo after a long time because I've been working hard and being productive. (I know it's not an actual Python snake, but still!)
✨🐍🌸
Oh wow, apparently you can now use
@github
code navigation from within code blocks in the README?! 🤯
This is really cool and will allow a whole new level of documentation.
I've been writing lots of code over the past few days 👩💻 And it made me realise how I've grown as a developer.
When I solve tricky bits, I add more comments & explain. When I was less confident I think I felt this would reveal my lack of experience. Now I know it's the opposite.
Thanks to everyone who came to my
@EuroPython
keynote on LLMs from prototype to production ✨
Here are my slides and a walkthrough of the talk as a Twitter thread 🧵
I feel incredibly honoured to become a Python Software Foundation Fellow and be part of this awesome community 🐍✨
Congrats to all the new fellows, including
@karobotco
,
@jhonjairoroa87
&
@mariatta
!
Okay, so this is SUPER experimental and might not work 100% of the time yet. But many code examples on the
@spacy_io
website are now editable and executable in your browser! 💖
You can see it in action here:
I've long admired tech zine creators like
@b0rk
&
@sailorhg
, and I needed a creative outlet. So I started on a little
@spacy_io
zine. I can't draw, so I'm going for "cheatsheet meets coffee table book" 📚🎨✨
If I got this printed as a mini book, would anyone be interested?
Developer path:
-Started code at 11
-Work on laser-plasma
@BerkeleyLab
-PhD on quantum physics
-Switch to IP law
-European Patent Attorney
-Discover Machine Learning at
@iclr2017
-Open-source first ML library
@huggingface
-1M download, HF raises $15M, hires crazy talented people
Travelling to Medellín today for
@PyConColombia
✈️🐍
My keynote will be about the what the future holds for NLP, and how we can enable a new level of developer experience to make the technology available to everyone 🙌
Great article in
@Guardian
on how they're using
@spacy_io
and to indentify quotes in text ✨
I especially like the focus on designing annotation schemes to deal with ambiguity in language – it's why iteration is SO important!
If you want to play with the new (or your own custom trained)
@spacy_io
v3 models, check out this app I built, hosted with the new
@Streamlit
sharing – also launched today, so perfect timing! 🥳
🖼️ App:
👑 spacy-streamlit:
People often ask me how I make my talk slides or for recommendations for getting started, so here are some of my not-so-secret secrets and beginner-friendly tips!
⚡ A beginner’s guide to making beautiful slides for your talks
With 2020 coming to an end and spaCy v3 just around the corner, we hope we can brighten up your 2021 with some stickers!
Including all new designs and ✨holographic✨ styles. It's FREE and we ship worldwide.
💌 Get yours:
Since pre-releasing
@spacy_io
v3 nightly, we've mostly been talking about the big features like transformer-based pipelines, the config system etc.
But there's also a bunch of smaller details that you'll love if you're a spaCy power user 💪💙
Here are some of my favs! (thread)
Seasonal reminder if you're looking for resources to learn data science: my
@spacy_io
course is free (with 5 translations)!
💬 English, German, Spanish, French, Japanese, Chinese
📺 Videos in English, German & Spanish
🐍 Interactive in-browser exercises
NEW BLOG POST: Applied NLP Thinking – a mindset for translating business problems into ML solutions.
TL;DR
🧠 Think about the "what", not just the "how"
📈 Check utility, not just accuracy
🚀 Make prototype to production smooth
📚 Linguistics can help!
Here's the video of my
@PyConIndia
keynote 🐍⚡
In "Let Them Write Code" I talk about our philosophy for building developer tools and why good tooling should be programmable.
My
#l3ai
talk on designing practical NLP solutions is now live! It's essentially a more detailed and fleshed-out version of the points in the thread below 👇
📺 Video:
📄 Slides:
@yoavgo
I think an important aspect here is that research (rightly) focuses on what generalises, while a lot of the most valuable aspects of "real world" and industry problems are the ones that are very *specific*. They're often not solved by just throwing some large model at them.
🔥 Coming soon to
@spacy_io
v3 and 🔥
◾️ span categorizer component to predict labels for overlapping spans & phrases
◾️ annotation mode for overlapping spans
◾️ mix with classic NER or custom suggestion logic
◾️ efficient Doc.spans data structure
Omg, I was just walking to brunch with
@honnibal
and we were talking about refactoring the tokenizer. Just by the canal, someone comes running up to us and is like "Hey, are you Matt and Ines from
@spacy_io
? Your work has made my life so much easier!" 🙈💕
If you want to find out what the new
@spacy_io
v3 is all about, check out this video I recorded with
@honnibal
😇 We're walking you through some of the most exciting new features!
📺 Watch it here:
Currently on the plane to Chennai & on my way to
@PyConIndia
✈️☀️
My keynote is titled "Let Them Write Code" and I'll be talking about our take on what makes a good developer tool.
Here's to a better 2021 🎊✨
In other, more personal news: I'm sort of living in Melbourne now! At least until it's safe(r) to return to Europe. Looking forward to settling down for a bit.
10 years ago today
@honnibal
pushed the first commit to
@spacy_io
🎉
Since then, the library has evolved as the field moved forward, but also stayed true to its core mission: industrial-strength NLP and bringing structure to unstructured text.
It's not always been easy building
I have no idea how
@wjb_mattingly
finds time to produce so much content, but this YouTube channel is a real treasure trove of NLP, ML, Python and
@spacy_io
tutorials, for Digital Humanities & beyond, from hands-on applied NER to analyzing Latin 🤓
📺
📣 This is hands-down the most useful thing I ever added to my .zshrc!
Activate a 🐍 virtual environment or create a new one if it doesn't exist and install the go-to packages I use in every project.
$ venv
$ venv .env37 # custom directory
📝 Code:
Our two-week hotel quarantine ended today & I finally got to see Perth! ☀️🌴
(Coming from Europe, it's pretty surreal entering this parallel universe with no lockdowns, zero community spread and life more or less back to normal. Almost makes me feel bad for enjoying it.)
🔥📺 Aaaaand another video! 📺🔥
This time I'm using to build fully custom annotation workflows and UIs for image captioning. And I'm showing how to plug in a simple
@PyTorch
image captioning model. (Also: cats! 😺)
Watch it here:
WE DID IT! 🔥🍾
It's a prerelease but we don't expect many changes, and there's SO much cool stuff, I don't even know where to start.
This was an incredible engineering effort by the team & I'm proud to be working with such cool people! Thanks
@honnibal
,
@OxyKodit
& Adriane 💪
IT'S HERE! Today we're releasing spaCy nightly, the first candidate for the upcoming v3.0.
🛸 Transformer-based pipelines for SOTA models
⚙️ New training & config system
🧬 Models using any framework
🪐 Manage end-to-end workflows
🔥 New & improved APIs
My
@GlobAICommunity
talk on
@spacy_io
v3.0 is now live! In it, I'm walking through some of the most exciting new features & the ideas behind them.
📺 Video:
🌙 spaCy v3.0 nightly:
💥 Blog post:
Currently working on a complete revamp of the docs and built this little CLI command visualizer. Can't wait to get it all ready and share it with you. I'm also writing lots of new content 🔥
Updating the popular train curve diagnostic in for spaCy v3 ✨📈
It can help answer the question: Will more data like this improve my model?
To do this we simulate training with less data (e.g. 25/50/75%) to see if accuracy still goes up towards the end.
Just tried
@allen_ai
's sciBERT with spacy-pytorch-transformers ✨ (Very easy to load it from a path, just make sure all files are in the same dir & there's a config.json.) Can someone who's used it before check that the outputs are reasonable?
Details:
Since so many of you expressed interest in my DYI course framework, here's a first attempt at a starter repo for Python to fork & customise. I'll try R next – if you're working with
#rstats
and want to help / answer some of my questions, let me know!
In my keynote at
@PyConIndia
, I talked about how "simple" UIs are often just leaky abstractions.
Autocat by
@MarkNeumannnn
at
@allen_ai
is a great example of a web-based UI for developers that adds a ton of value and complements the development process 🚘🐱 Here's why. (thread)
Something i've been working on for the last couple of months:
Train practical
@spacy_io
models on your data, download a pip installable model, with the idea of reducing the amount of time between idea and implementation.
#NLProc
Little shortcut I wish I had added to my .zshrc sooner:
1️⃣ Check if .env directory (my standard virtual env) exists.
2️⃣ If not, create a new virtual env.
3️⃣ Activate env and set Python path to current dir.
Code:
This is our biggest release so far & I probably wrote the equivalent of a book in docs over the past few weeks 🤓✍️ 💕
2+ years of best practices & annotation/training tips. Even if you've been using the tool, there's lots of new stuff to discover!
Introducing Prodigy v1.9, our biggest release so far!
🎁 new site & tons of new docs
🚂 new training & data conversion recipes
🖼 "blocks" UI for combining interfaces
✍️ UI for free-form text input
✨ lots & lots more
Changelog:
Quick personal update: Caught a flight to Australia with
@honnibal
to escape EU lockdowns & currently in mandatory quarantine in Perth 😷🌴
I guess my life won't be substantially different – lots of computing, minus cooking, plus cops showing up to check that I'm still inside 😅
One of the most significant changes in
@spacy_io
v3 is the new config system for defining & training pipelines.
Validation is crucial here, but how does it work? A key part is Pydantic by
@samuel_colvin
:
Here's how it looks under the hood (thread) 👇
Oh wow, this is super useful: looks like
@github
added (or is testing?) a separate bugs tab that only shows issues labelled as "bug" 🐞
Issues are so much more than just bug reports and many open issues != many problems.
Almost forgot to post this, but here are the slides from our closing talk at
#spaCyIRL
. Featuring the history of
@spacy_io
, the roadmap for v3, how we make money and a bonus pic of
@honnibal
with a cat on a leash 🐈
📺
@honnibal
's
@MLOpsWorld
talk "How many Labelled Examples do you need for a BERT-sized Model to Beat GPT-4 on Predictive Tasks?"
1. Predictive tasks still matter.
2. In-context learning isn't optimal for predictive.
3. Workflow for labelled examples.
Ahh, I was wondering where all my new followers today were coming from. Thanks for the shoutout
@kdnuggets
, honoured to be part of that list 💕 And hi new followers! 👋
Looking forward to my keynote at
#TeachingNLP
✨
I'll be talking about what I call "Applied NLP Thinking": how to develop the right mindset for solving real-world NLP problems & why it's different from research.
📆 June 10, 18:00 UTC-4
📖 Details:
We are so pleased to have
@_inesmontani
as one of our
#TeachingNLP
keynote speakers!
Please join us at 18:00 UTC-4 next Thursday June 10 for Ines' talk on :
What does “real-world NLP” look like and how can students get ready for it?
Just going through the spaCy dependency graph on GitHub & there's so much cool stuff I hadn't seen 🤩 Anyone want to help with adding some of those to the spaCy Universe ()?
I really enjoyed making this video 💙
I finally got to talk about developer experience in-depth, why we designed spaCy v3 this way, how the config system works & why you don't solve any problems by just abstracting away the complexities of ML.
📺 Video:
spaCy v3 introduces several new concepts, including the config system, function registries & type-based validation.
In this video,
@_inesmontani
explains how they work, how we’ve implemented them & most importantly, WHY.
📺 Watch it here:
Btw, I've reorganized our YouTube playlists so it's easier to find stuff. If you feel like watching some NLP content, check it out 👀
We have all talks from
#spaCyIRL
, end-to-end tutorials,
@fishnets88
's spaCy series & talks from various conferences.
📺
Our interactive spaCy course was just updated 💙 Featuring support for spaCy v3, latest best practices and new exercises introducing the training CLI and config system.
➡️
We've updated our interactive NLP course for spaCy v3!
💬 The updated course is available in English, Spanish, German and Japanese
📚 4 interactive chapters: from the first steps to your own spaCy model
🍰 New exercises about the training CLI & config
Thanks
@EuroPython
for a fantastic conference 🇪🇺🐍 So many great conversations and interesting talks. Here are the slides from my keynote – thanks to everyone who came by or watched the live stream!
Excited to launch spaCy Tailored Pipelines today, our new service for custom applied NLP pipelines, built by the
@spacy_io
team.
We'll build you a production-ready pipeline with up-front fees, try before you buy & maintainable code to adapt & retrain ✨
"Sometimes when we look at a polished interface we can acknowledge that it looks good but it’s hard to articulate why it looks good."
If you ever felt this way & struggled to make things look ✨ nice ✨, I highly recommend this talk by
@steveschoger
.
Experimented with packaging my online course framework as an Electron app and it... just works 🤯
Wrapping web apps isn't always a good idea. But it does open up some new possibilities: easily share it without public hosting & sell premium courses online 100% independently 💸
🖼️ UI design: In terms of actual *design* & building UIs, Refactoring UI by
@steveschoger
and
@adamwathan
is an amazing resource. It especially helps pointing out those small little details that just "look good" but that you may have never noticed before.
Looking forward to speaking at the
@GlobAICommunity
NLP edition tomorrow. It's my first talk on
@spacy_io
v3 and I'll walk you through some of the new features & the ideas behind them!
📆 October 22, 12:00 PST/15:00 EDT/21:00 CEST
🎟️ Info & registration:
Even beyond portfolios, I think there's a huge benefit in mixing different skills & areas of development 💯
It's also key to our company. I wrote a post on "how front-end can improve AI" back in 2016 (!) which outlined that philosophy & product vision:
I get the impression that many computer scientists look down on webdev, but I believe that everyone should learn some basic webdev skills.
Being able to quickly create a website to showcase your portfolio can make a huge difference in how people find and perceive your work.