today
@continuedev
released v1 of tab autocomplete
it's 100% local and open-source
for the next few months i'm going to share (live, as i learn) the tricks that improve our acceptance rate
if you follow along you might learn...
⚡️ excited to share that
@continuedev
is releasing our open-source tab-autocomplete! ⚡️
why might you want to use it?
- local (code remains on your machine)
- customizable (change model, temp, and more)
- open-source
- and free!
here's how we built it:
yet another obvious autocomplete update that makes a huge difference:
even if truncating the completion after one line, let the model keep going. 90% of the time the user just presses enter, and then you can present them with the 2nd, 3rd, etc. lines almost immediately!
better cross-encoders are a big deal. out of 300,000 loc,
@Voyage_AI_
embeddings + reranker found 300 loc that allowed for basically the answer i'd give as the author
We have many difficult problems to solve, but for each that we do, software becomes easier for the world to build.
If you want to help tackle these, Continue is now hiring. Join us:
Today we’re releasing codebase retrieval! If you want to edit a large codebase but don’t know where to start, just use ⌘+⏎ and Continue will automatically pull in the relevant snippets of code.
seeing really impressive autocomplete results from
@deepseek_ai
's 1.3b model
it's become clear that
a) at least 1/2 the work is in constructing the right prompt, making the model's job easier
b) once you do this, small (local!) models will shine
Not everyone has the freedom and motivation to be idealistic. But if you do, time spent thinking long-term is underrated. Question your foundations and rigorously determine your angle.
whenever you type the opening parenthesis of a function call,
@continuedev
's autocomplete will now use the language server protocol () to add the function definition to the prompt
a nice part of building a dev tool is coming across other dev tools and deeply appreciating the work that went into simplifying an interface
is one that i’ve appreciated lately
at the core of most tab autocomplete systems is a tool called tree-sitter ()
tree-sitter makes it fast and easy to parse abstract syntax trees in any programming language
where we've found it extremely helpful so far is by using the "ast path"
This is why Continue helps you collect your own “development data” (dumped to a local .jsonl file)
Training on fine-grained dev data isn’t currently commonplace, but there have been explorations like Google Research’s DIDACT
And the longer you’ve been
The ideal training data for an LLM is not what you wrote. It's the full sequence of your internal thoughts and all the individual edits while you wrote it.
But you make do with what there is.
@metcalfc
@Ollama_ai
@continuedev
It should! I’ve got Continue x Ollama working on my own Windows machine with WSL. And I believe all instances of WSL would share a loopback interface
and as usual, Continue lets you customize. any stop token can be added through config.json, and if you never want multi-line, there's an option to `disableMultilineCompletions`
2. mis-matched brackets will be saved for another day
today
@continuedev
released v1 of tab autocomplete
it's 100% local and open-source
for the next few months i'm going to share (live, as i learn) the tricks that improve our acceptance rate
if you follow along you might learn...
@FilterPunk
@continuedev
@MetaAI
@replicatehq
@togethercompute
You can edit the `server_url` of the model you are using in the config file. For example with Ollama: `default=Ollama(model="codellama", server_url="<your_hosted_endpoint>")`. Many other options as well if you're self hosting:
today
@continuedev
released v1 of tab autocomplete
it's 100% local and open-source
for the next few months i'm going to share (live, as i learn) the tricks that improve our acceptance rate
if you follow along you might learn...
this finds useful code in a surprising number of cases, but we add in one more source of context
by also keeping track of recently edited ranges of code, we can prompt the language model with the exact few lines that you just got done editing
5) data cleaning
eventually models will need to be fine-tuned. the hard part is finding the right data. what heuristics will distinguish useful training examples from that time you pressed tab just to see what happens?
@erhartford
@SourcegraphCody
@ollama
With
@continuedev
you can use any model from any provider for chat, Codellama-70b on Ollama included! And we’re releasing tab-autocomplete tomorrow (also allowing any model, and access to configure basically every setting you could want)
between these two retrieval tricks, autocomplete feels on another level
next up will be using "go to definition" in more situations, i think likely to have similar impact
so instead of cancelling and sending another request, we just keep listening to the current one until it's invalidated
by the time i've typed 'con', the response might be complete and we can display this to the user
because i can't type 1000+ wpm, this will come in handy
next up is
- applying the same to implementations of interfaces/methods
- collecting information about surrounding variables
- and resolving imports to important snippets
but now that we know to generate multiple lines, when do we stop?
there are two main issues:
1. generating too much code
2. mis-matched brackets / parentheses / quotes
first, we construct the abstract syntax tree (ast) for the current file and find the leaf node that contains the current cursor position
then, repeatedly moving up one parent node, we walk to the top of the tree, keeping track of all the nodes seen on the way: the "ast path"
whenever you type the opening parenthesis of a function call,
@continuedev
's autocomplete will now use the language server protocol () to add the function definition to the prompt
this is another chance to use the "ast path"
we check whether any parent node is of the "statement_block" type (a fancy way of saying you're in a `{ }` in .js/.ts) and whether the cursor is at the start of this block
if so, then we use multi-line
at the core of most tab autocomplete systems is a tool called tree-sitter ()
tree-sitter makes it fast and easy to parse abstract syntax trees in any programming language
where we've found it extremely helpful so far is by using the "ast path"
the outcome is seen below. only two requests are sent: immediately after i type the first character (generates '// ...') and again after I'm done typing
but there's still more to do
what we want instead is:
1. immediately send a request after the first keystroke, but also set a variable `debouncing = true`
2. set a timer for ~300ms. when it reaches zero, set `debouncing = false`
3. whenever `debouncing === true`, use the same policy as vs code search
step 1 is just keeping track of the most recent ~10 files
step 2 is to search over them. we use a sliding window matcher to find the range in each file that is closest to the range around the cursor
at the core of most tab autocomplete systems is a tool called tree-sitter ()
tree-sitter makes it fast and easy to parse abstract syntax trees in any programming language
where we've found it extremely helpful so far is by using the "ast path"