sqlcoder is an open-source LLM that converts natural language to high-quality SQL queries, making it easy to query data for even complex schemas and questions
Run it locally with Ollama:
TinyLlama is a 1.1B model with the Llama 2 architecture, trained on 3 trillion tokens.
Its small size means it can run fast with little memory and compute requirements.
Neural Chat is a new model based on Mistral and fine-tuned by Intel.
It's currently the highest-ranked 7B model on the HuggingFace H4 open-source LLM leaderboard.
Ollama 0.2 can now:
* Run different models side-by-side
* Process multiple requests in parallel
This enables a whole new set of RAG, agent and model serving use cases. Ollama will automatically load and unload models dynamically based on how much memory is in the system.
Ollama 0.2 is here! Concurrency is now enabled by default.
This unlocks 2 major features:
Parallel requests
Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases
zephyr is a 7B model created by the HuggingFace H4 team.
It's a fine-tuned version of Mistral 7B that beats Llama 2 70B Chat on a series of benchmarks. Built-in alignment layers were removed to improve results.
Docker + Ollama
Deploy and run LLMs such as Llama 2 and Mistral in Docker using Ollama.
Chat with models locally in containers + export a port to serve models over a REST api.
GPU acceleration built in with both Intel and Arm image versions available
Notux is a new 8x7B mixture of experts (MoE) model by
@argilla_io
, created by fine-tuning Mixtral on a high-quality dataset.
It's currently the top-performing mixture of experts model on the Hugging Face Open LLM Leaderboard!
Starling is a new 7 billion parameter large language model by
@BanghuaZ
& team.
This model outperforms every model to-date except GPT-4 and GPT-4 Turbo on MT-Bench, a benchmark to assess chatbot helpfulness.
CodeGemma is a new collection of 2B and 7B models by Google that specialize in coding tasks:
* Fill-in-the-middle code completion
* Code generation
* Mathematical reasoning
* Instruction following
Dolphin 2.6 Phi-2 is a new uncensored chat model by
@erhartford
Since this version of Dolphin is based on the small 2.7B Phi model by Microsoft Research, it's fast and can run on a wide range of machines using
@ollama
.
codebooga is a new 34 billion parameter code model created by the infamous Oobabooga.
It's a merge of two other popular coding models and has been praised by model creator
@erhartford
for being the best code instruct model next to GPT-4.
Dolphin 2.1 Mistral is
@erhartford
's most recent 7b, instruct-tuned model based on the popular Mistral foundational model.
It's currently one of the highest performing models on the open-source LLM leaderboards
Dolphin 2.5 Mixtral 8x7b is a new uncensored model by
@erhartford
based on Mixtral, the new mixture of experts model by
@MistralAI
.
It was trained on a wide variety of datasets and is especially strong at coding tasks.
Fully-local AI Town!
AI Town is a virtual town where characters live and interact. It has wonderful visuals, is highly customizable, and can now run entirely on your local machine!
Run an AI Town locally, powered by llama3 🎉
No cloud signups needed. Make your own world, and then talk to it :)
Runs the open-source
@convex_dev
backend locally.
Use
@ollama
locally or
@togethercompute
for cloud LLM.
@realaitown
LLaVA 1.6 from
@imhaotian
has been released with improved resolution support, visual reasoning, and OCR capabilities, all while maintaining minimalist design and data efficiency.
Ollama countdown: Day 11
Mixture of experts models are now supported in v0.1.16!
Mixtral 8x7B:
ollama run mixtral
Dolphin Mixtral
ollama run dolphin-mixtral
Dolphin Mixtral is an uncensored, fine-tuned model created by
@erhartford
!
(48GB+ memory required)
Ollama Python & JavaScript libraries are here!
Both libraries make it possible to integrate new and existing apps with Ollama in a few lines of code, and share the features and feel of the Ollama REST API.
Learn more:
Ollama 0.3 with tool support!
You can now use tool calling with popular models such as Llama 3.1!
👇👇👇
Example tools include:
- Functions & APIs
- Web browsing
- Code interpreter
- and much more!
🧵 quick thread
.
@MistralAI
's Mixtral 8x22B Instruct is now available on Ollama!
ollama run mixtral:8x22b
We've updated the tags to reflect the instruct model by default. If you have pulled the base model, please update it by performing an `ollama pull` command.
Phi-2 is a new 2.7B small language model from Microsoft Research.
Trained on a highly curated dataset of 1.4 trillion tokens, this model's performance surpasses several 13B models in common sense reasoning and language understanding.
Zephyr 7B Beta is the second model in the Zephyr series.
Also based on Mistral, this iteration is fine-tuned on a distilled dataset making it even better at chat use cases. In many cases it can provide better responses than the much larger Llama 2 70B.
Stable Code 3B has been updated to a new instruct version, making it possible to use in conversations. Performance remains on par with larger models such as Code Llama 7B:
Dolphin Mistral 2.8 is a new version of Dolphin Mistral by the amazing
@erhartford
, fine tuned from the recent Mistral 0.2 model with support for a context window of up to 32k tokens.
StableLM Zephyr is a new chat model by Stability AI: the first of its type with the efficient size of only 3B parameters.
It has strong capabilities in generating contextually relevant, coherent, and linguistically accurate text.
Nous Hermes 2 is the latest model by
@Teknium1
and
@NousResearch
Based on the Yi 34B model, it's the highest performing in their "Hermes" series and excels in scientific discussion and coding tasks.
ollama run llava:34b
>>> What's in this picture? ./img_0064.jpg
This is an image of a graffiti on a wall. The graffiti features a blue whale with white fins, and the words "BUILD SHIP RUN LOVE" are written in white above it.
Llama 3 feels much less censored than Llama 2.
Llama 3 has a much lower false refusal rate compared to Llama 2 (less than 1/3). It's willing to discuss topics its predecessor wouldn't!
After seeing a post here by
@erhartford
, we tested a few prompts:
We tried manually prompting llama 3 to see how it fares against llama 2 on many basic safety questions.
It feels significantly less censored! So much better!
👀 quick read: 👇
Running
@MistralAI
's Mixtral 8x7b on your laptop is now a one-liner! Check out this post in which we show you how to use
@OLLAMA
with LlamaIndex to create a completely local, open-source retrieval-augmented generation app complete with an API:
Bonus: see
@steipete
I'm sorry this happened. It might have hit the context window limit in which case it will try to free up some context window – this works for most models but definitely not well for Llama 3. I've seen it too and am working on fixing it so it doesn't happen.
In the meantime you
Fast model downloads 🚀
Running local models involves pulling GBs of model weights, usually leading to a long wait time before a response.
Ollama 0.1.3 will now pull models in several parts simultaneously, significantly reducing time required to go from a new machine to a
🙌 Ollama for Linux is here! 🙌
👀 Nvidia GPU support now comes out of the box
💪 WSL2 will work with Ollama + Nvidia GPU acceleration.
👨💻 Ollama works on cloud servers!
Try it: 👇
Ollama can now be used to generate embeddings using LangChain.
What's great is the same in-memory model will be shared for generating both embeddings and completions.
This means for larger models: faster performance!
🦙 Ollama Local Embeddings 🦙
Search with local Llama-2 embeddings in
@LangChainAI
JS/TS 🦜🔗 0.0.146. GPU-boosted in minutes with
@jmorgan
's !
Incredible what you can do on a Macbook these days.
Thank you GH user Basti-an!
Langchain + Ollama!
Build apps that run entirely locally with Llama 2
Great for a ton of use cases like answering questions from local, private documents (see the example
@Hacubu
shared below) or building apps without incurring costs from cloud hosted LLMs.
🚨🏠Set up all-local, JS-based retrieval + QA over docs in 5 minutes in
@LangChainAI
JS/TS 0.0.124!
@TensorFlow
JS embeddings, HNSWLib vector store, and the last missing piece: GPU-optimized 🦙Llama 2 w/
@jmorgan
's
Use case + 🧵:
Stable Code 3B is a new 3B model with code completion results on part with Code Llama 7B.
It supports Fill in Middle Capability (FIM) making it a great model to use for code completion tools.
ollama run stable-code
Try Stability AI's Stable Code 3B model. Learn more:
Thank you
@StabilityAI
@ncooper57
@EMostaque
and team for creating the model!
Dolphin 2.2 Mistral: a new uncensored model by
@erhartford
that is enhanced with additional data from the Airoboros and Samantha projects.
Without assuming an identity of its own, this model is more empathetic and better handles long conversations.
This isn't Copilot 😱!
This is
@codegptAI
with your own llama, codellama or mistral model running locally with absolute privacy for your code
Available from version 2.1.28 onwards 🤩
Powered by
@LangChainAI
and
@Ollama_ai
Download the extension here:
Bonjour Ollama! Friends and Ollama are heading to Paris. If you are in the area, please join us for a developer meetup on Thursday, March 21st 6pm at Station F!
⭐️ Private chat w/ SQL using LLaMA2 ⭐️
LLMs can serve as a natural language interface to structured data in SQL DBs.
But, many text-to-SQL prompts rely on database schema / tables, e.g:
Open source LLMs, like LLaMA2, are a great way to unlock LLM+SQL
DeepSeek LLM is a new language model by
@deepseek_ai
available in 7B and 67B parameter counts.
This model has strong results in coding & math, and is trained on over 2 trillion bilingual tokens making it effective in both English and Chinese.
Today we're adding native AI support in
@supabase
Edge Functions
◆ Embedding models
◆ Large language models (powered by
@ollama
)
We've removed the cold-boot by placing the models inside the edge runtime and we're rolling out a GPU-powered sidecar.
See it in action:
💫 Scale quantized LLMs on your cloud/k8s with Ollama and SkyPilot!
📖 Use
@ollama
to run Mistral, Llama2-7B with just 4 CPUs and scale it up with SkyServe. Add GPUs w/ 1 line to make it faster!
💻 No cloud access, no problem - runs on your laptop too!
Run open-source LLMs on
@flydotio
with
@Ollama_ai
1. Download Ollama:
2. Run a model remotely on Fly:
OLLAMA_HOST= ollama run mistral
This is a demo instance, but sign up for the Fly GPU waitlist for your own!
sqlcoder is an open-source LLM that converts natural language to high-quality SQL queries, making it easy to query data for even complex schemas and questions
Run it locally with Ollama:
OpenHermes 2 Mistral 7B by
@Teknium1
A new fine-tuned model based on Mistral, trained on open datasets totalling over 900,000 instructions.
This model has strong multi-turn chat skills, surpassing previous Hermes 13B models and even matching 70B models on some benchmarks.
🦙 Ollama Local Chat Models 🦙
New in
@LangChainAI
: call local OSS models as chat models with
@jmorgan
’s !
Llama 2 13B gets 50 tok/s on an M2 Mac with < 5 minutes of setup.
S/o
@RLanceMartin
!
🐍:
☕:
Want to build your own local, open-source ChatGPT?
It's easy with
@Ollama_ai
!
Plus, you can run MANY models in parallel without spending a ton of $$ on GPUs.
Here's how:
[NEW LAUNCH] 0/ ✨AI-tamago🐣: A local-ready LLM-generated and LLM-driven tamagotchi with thoughts and feelings. 100% Javascript and costs $0 to run. 🧵
Stack:
- 🎮 Game state & reliable AI calls:
@inngest
- 🦙 Inference:
@Ollama_ai
(local),
@OpenAI
,
@__thetaphipsi
@steipete
Should be fixed now! Need to re-pull the model:
ollama pull llama3:70b
(or the model you used – e.g. ollama pull llama3)
Note: the download should be instantaneous since it's a small change to the runtime parameters. Sorry again this happened!!
As of version 0.0.19 Ollama supports running Falcon 180B with a single command:
ollama run falcon:180b
The smaller siblings work too: falcon:7b and falcon:40b
@rishdotblog
@dharmesh
@defogdata
@mchiang0610
The 7b version of sqlcoder is available on Ollama:
ollama run sqlcoder:7b
GPU accelerated on Linux & Mac (Windows coming soon :-) There's an API as well for building apps (incl Mac apps!)
@colhountech
@ollama
Sorry about this. Working on a fix. It seems to be from hitting the context length of the model. It can be increased by running `/set parameter num_ctx 8192` in with ollama run command.
If you're using the API, it's the `num_ctx` option.
Chat with local LLMs using a friendly user interface on macOS
@itsKGhandour
's Ollama SwiftUI is a native interface on macOS for downloading and chatting with LLMs, written in Swift and open-source
After getting exposed to
@Ollama_ai
, I decided to learn Swift over the past week and create a native user-friendly interface to be able to quickly chat with LLMs. It is now public, open source and waiting for your feedback!
Check it out now!
We’re now integrated with (
@jmorgan
) 🦙🎉
Our favorite part is the simplicity; do `ollama pull` and `ollama run` to run Llama 2, Code Llama, or other open LLMs.
Easily plug it into
@llama_index
RAG pipeline. Thanks
@husjerry1
! 🙌
LiteLLM: a lightweight python package for interacting with a variety of LLMs like OpenAI and Anthropic – taking care of input & output translation.
Now you can use LiteLLM with Ollama models too:
💥
@LiteLLM
x Ollama Integration live now
🦙Call your local llama2 models using chatGPT Input/Output
Use our proxy to get Caching, logging, error handling, 50+ LLM APIs (OpenAI, Azure, Anthropic)
Try it here:
We love the work being done
@jmorgan
on this
Local LLMs are incredible, but their current reach is just engineers. How do we change that?
I gave a
@GoogleAI
WebML Summit talk on how e.g.
@LangChainAI
&
@ollama_ai
enable client-side AI in web apps.
The punchline: we need a new browser API!
(1/9)
@visheratin
@erhartford
Yes! It works today! Check out
Congrats on getting this model to such an impressive size. How similar is this to the previous Llava 7b/13b models?
@erhartford
@bartowski1182
Sorry you hit this, it will be fixed in the next release of Ollama.
The pretokenizer change wasn’t backward compatible and new converts may not run on older versions of llama.cpp