Homebrew Research
@homebrewltd
Followers
567
Following
272
Media
30
Statuses
85
Local AI. Homebrew is an R&D studio built 👋 @jandotai , 🤖 @cortex_so , and 🍓 Ichigo
Singapore
Joined May 2024
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#Perfect10Liners
• 101034 Tweets
WONWOO
• 91001 Tweets
JENO FIRST PITCH
• 64059 Tweets
DeNA
• 46543 Tweets
राष्ट्रीय स्वयंसेवक संघ
• 43700 Tweets
Pilar
• 40788 Tweets
Hispanidad
• 38934 Tweets
#chibalotte
• 38521 Tweets
#baystars
• 30813 Tweets
FINAL SEASON
• 28127 Tweets
#最後の青汁劇場
• 26762 Tweets
ホームラン
• 26529 Tweets
SVT RIGHT HERE IN GOYANG D1
• 23372 Tweets
JAMESSU WITH ADDA
• 17226 Tweets
Viva España
• 16029 Tweets
キングオブコント
• 14413 Tweets
6TH MILKLOVE FANSIGN
• 13996 Tweets
オースティン
• 10610 Tweets
ベイスターズ
• 10577 Tweets
Your Sky Series Q23
• 10065 Tweets
Pinned Tweet
Our homebrewed early-fusion speech model now has a name and a voice.
Say hi to 🍓 Ichigo, the local real-time voice AI.
Next up: You can try Ichigo soon on
@huggingface
&
@Gradio
.
9
23
155
Control AI with your voice.
The upcoming Llama3-s checkpoint introduces voice-based function calling. It's an early-fusion model built to give
@AIatMeta
's Llama 3.1 model listening capabilities.
Coming soon to
@huggingface
&
@Gradio
.
4
40
166
Llama3.1 just got ears with llama3s! 🦙👂
We're teaching Llama3.1 to listen - an open, ongoing experiment with
@AIatMeta
's llama3.
This v0.2 can understand human speech, but it's in its early days with limitations/bugs.
1
20
53
Latency? 🍓 Ichigo is a local real-time voice AI with <150ms.
7
4
43
Introducing 🖖 Homebrew.
We started as an open-source team working on Jan, a personal AI.
Along the way, we encountered challenges in product, software, hardware, and even data center levels. These challenges have inspired to us think bigger to find better solutions.
4
8
30
We're using
@AIatMeta
's Chameleon approach in our early-fusion speech model. Here's why you should too:
Context: We're training Llama3-s publicly. Llama3-s is an early-fusion speech model, that extends Llama3 with native listening capabilities.
🧵1/5
1
7
21
How we're training llama3-s in the next phase
We're publicly training Llama3-s, an early-fusion speech model that extends
@AIatMeta
's Llama3 with native listening capabilities.
Currently llama3-s v0.2 works ok in a quiet environment.
Demo:
But it very
0
8
20
If you have at least 12GB VRAM and you're running Llama3.1 at Q4, you’re over-quantizing.
This is a quick comparison from our researcher
@pokachi2023
on max VRAM utilization in practice, relative to the actual context size you are getting from your LLMs.
- Typically, Llama3.1
0
8
17
Faster open-source voice AI processing?
llama3-s vs. traditional cascaded (transcription-first) approach:
- 13x faster processing (skips transcription step)
- Efficiency gap widens with longer audio
- More responsive real-time interactions
1
3
11
We're bringing AI companies together the day before Tech Week Singapore ❤️
Join us for talks & chats:
Talks line-up (so far):
- Jenni AI,
@Calclavia
- Test with users' AI twins,
@carboncopies_ai
- AI Neoclouds and 300k GPU Megaclusters,
@dnishball
-
1
1
11
We're starting an experiment to train llama3-s to have native listening ability! 🦙
- Using sound compression for discrete audio token representation
- Early fusion of audio and text in a multimodal model
- Training on synthetic audio data
Learn more:
1
1
9
👉👈
Introducing 🖖 Homebrew.
We started as an open-source team working on Jan, a personal AI.
Along the way, we encountered challenges in product, software, hardware, and even data center levels. These challenges have inspired to us think bigger to find better solutions.
4
8
30
0
1
9
To all our users and contributors: thank you 🙏
Our team is grateful for your bug reports, criticism, ideas, and feedback.
We’re continually improving, and excited to ship new products in the months ahead.
1
0
8
At our core, Homebrew’s culture is driven by who we are: AI enthusiasts who want to build practical, useful tools for ourselves and others.
We are tinkerers: always excited about new innovations, and relentlessly finding ways to improve our product.
1
0
8
We're bringing AI companies together one day before TechWeek Singapore!
📅 8 October, Tuesday
🕒 6.30 pm
📍 National Library Board, Level 7
Lineup so far: Jenni, CarbonCopies, Pints AI, Jan, Cortex, Ichigo.
Register here:
0
3
8
Take control of your tools and keep your data where it belongs - with you.
Self-hosting is the future.
Mind boggling to me how many people are using home cooked /self hosted tools for this.
9
0
44
1
0
7
Feel free to brew a cup of coffee while we work on integrating an AI model into Jan that can understand human speech.
1
1
7
Today, we’re becoming 🖖 Homebrew, an AI research lab.
We believe deeply in AI’s potential to push the human race forward, but recognize there are difficult infrastructure problems that require time and determination to solve.
1
0
7
Every Jan update makes the experience faster and smoother. With v0.5.4, you'll see AI running quicker on your CPU.
Release v0.5.4 is out: Jan is faster now 👋
Highlights 🎉
- Faster CPU inference
- Direct model downloads in Threads
- Consistent performance warnings
Fixes 💫
- Phi-3 model issues resolved
- Persistent thread settings
- UI and bug fixes
Update your version or download the
2
5
11
0
0
6
At Homebrew, what we’re working on:
- Software: 👋
@jandotai
personal AI, and 🤖
@cortex_so
, a local AI engine
- Research: Practical models that solve problems (and the datasets that power them)
- Hardware: how do we make AI energy-efficient?
1
0
5
Huge thanks to OpenSLR,
@PyTorch
's Torchtune, and the teams behind multimodal architectures, Whisper,
@Collabora
's WhisperSpeech, AudioBench, and
@AIatMeta
's Chameleon. Your work on warmup mechanisms was crucial 🩵
Special shoutout to contributors on our Discord and r/LocalLLaMA
1
1
5
What if this 🍓 speaks to you?
0
0
5
@waterdoggie
@victormustar
@huggingface
@Gradio
yeah, it’s because it's on a server now for testing with all team members. Check out this video:
Latency? 🍓 Ichigo is a local real-time voice AI with <150ms.
7
4
43
0
0
4
It’s also an open call for LLM researchers & audio experts!
Join our "throw sh*t at the wall" approach on Discord.
💫 The methodology, results, and relevant links on our website.
1
0
4
With llama3s, llama3.1 can understand human speech.
llama3s v0.2 performs well on speech understanding benchmarks (check our website to see results). More analysis needed, but we're excited to share & get feedback.
Try the demo on 🤗
@huggingface
or build from scratch.
1
1
4
In the multimodal topology, Chameleon adopts a simple and elegant architecture:
- Various modalities (images, video, audio) are tokenized
- These multimodal tokens are directly fed into a single transformer
- Thus the encoder can be decoupled from the decoder, and trained
1
1
4
How Chameleon does the Training:
Pretraining was done on a mixture of data, from text only, to interleaving image/text data
In the second stage, higher-quality training data is introduced
Post-training alignment was done through SFT and human evals.
🧵4/5
1
1
3
How Chameleon tokenizes images:
At the core, Chameleon used a VQ-GAN, which compresses non-overlapping patches of the image into discrete representations (tokens)
Then, more loss functions are added to focus on the quality of specific image regions like faces and subject matter
1
1
3
💫Resources & awesome researchers who inspired us:
For the Chameleon paper, thanks to
@ArmenAgha
,
@LukeZettlemoyer
,
@gargighosh
,
@scottyih
,
@real_asli
,
@VictoriaLinML
,
@violet_zct
,
@liliyu_lili
,
@omerlevy_
and all contributors!
For The Evolution of Multimodal Model
0
1
3
0
0
2
Results 👇
Its image-to-text capabilities were comparable to SOTA.
And that’s it! Hope you enjoy the paper as much as we did. You can find the original paper here:
🧵5/5
1
0
2
@sbeastwindy
@huggingface
@Gradio
As long as we could still edit the tweet, no opportunity was lost...
we've updated it, thanks a lot!
0
0
1
@tr_twtr
@MacHomebrew
Hey, thanks for the suggestion!
We've carefully considered our name choice:
- The name reflects our DIY innovation spirit, inspired by the Homebrew Computer Club.
- We respect
@MacHomebrew
and we're not worried about the mix-up with them. It'll become an IFKYK.
0
0
1
0
0
1
Event: AI Training: From Pytorch to GPU Clusters
@hellogabbo
,
@nczhu
and
@onggunhao
from Homebrew will break down the end-to-end process of AI training across different stack layers - from software to GPUs, multi-clusters, and data centers.
📅 12.09.2024, Thursday
🕒 7 pm -
0
0
1
🔊 Join us in developing this open-source speech adapter for llama3-s.
Contribute via Discord:
0
0
1