🥳🛠️Introducing ToolBench!🤖🎉
🌟Large-scale instruction tuning SFT data to equip LLMs with general tool-use capability
🔖 We release 98k data with 312k real API calls. We also release a capable model ToolLLaMA that matches ChatGPT in tool use
Github:
🚀 Introducing XAgent: The next evolution of AI agents designed for intricate task-solving. XAgent completely outperforms AutoGPT and GPT-4 on various tasks and benchmarks.
💡 XAgent's dual-loop mechanism bridges the gap between high-level planning and detailed task execution.
It's Time to Scale Down the Data
LLaMA3 revealed a grim reality: without changing the model architecture, increasing the data from 2T to 15T can forcefully produce miracles. This tells us that, in the long run, foundational models are only an opportunity for big companies. On
What is the Vision-Language Model (VLM) field researching?🧐
VLM is a rapidly developing field since the end of last year, with a large amount of "gold mines" yet to be discovered for researchers. The current exploration is still very preliminary, and it is relatively easy for
🥳Successfully defended my thesis! 🎓✨
It's been an incredible journey at Tsinghua Univ. Thank you to everyone who supported me along the way!
Here’s to new beginnings ahead!
#ThesisDefense
#PhDone
Is post-training the original sin of LLM performance drop, not quantization?
It is widely acknowledged that quantization leads to a significant loss in model performance, especially for tasks that require high reasoning abilities. Quantized LLMs (even after post-training) are
Thanks for sharing🥰!
Highlights of our work:
1 🛠️16000+ real APIs from RapidAPI
2 🤔Better reasoning strategy (DFSDT) for LLMs than CoT
3 🤖ToolLLaMA achieves comparable performance than turbo-16k
4 🔍An API retriever
...
Codes, models, demo are out:
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
paper page:
Despite the advancements of open-source large language models (LLMs) and their variants, e.g., LLaMA and Vicuna, they remain significantly limited in performing
🚀Plenty of adaptation methods have been proposed for tuning PLMs, and all of them can find high-performance minima.
🧐However, what is the connection among various minima reached under different adaptation configurations?
🥳Check our paper on EMNLP 2022!
#NLProc
Thanks for sharing🥰! Highlights of our work:
(1) 🛠️16000+ real APIs from RapidAPI
(2) 🤔Better reasoning strategy (DFSDT) for LLMs than ReACT
(3) 🤖ToolLLaMA achieves comparable performance than turbo-16k
(4) 🔍An API retriever
(5) 🏁ToolEval for auto-evaluation
...
Knowledge Inheritance for Pre-trained Language Models
pdf:
abs:
github:
pre-training framework, knowledge inheritance, combines both self-learning and teacher-guided learning to efficiently train larger PLMs
📌📌📌
Humans possess an extraordinary ability to create and utilize tools 🪛🔨🔧
With the advent of foundation models, does AI🤖 have the potential to be equally adept and capable as its creators? 🧐
Check out our survey on
#ToolLearning
📜
Paper:
How Far Are We from the Ideal AutoGPT? [1/4]
AutoGPT has already garnered 163k stars, and its developers have been working on it for over a year. However, it remains in the demo stage and can't be considered a fully-fledged product (even for developers). This trajectory differs
Heading to Vienna tomorrow for
#ICLR2024
, will be presenting our ToolLLaMA on 5/8 4:30 pm. Feel free to grab a coffee for (1) LLM agent, (2) GUI automation (VLM), and (3) repo-level coding/debugging.
@tianle_cai
@Google
Very interesting and inspiring work! We've also done similar things (i.e., investigating LLM's tool creation capabilities) recently, check out:
Amazing experience using
@AdeptAILabs
Adept Experiments. Glad to see Adept first introduce the concept of AI workflow in the industry🥳
Actually we've also released a paper about the next generation of RPA -> "Agentic Process Automation" (APA). Personally I believe APA (autogpt
Why could different parameter-efficient tuning methods with distinct structures achieve comparable performance to fine-tuning? Are they intrinsically connected? Check our recent work on Findings of
#emnlp2022
🎇
#NLP
Thanks for sharing!! We've also created a must-read paper list on tool learning, and would be continually updating it in the future.
#ToolLearning
Link:
Just discovered ScrapeGraphAI - a Python library that makes web scraping a breeze! 🕷️
You tell it what info you want, and it builds the scraping pipeline for you.
Perfect for devs who hate writing complex scraping logic.
Check it out if you're into AI-powered data
Wanna tune few-shot NLP tasks with 5 free parameters😉?
Check our recent preprint: Exploring Low-dimensional Intrinsic Task Subspace via Prompt Tuning. We explore how can PLMs effectively adapt to broad NLP tasks differing a lot superficially.
I think AI agentic workflows will drive massive AI progress this year — perhaps even more than the next generation of foundation models. This is an important trend, and I urge everyone who works in AI to pay attention to it.
Today, we mostly use LLMs in zero-shot mode, prompting
Large models have always struggled with "counting" 😔
For example, tasks like "having the model continue writing for 137 characters" or "counting the occurrences of 'he' in this passage" perform very poorly without showing the CoT, and it doesn't seem like scaling can solve
News from our XAgent Team:
🥰XAgent on DockerHub: Official release of XAgent Container Images now available! 📷 Check it out.
🥳Localhost Models on HuggingFace: XAgentLlama-7B and 34B models are live! 📷 Explore here.
😊XAgentGen Released: Enhance your experience with our new
📢Release our work on agent self-evolution!
We propose the ICE experience learning strategy that makes agent deployment significantly more effective and efficient.
ICE is also integrated into XAgent 2.0 about to be released soon!
Find out more here:
🎉Thrilled to introduce our latest work, Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models. We perform a comprehensive theoretical analysis and experimental validation of the parametrically efficient paradigm for
#PLMs
.
New Work Released🥰!
We combine ELO rating system 🎮into autonomous decision making, which significantly improves the efficiency and effectiveness of machine tool use.
We just released mistral-finetune, the official repo and guide on how to fine-tune Mistral open-source models using LoRA:
Also released Mistral-7B-Instruct-v0.3 with support for function calling with Apache 2.0 license:
🚀 Introducing XAgent: The next evolution of AI agents designed for intricate task-solving. XAgent completely outperforms AutoGPT and GPT-4 on various tasks and benchmarks.
💡 XAgent's dual-loop mechanism bridges the gap between high-level planning and detailed task execution.
Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
Trains a powerful model that proactively assesses task vagueness, inquires user intentions, and refines them into actionable goals before starting downstream agent task execution
repo:
🚀 Excited to introduce MiniCPM-Llama3-V 2.5! With 8B parameters, it’s our latest breakthrough, outperforming top models like GPT-4V. 📈
💪 Superior OCR capabilities
🔑 Supports 30+ languages
HuggingFace:
GitHub:
Many long-context methods have been proposed recently. Despite their superficial differences in the design, most of them can be unified through the lens of memory augmentation under one framework (i.e., they are actually the same!!). See our new work UniMem!!🥳
Introducing UniMem: A unified framework redefining long-context processing in large language models (LLMs)!
🌟 By integrating a spectrum of long-text strategies within a memory enhancement framework, UniMem harmonizes these techniques across four key dimensions: memory
ProAgent: From Robotic Process Automation to Agentic Process Automation
paper page:
From ancient water wheels to robotic process automation (RPA), automation technology has evolved throughout history to liberate human beings from arduous tasks. Yet, RPA
Scanning AI papers on arxiv every day is very troublesome. We have made a website to facilitate everyone to match the most relevant/highest quality papers. If you are interested, please give us some feedback. We will quickly optimize.
🔥NexusRaven-13B, a new SOTA OSS LLM for function calling.
📊Match GPT-3.5 in zero-shot for generic domain and beat it by 4% in security software.
🛠️Outperform GPT-4 by up to 30% with retrieval augmentation for software unseen during training.
Blog: (1/3)
Enabling LLMs with tool-use capabilities is where I am noticing the greatest potential for companies to go big with LLMs.
Gorilla is a good popular example but I have seen a ton of other examples, especially from people building with AI-powered agents. I also think this is one
All the data is automatically generated by OpenAI API and filtered by us, the whole data creation process is easy to scale up. We built ToolBench based upon BMTools:
今天调GPT-4o的API做个QA测试,发现回答的内容里有前几天刚发的yolo-v10,所以GPT-4o的API已经默认把GPT4All-tools merge进来了嘛?
Today, while testing the GPT-4o API for QA, I found that it referenced the recently released YOLO-v10. Does this mean the GPT-4o API has already integrated
Many long-context methods have been proposed recently. Despite their superficial differences in the design, most of them can be unified through the lens of memory augmentation under one framework (i.e., they are actually the same!!). See our new work UniMem!!🥳
✨ Features of ToolBench
1. Multi-step tool invocations and diverse scenarios like weather inquiry and PPT automation
2. Support both single-tool (LangChain) and multi-tool (AutoGPT) settings
3. Include model's inner thought process
4. Diverse real-world tools and APIs
Whispers in the machine... 🤖🌌
ChatTTS emerges from the digital shadows. A voice synthesizer unlike any other. It laughs. It pauses. It speaks in tongues. 100,000+ hours of whispered secrets.
Dare you give AI its voice?
#ChatTTS
#AIAwakens
#DigitalWhispers
Link:
After 5 month dedicated work from >15 researchers & developers, we're thrilled to introduce 🚀OPEN-SOURCE language model Agents🚀!
Try demos: 🥑
Stay tuned for open-source code, model, framework, evaluation & more at !
Introducing Claude 3.5 Sonnet—our most intelligent model yet.
This is the first release in our 3.5 model family.
Sonnet now outperforms competitor models on key evaluations, at twice the speed of Claude 3 Opus and one-fifth the cost.
Try it for free:
Pokémon: In the game, agents can visit shops, train their Pokémon at the gym, and interact with one another. As a player, you take on the role of an agent and engage with others at any time. There are 6 characters in the Pokémon environment who appeared in Pokemon Emerald. 6/n
@_florianmai
@gg42554
Same here. My March ARR submission (4, 4.5, 2.5, meta 4) only gets into Findings of EMNLP, with 2.5 dude giving ridiculous reviews. It seems PCs are too busy scanning the submission. Not sure whether ARR is worth considering anymore. 😞
We strive to investigate three facets: (1) the effects that affect PLM's mode connectivity, (2) the change of mode connectivity during pre-training, and (3) the change of task knowledge along the path connecting two minima.
Paper:
In the found unified subspace, the minima of all DETs have excellent transferring performance, forming a low-loss / high-performance manifold in the parameter space. Such a phenomenon indicates that DETs are mode connected intrinsically.
#mode_connectivity
From the perspective of parameter space, PLMs provide generic initialization, starting from which high-performance minima could be found. For the first time, we investigate the geometric connections of different minima through the lens of mode connectivity.
To fathom the connections among various delta tuning (aka parameter-efficient tuning) methods (DETs), we hypothesize that the adaptations of different DETs could all be reparameterized as low-dimensional optimizations in a unified optimization subspace.
@LChoshen
@jang_yoel
Merging adaptors actually works, but not that well compared with full-parameter fine-tuning. To attain good linear connectivity, it is always better to use the same initialization. Also, finding a merging point using a non-linear path is another choice.
LLM Agent for Large-Scale API Calls
Cool research paper presenting AnyTool, an LLM-based agent that can utilize 16000+ APIs from Rapid API.
Proposes a simple framework consisting of
- a hierarchical API-retriever to identify relevant API candidates to a query
- a solver to
@omarsar0
Temporarily, folks could use our dataset, which supports real-world APIs, and multi-step, multi-tool processes with reasoning traces and real tool executions. 100k+ SFT data released at