Elicit
@elicitorg
Followers
13K
Following
1K
Media
113
Statuses
1K
The AI research assistant https://t.co/04vRTWtqEA • Demos https://t.co/IqWcqcVn1Z • Jobs https://t.co/6Lxhl2kNu7
San Francisco, CA
Joined October 2020
🚨 We completely overhauled our pricing! We now offer an UNLIMITED FREE TIER that is the most generous of any research tool. Searches, summaries, and chats are completely free and completely unlimited, every single month. We also launched Elicit Pro, a new plan specifically
6
11
70
1/ Announcing our spinoff from @oughtinc into a public benefit corporation, our $9 million seed round, and a much more powerful Elicit!. This new Elicit takes the components of the popular literature review workflow and extends them to automate more research workflows.
14
68
311
Introducing Elicit Notebooks! . Notebooks take your research broader & deeper, all on one page. You can now:.- Run multiple queries on the same page.- Combine papers across the Elicit database & your uploads.- Combine papers across queries.- Chat with papers.- Select papers to
7
21
123
Language model-based tools are new technologies with exciting potential but some current limitations. This is V0 of sharing how Elicit works & where it doesn't so that researchers know how to integrate it into their workflows and what to expect from it.
2
22
111
We’ve completely overhauled Elicit’s semantic search!. Quality has almost doubled in some cases - Elicit search results will now contain 78% more relevant papers, if you are searching for papers for a systematic review. Across all Elicit searches, you will find 40% more relevant
3
13
79
Elicit is surging in popularity!. We've just had a big influx of new users in the last couple of hours 👋. Unfortunately, that means we've started hitting our rate-limits with some underlying service providers, causing slowness and some timeouts.
2
2
56
1/ Elicit now uses a custom-built semantic search engine that searches over 175 million titles and abstracts. This engine finds papers that are relevant to your query even if there is no keyword match. (The previous model was a keyword-semantic hybrid).
1
8
42
Elicit Plus now uses tables from papers for viewing & extracting data!. 🧵
1
8
36
While helping teams with systematic reviews and data extraction, we compared Elicit's accuracy to manual work by trained research assistants and staff (whatever the status quo was). For a fraction of the time and cost, we were able to significantly improve accuracy. We're hoping
1
3
34
One underrated workflow in Elicit: build out a literature review starting from 1 paper. Instead of copying & pasting references or dealing with tab explosion as you're reading, you can collect a bunch of papers, then organize your "read next" list.
3
6
34
We're collecting datasets that researchers have tagged – like categorizing survey responses or types of AI orgs – to make a new NLP benchmark. If you want to advance ML research for tasks people care about and have a dataset to share, let us know!
0
7
33
🚨 Exciting news! We just launched the biggest shift in how you can use Elicit with your colleagues and friends to date. Introducing: Sharable Notebooks! 🎉.
1
8
31
To make sure Elicit delivers the most trustworthy research results, we must constantly evaluate it, and we must do that *at scale*. In this post @etiennefd describes how we evaluate LLM output, the tradeoffs between trust, scale, and flexibility and how we find our way around
7
7
31
We're continually working to put new search technologies to work for you. Our infrastructure engineer @VivaLaPanda_ walks through many of the traditional and machine learning approaches to semantic search in our latest blog post:
0
5
30
Lastly, we're thrilled to be adding @fiftyyears @BasisSet @Julian @MythosVentures, Illusian, @arashf @mojombo @blakeweb @JeffDean @noahdgoodman @bentossell @labenz and other incredible angels to our team.
0
2
29
2/ You can now upload up to 100 of your own papers to extract data from. Great for automating the screening or extraction steps of systematic reviews and meta-analyses.
1
2
26
🚨 We’ve substantially improved Elicit’s answers across the app:..- Summaries of top papers now have 78% fewer errors.- Abstract summaries better follow your query with 88% fewer errors .- Standard custom columns now have 37% fewer errors than before..All the features above.
1
4
27
Can AI help with systematic reviews?. A team at Australia's national science agency @CSIRO found AI tools are getting better than human researchers at data extraction, the most laborious step in systematic review. Elicit outperformed ChatGPT. Details in 🧵:
2
7
25
4/ The most computationally intensive new workflow is "List of concepts.". "List of concepts" searches across 200MM+ papers, finds examples of what you're looking for in each paper, then groups papers by concept. You can create lists of:.- Datasets for medical imaging
1
1
24
Together with @huggingface we're launching the RAFT benchmark to test the few-shot capabilities of language models on real-world tasks 📊. If you perform well, we may implement your solution in Elicit to help researchers! . Find out more at
1
6
24
Bet suggestion: If Biden gets between 328 and 382 electoral college votes, @FiveThirtyEight should pay @TheEconomist $6400. Otherwise, @TheEconomist should pay @FiveThirtyEight $3600. How did we generate this bet? 1/5.
1
6
18
Introducing High-accuracy mode in Elicit! Data from papers can now be extracted at half the error rate and with much more customization.
1
5
18
3/ The papers you upload are saved in your library (only visible to you) so that you don't have to keep uploading them. Eventually, you'll be able to upload 10,000+ papers, run more operations on them, organize into projects, and share collections with teammates.
1
1
18
1/ You can now add natural language filters in your Elicit queries! If you try a query like . "papers about the benefits of taking l-theanine published after 2020" . Elicit will automatically filter to papers published after 2020.
1
7
18
What's new in semantic search and retrieval?. Informal tech talk by @elicitorg ML engineer Justin Reppert
1
3
17
7/ There is so much left to do! . Help us build intuitive and general interfaces that can run language models at high accuracy and superhuman scale to automate important research -
1
2
15
- and a particularly exciting one: findings & ideas that generalize across domains. From the earliest days of Elicit, researchers have been begging us to help them translate across domains
1
2
15
New tutorial explaining how to use Elicit to screen papers & extract data for systematic reviews and meta-analyses!
1
3
16
One of our primary goals is to make quality research faster while exceeding human levels of accuracy, so we're especially excited about some recent quality improvements to Elicit: . - Data extraction in columns without high-accuracy enabled is now 15% more accurate.- Using
1
1
14
In the last few weeks, the @PredictIt price for voter turnout > 160M has almost doubled. If you want to get alerts when markets change, set custom PredictIt notifications at
0
5
13
- Machine learning techniques, like reducing hallucination in language models
1
1
14
Want a peek under the hood on how Elicit's magic works?. 𝗟𝗶𝘃𝗶𝗻𝗴 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 𝗮𝘀 𝗮 𝗨𝗫 𝗽𝗮𝘁𝘁𝗲𝗿𝗻 𝗶𝗻 𝗔𝗜. It's about systematic review, and the design challenges of using AI to create an editable document with everything science knows.
1
2
14
At Elicit, we're advancing transparent and systematic approaches to AI so that we can improve the quality of reasoning in science, policy and beyond. Now Elicit is collaborating with the National Institute of Standards and Technology (NIST) in the Artificial Intelligence Safety.
0
5
13
We've added a filter for journal quality to help you restrict your searches to only top-ranked journals any time you want. Simply move this quality slider to gradually narrow in on higher ranked journals, quartile by quartile. 🙌
1
3
14
“Elicit is a step above other tools I've tried. I prefer Elicit when it comes to actually interpreting evidence. It doesn't make things up like ChatGPT.” - James Compagno, Director of Marketing. MicroGenDX is a clinical laboratory that offers microbial diagnostic tests via.
0
0
13
We added 19 more columns that you can use to extract details from over 200 million academic papers, or any PDF that you upload! . These will work much better for fields like:.- Social science.- Economics and business studies.- Computer science
1
3
13
Elicit now lets you filter papers by any criteria, including effect direction, population studied, technique used, and more!. Add a column and filter by keywords mentioned in the column. Learn more about filtering with columns here:
1
6
13
6/ You’ll still get the Elicit you know and love in the "Find papers" workflow!
1
1
12
New in Elicit: Summary columns, including for uploaded papers! Now you can summarize any paper that you upload to Elicit. We also added "Summary of introduction" and "Summary of discussion". "Summary of introduction" can show how a paper fits into the literature on a research
2
2
12
It's interesting how much narrower and peakier @TheEconomist's prediction for Dem Electoral College votes is compared to @PredictIt, @metaculus, and @FiveThirtyEight .
0
5
11
Our head of engineering @james_elicit outlined how we hire for AI engineers, from skillset to hiring process. On the latest @latentspacepod, he discusses the surprising chaotic qualities of language models and how pre-AI chaos engineering skills can play a role.
0
3
12
Elicit for Health Economics & Outcomes Research (HEOR)! . Use Elicit to estimate the market for a health intervention, costs, burden of illness, and more. Search through 125 million public papers (including PubMed) and extract details about datasets used, study methodologies,
4
3
12
1/ Last week, we launched question & answering over the full text of open access papers. This query extracts the inclusion criteria from inside a systematic review. Clicking on the cell shows the source from inside the paper. You can also flag mistakes.
1
0
10
"Powerful generalizable language-based AI tools like Elicit are here, and they are just the tip of the iceberg; multimodal foundation model-based tools are poised to transform business in ways that are still difficult to predict." - @rossgritz .
0
3
11
This week's updates:. 1. Add "DOI" & "Detailed study design" to your table. "Detailed study design" is useful for identifying methodologies beyond randomized controlled trials, reviews, and meta-analyses (shown in "Study type").
1
5
11
1/ We've integrated the "Brainstorm Research Questions" task with the "Lit Review" workflow as suggested queries!. Brainstorm Research Questions is the second most popular task in Elicit, after Lit Review. Many users were already copying research questions into Lit Review.
1
3
10
You can skip steps 2-6 by doing this directly on the home page! . Click on the button that says "Brainstorm questions for. " underneath the search bar. @MushtaqBilalPhD.
In the search bar, type in the question you just selected.
1
3
10
If you're curious about how Machine Learning and LLMs work--it's not magic or the robot uprising!--we've compiled a comprehensive reading list that we share with new Elicit employees. We find the Applications in Science section particularly interesting:
0
0
10
The Elicit Basic plan is now completely, indefinitely free. It’s designed for students and casual users. You can run as many searches as you want, create as many summaries of papers as you want, and chat with papers as much as you want. You can try some data extraction.
1
0
9
In addition to more relevant results, you can now run advanced searches using 14 filters. That means you can search based on:. • Title.• Journal.• Author.• Keywords.• PubMed ID.• DOI.• Publication year.• Open access status.• Citation count. And more. Here's an example
1
0
9
We got to launch one of our most requested features in Elicit this morning: Custom tags for your uploaded PDFs. Starting today, you can now create and add tags for uploaded papers in your library. Group papers by project, category, favorites, reviewed status, step, or anything
1
1
8
We've been experimenting with some interesting use cases for Chat with Papers:.-How papers relate to each other.-How authors define certain terms.-General context about what's being discussed in the papers.-What a single paper did in detail.
0
1
9
Spillias et al. compared 3 AI approaches (Elicit, 1 call to GPT-4-Turbo, multiple calls to GPT-4-Turbo) to a human baseline on data extraction for a review about managing fisheries. Elicit consistently outperformed the GPT-4 implementations (p < 0.001) and had a near-zero
1
1
9
So what we need is automated evaluation, but with human supervision. Benchmarks are a way to achieve this, but they’re not easy to adapt to specific tasks. Instead we use LLM-based auto-evals with human supervision. We come up with an eval process, where we grade typical outputs.
1
0
9
Elicit can be your best friend in sourcing a large body of papers for your research. Search citation trails, filter results based on your inclusion criteria, and run multiple queries quickly to broaden your results!
0
1
7
We're working to urgently increase those limits and/or find workarounds to get our user experience back to where it should be. Our apologies for any disruption – we'll update our status page as we work through the problem.
0
1
8
Over 150 students and researchers in Indonesia are now using Elicit to support their research thanks to this incredible presentation by @manda_ngo .
1
2
8
The CSIRO team knew when the output was AI-generated, but were blinded to the specific AI implementation they were grading.
1
0
8
From forecasting weather to searching for new materials for batteries and solar panels and controlling nuclear-fusion reactions, AI is making research faster, better, and more productive
0
2
8
For the full list of filters and how you can use them to control your search, check out this post: And kudos to Justin Reppert, @BenRachbach @VivaLaPanda_ for leading this massive project!.
0
0
8
We’ve found that auto-evals using the latest LLMs allow us to quickly iterate at scale, without sacrificing too much trust or flexibility. But this isn’t easy and there are always challenges! We must constantly refine our evaluation process as we refine Elicit’s features and.
6
0
7
In one Elicit Notebook, you can: .-Find papers.-Extract data from PDFs.-Generate a list of concepts across a topic.-Narrow your results with filters.-Chat with papers.-Summarize multiple abstracts.-Create custom columns. Start building a Notebook for free!
1
0
7
New episode on @latentspacepod for those who are interested in what's under the Elicit hood.
Cost v Reliability v Performance: Choose Three!. 🆕 pod: w/ @jungofthewon and @stuhlmueller . How do you improve prompt engineering and ai agents? The emerging paradigm is sometimes called "Flow engineering", but @elicitorg has reinvented the Notebook, in
1
1
7
"How do we as a society want to turn compute into more work? One answer is we train larger and larger models and hope for the best. We are working towards a different answer." . New episode on The Cognitive Revolution digging into the Elicit product philosophy and architecture!.
In this episode of The Cognitive Revolution: @NathanLabenz sits down with @elicitorg co-founders @stuhlmueller and @jungofthewon to discuss their mission to make AI-assisted research more accessible and reliable. Learn about their unique approach to task decomposition, which
0
3
7
Elicit Plus is for serious researchers like graduate students and professors. You get everything in Basic but over more papers. You can also extract data from 300 papers at once if you get the annual plan. (The monthly plan splits the 300 across 12 months.) . If you want to
1
0
7
Coming soon: We’re launching binary predictions on Elicit! You’ll be able to make questions, predict on others’ questions, compare predictions, share groups of questions, track updates, and see your calibration. Sign up to get notified when we launch -->
1
1
7
Since this study was conducted (January) all of these AI approaches have improved significantly. We’ve reduced extraction errors by more than 50% and made extractions easier to check. e.g. Elicit supports each claim (left) with quotes from the original paper (right).
1
0
7
What does this mean for systematic reviewers?.1. Save time: We estimate you can now extract data in 70% less time vs. doing it manually. 2. Work iteratively: You can now work iteratively, effortlessly re-running extractions upon uncovering new questions/considerations without.
1
1
7
1/ We added 3 new columns in Elicit: funding source, limitations, and question-relevant summary. Funding source & limitations usually need the full text of papers so filter for "Has PDF" to see results for these columns. Try in Elicit:
1
0
7
Elicit Pro is an entirely new plan designed for researchers extracting data across a large number of papers, typically for systematic reviews. With Pro, you can extract data from up to 1200 papers at any point over the course of the year. The annual version of this plan is
2
1
7
Elicit's main limitation was sometimes providing extraneous information, which researchers still needed to sift through and delete. Fortunately, that's way less work than having to re-read the original paper.
1
0
7
Welcome to the team @k1bird!.
2024 is off to an amazing start because @k1bird joined us this week as Head of Product! . As the first PM at Nubank, Kevin deeply understands hypergrowth and building a transformative product that users love. Kevin helped scale Nubank from 45,000 to 80 MILLION users. He helped
0
0
5
We’re excited to see independent evaluations of AI tools popping up! Check out the preprint yourself here: & watch this video for instructions on how to use Elicit for systematic review
0
1
6
Evaluation is the process determining the quality of LLM output. But there are tradeoffs. At one extreme, you can have experts look up all outputs: high trust, but abysmal scalability. At the other extreme, you just let LLMs evaluate their own outputs.
1
0
6
If you export a table from Elicit, it'll include not just the citation information for papers but also any columns you added to the table, including abstract summaries, main findings, limitations, and more. Even custom columns!
1
0
6
New in Elicit! .1. Delete notebooks.Once you have a Notebook open, go to the top left and click on the three dots next to the Notebook name. This will declutter your Notebook history. 2. Delete notebook steps.For each step in a Notebook, click on the trash can button next to the
0
2
6
Each tier has a particular researcher in mind and is designed to give that specific researcher an unlimited experience at the quality and scale they need. Sign up for free at and choose the plan that supports your research. Or upgrade from your settings.
1
1
6
New in Elicit: Confidence ratings. When Elicit is extracting data from papers, you'll now see a ✓ when Elicit is confident in the accuracy of the answer.
1
1
5
What does this mean for everyone else?. More comprehensive, accurate, and less biased systematic reviews in less time and lower cost. In other words, more and better synthesized evidence on the topics that matter.
1
0
6
.@PredictIt currently implies a 61% chance of a Biden victory. @FiveThirtyEight believes the odds are 87%. The largest gap this election season. To get alerts when markets change, set custom PredictIt notifications at
1
0
6
Or for this query:. “Randomized controlled trials of cognitive behavioral therapy plus standard care versus standard care alone for persistent aggression in adults with schizophrenia". We see more papers specifically comparing CBT to standard of care, and about persistent
1
1
5
If you want to compare two predictions, Elicit now tells you a set of summary stats and generates a fair bet – like this one between @PredictIt and @FiveThirtyEight:.
0
1
5
Compare the results for the query. "Studies of antimicrobial stewardship interventions with multidisciplinary teams published 2010 or later.". The new search shows more results that match the "multidisciplinary teams" criteria.
1
0
5
Apologies if any of you are seeing errors on Elicit! We're investigating a possible service provider (AWS) outage that seems to be affecting many companies right now.
0
0
4
Excited to launch one of our most requested features! Save your custom columns in Elicit to reuse across all of your tables and Notebooks. No more defining custom columns for each table 🎉 .
0
0
5
In case you missed it, Elicit's search recently became much more powerful and precise! . Here are some hidden filtering options in Elicit you may not know about:.
In addition to more relevant results, you can now run advanced searches using 14 filters. That means you can search based on:. • Title.• Journal.• Author.• Keywords.• PubMed ID.• DOI.• Publication year.• Open access status.• Citation count. And more. Here's an example
1
2
5
Have you taken a Speedrun with Elicit? 30-45 minute live sessions that demonstrate just how far you can get into your research topic quickly. Check out past Speedruns for ideas on how to expedite your research with Elicit:
0
1
5
For more information check out this guide: If your team has an even bigger project or other custom requirements, please reach out to sales@elicit.com.
0
1
5
New in Elicit Plus: You can now export Elicit results as a .ris file to import into reference managers like Endnote & Mendeley!. Subscribe to Elicit Plus by going to
0
1
5
@AshliePankonin @MushtaqBilalPhD Undermind focuses more on search and surfacing papers to you whereas Elicit prioritizes data extraction and more complex workflows highly.
0
0
5
Elicit has several export options to take your research with you:.-Export Find Papers steps in .csv, .bib, or .ris format.-Export Extract Data from PDFs and List of Concepts steps in .csv format.
0
1
4
Feedback loops are key to getting better at forecasting. We've launched a rapid tournament to help with this – all questions resolve in a week, and you'll get back scores and calibration info. Forecast here:
0
2
5