Our ScienceAgentBench in @Nature news! DeepSeek R1 @DeepSeekR1 vs. @OpenAI o1 on data-driven scientific coding tasks: We sampled 20 tasks from ScienceAgentBench, with 5 tasks from each of the four scientific disciplines (bioinformatics, comp. chemistry, geo info science, phych. & cog. neuroscience). 1. Performance: Given three attempts, DeepSeek R1 can solve 7 out of the 20 tasks, while OpenAI o1 only solves 6 of them. In addition, o1 is able to generate 12 executable programs, and R1 can generate 10 executable programs. This suggests that DeepSeek R1 achieves the same level of performance as OpenAI o1 on ScienceAgentBench. 2. Cost: We find that OpenAI o1 is around 13 times more expensive than DeepSeek R1. On average, o1 requires 0.13 USD to solve a task in ScienceAgentBench, while R1 only costs 0.01 USD. In fact, this is also cheaper than other proprietary models evaluated in our paper, such as GPT-4o and Claude-3.5-Sonnet. 3. Latency: Currently DeepSeek R1 takes a longer time to "think" (1-3min) than OpenAI o1 (<1min) on ScienceAgentBench tasks. Some promising future work could be to further improve the CoT reasoning efficiency with length control, to make the model even better and more practical for daily use. More details will be out soon! Great efforts led by our awesome @RonZiruChen and @ShijieChen98 @osunlp!

Our preprint of AMUSET-TICA is out now. It identifies CVs for biomolecular dynamics with a tensor based approach. The quality of CVs is comparable to deep learning methods. AMUSET-TICA is also fast and numerically stable due to its deterministic algorithm.