Can #LLMs replace developers? Introducing RepoCod-Lite 🐟 for faster evaluation to answer this: 200 of the toughest #RepoCod #code-generation tasks: - GPT-4o and other LLMs have < 10% accuracy/pass@1 on RepoCod-Lite tasks - Leaderboard - 67 repository-level, 67 file-level, and 66 self-contains tasks - Detailed problem descriptions (967 tokens) and long canonical solutions (918 tokens) - Dataset: Thanks to the great feedback from #swe-bench’s @OfirPress, here are some clarifications about #RepoCod (: Compared to #SWE-Bench, RepoCod tasks are - General code generation tasks,  while SWE-Bench tasks resolve pull requests from GitHub issues - With 2.6X more tests per task (313.5 compared to SWE-Bench’s 120.8) Compared to #HumanEval, #MBPP, #CoderEval, and #ClassEval, RepoCod has 980 instances from 11 Python projects, with - Whole function generation - Repository-level context - Validation with test cases, and  - Real-world complex tasks: longest average canonical solution length (331.6 tokens) and the highest average cyclomatic complexity (9.00) #LLMs #LLM4Code #security #codegen

@anikbera @yiwu5cs @huyiran1007 @NanJiang719 @cerias @PurdueCS @ieee_ras_icra Thank you! @anikbera It was a fun and productive collaboration for interdisciplinary research!

@yiwu5cs @huyiran1007 @NanJiang719 @anikbera @cerias @PurdueCS @ieee_ras_icra 3/3 📊 Our experiments demonstrate SELP’s effectiveness across diverse tasks. In drone navigation, SELP outperforms state-of-the-art LLM planners by 10.8% in safety rate and by 19.8% in plan efficiency. For robot manipulation, SELP achieves a 20.4% improvement in safety rate.

@yiwu5cs @huyiran1007 @NanJiang719 @anikbera @cerias @PurdueCS @ieee_ras_icra 2/3 3️⃣ Domain-Specific Fine-Tuning: Customizes LLMs for specific robotic tasks, boosting both safety and efficiency.

@yiwu5cs @huyiran1007 @NanJiang719 @anikbera @cerias @PurdueCS @ieee_ras_icra 1/3 💡SELP has 3 key insights: 1️⃣ Equivalence Voting: Ensures robust translations from natural language instructions into LTL specifications. 2️⃣ Constrained Decoding: Uses the generated LTL formula to guide the inference of plans, ensuring the generated plans conform to the LTL.

@FSEconf This is awesome! Isn’t the new conference name just FSE 2025 now? :-)

2/3 For example, when performing tasks such as identifying attacks and malware, security analysts often only have access to assembly, i.e., the human-readable representation of binary code, which is extremely difficult to understand.

3/3 Thus, combined with the increasing sophistication of cybercrime that poses significant threats worldwide (e.g., cybercrime is predicted to cost the world $10.5 trillion annually by 2025 (Sausalito, 2020)), effective binary analysis techniques are in high demand.

1/3 Binaries are a form of code. Think about binaries when you work on LLM4Code. Why binaries and binary models? Binary code plays an irreplaceable role in crucial tasks including vulnerability detection, malware detection, binary recovery, and legacy software maintenance.

@AbhikRoychoudh1 Abhik, congratulations! Very well deserved!

RT @AbhikRoychoudh1: Shonan meeting 217 on Trusted Automatic Programming is currently underway in Japan this week! The discussions are buzz…

RT @sg1753: 📢📢📢 1st IEEE Intl. Conference on LLM-Aided Design (ICLAD) papers due Feb. 28th, 2025! We invite papers on a broad range of topi…

RT @chun_yang_chen: 🚨 Join @TU_Muenchen's School of CIT! We're hiring: 1️⃣ W3 Associate/Full Professor in Healthcare Robotics 2️⃣ Tenure Tr…

@timmenzies Tim, Congratulations! Well deserved!

RT @chun_yang_chen: 🚀 Call for Papers! 📢 Excited to announce our new special issue @emsejournal: 🎯 “When Software Security Meets #LLM: Opp…

RT @Kexin_Pei: The 8th Deep Learning Security and Privacy workshop co-located with IEEE S&P @IEEESSP May 15, 2025, San Francisco ( https://t…

RT @ManlingLi_: [Long Tweet Ahead] Faculty Interview Tips & Common Questions: 🧘‍♀️0. Firstly, do not be nervous - Almost everything can…

@LingmingZhang Thank you, Lingming!

@maxxbw54 Bowen, Thank you so much for hosting Nan!