Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
Shashank Kirtania, Param Biyani, Priyanshu Gupta, Yasharth Bajpai, Roshni Iyer, Sumit Gulwani, Gustavo Soares
2026 COLM | July 2026
Publications
-
-
-
-
SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
Sanket Mhatre, Yasharth Bajpai, Sumit Gulwani, Emerson Murphy-Hill, Gustavo Soares
IEEE/ACM International Conference on AI-powered Software (AIware) | November 2025
-
IndiMathBench: Autoformalization Mathematical Reasoning Problems with a Human Touch
Param Biyani, Shashank Kirtania, Yasharth Bajpai, Ashish Tiwari, Sumit Gulwani
ArXiv | October 2025
-
Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Gustavo Soares, Emerson Murphy-Hill
IEEE/ACM International Conference on Automated Software Engineering, ASE | October 2025
-
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Jenny T. Liang, Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Vu Le, Chris Parnin, Arjun Radhakrishna, Ashish Tiwari, Emerson Murphy-Hill, Gustavo Soares
ACM Transactions on Computer-Human Interaction | September 2025
-
-
-
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
-
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
-
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
-
-
-
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
-
Augmented Embeddings for Custom Retrievals
Anirudh Khatry, Yasharth Bajpai, Priyanshu Gupta, Sumit Gulwani, Ashish Tiwari
arXiv | October 2023
-
-
-
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
-
-
-
Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
Shashank Kirtania, Param Biyani, Priyanshu Gupta, Yasharth Bajpai, Roshni Iyer, Sumit Gulwani, Gustavo Soares
2026 COLM | July 2026
-
SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
Sanket Mhatre, Yasharth Bajpai, Sumit Gulwani, Emerson Murphy-Hill, Gustavo Soares
IEEE/ACM International Conference on AI-powered Software (AIware) | November 2025
-
IndiMathBench: Autoformalization Mathematical Reasoning Problems with a Human Touch
Param Biyani, Shashank Kirtania, Yasharth Bajpai, Ashish Tiwari, Sumit Gulwani
ArXiv | October 2025
-
Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Gustavo Soares, Emerson Murphy-Hill
IEEE/ACM International Conference on Automated Software Engineering, ASE | October 2025
-
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Jenny T. Liang, Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Vu Le, Chris Parnin, Arjun Radhakrishna, Ashish Tiwari, Emerson Murphy-Hill, Gustavo Soares
ACM Transactions on Computer-Human Interaction | September 2025
-
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
-
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
-
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
-
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
-
Augmented Embeddings for Custom Retrievals
Anirudh Khatry, Yasharth Bajpai, Priyanshu Gupta, Sumit Gulwani, Ashish Tiwari
arXiv | October 2023
-
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
-
-
-
SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
Sanket Mhatre, Yasharth Bajpai, Sumit Gulwani, Emerson Murphy-Hill, Gustavo Soares
IEEE/ACM International Conference on AI-powered Software (AIware) | November 2025
-
IndiMathBench: Autoformalization Mathematical Reasoning Problems with a Human Touch
Param Biyani, Shashank Kirtania, Yasharth Bajpai, Ashish Tiwari, Sumit Gulwani
ArXiv | October 2025
-
Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Gustavo Soares, Emerson Murphy-Hill
IEEE/ACM International Conference on Automated Software Engineering, ASE | October 2025
-
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
-
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
-
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
-
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
-
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
-
-
-
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Jenny T. Liang, Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Vu Le, Chris Parnin, Arjun Radhakrishna, Ashish Tiwari, Emerson Murphy-Hill, Gustavo Soares
ACM Transactions on Computer-Human Interaction | September 2025
-
-
-
Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Gustavo Soares, Emerson Murphy-Hill
IEEE/ACM International Conference on Automated Software Engineering, ASE | October 2025
-
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Jenny T. Liang, Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Vu Le, Chris Parnin, Arjun Radhakrishna, Ashish Tiwari, Emerson Murphy-Hill, Gustavo Soares
ACM Transactions on Computer-Human Interaction | September 2025
-
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
-
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
-
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
-
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
-
-
-
Augmented Embeddings for Custom Retrievals
Anirudh Khatry, Yasharth Bajpai, Priyanshu Gupta, Sumit Gulwani, Ashish Tiwari
arXiv | October 2023
-
-
-
Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
Shashank Kirtania, Param Biyani, Priyanshu Gupta, Yasharth Bajpai, Roshni Iyer, Sumit Gulwani, Gustavo Soares
2026 COLM | July 2026
-
SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks
Sanket Mhatre, Yasharth Bajpai, Sumit Gulwani, Emerson Murphy-Hill, Gustavo Soares
IEEE/ACM International Conference on AI-powered Software (AIware) | November 2025
-
Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Gustavo Soares, Emerson Murphy-Hill
IEEE/ACM International Conference on Automated Software Engineering, ASE | October 2025
-
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
-
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
-
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
-
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
-
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
-
-
-
IndiMathBench: Autoformalization Mathematical Reasoning Problems with a Human Touch
Param Biyani, Shashank Kirtania, Yasharth Bajpai, Ashish Tiwari, Sumit Gulwani
ArXiv | October 2025
-
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Jenny T. Liang, Aayush Kumar, Yasharth Bajpai, Sumit Gulwani, Vu Le, Chris Parnin, Arjun Radhakrishna, Ashish Tiwari, Emerson Murphy-Hill, Gustavo Soares
ACM Transactions on Computer-Human Interaction | September 2025
-
Augmented Embeddings for Custom Retrievals
Anirudh Khatry, Yasharth Bajpai, Priyanshu Gupta, Sumit Gulwani, Ashish Tiwari
arXiv | October 2023
-