My Research
I am a first-year Ph.D. student at the Software Evolution and Analysis Laboratory (SEAL) of the Computer Science Department at UCLA directed by Prof. Dr. Miryung Kim.
My current research interests include Active Learning for separating faulty static analysis results from the correct ones, and understanding the code-generation capabilities of frameworks that use a Large Language Model (LLM) as the underlying structure via benchmarking.
Previously I was a member of BILSEN (Bilkent University Software Engineering and Data Analytics Research Group) of the Computer Engineering Department at Bilkent University, led by Dr. Eray Tuzun.
Publications
Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT
April 21, 2023
Burak Yetiştiren, Işık Özsoy, Miray Ayerdem, and Eray Tüzün. 2023. Evaluating the code quality of AI-Assisted Code Generation Tools: An empirical study on github copilot, Amazon CodeWhisperer, and chatgpt. https://doi.org/10.48550/arXiv.2304.10778
Assessing the Quality of GitHub Copilot’s Code Generation
November 17, 2022
Burak Yetistiren, Isik Ozsoy, and Eray Tuzun. 2022. Assessing the quality of GitHub copilot’s code generation. In Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE 2022). Association for Computing Machinery, New York, NY, USA, 62–71. https://doi.org/10.1145/3558489.3559072
Invited Talks
Microsoft PROSE Team
January 18, 2023
"Assessing the Quality of GitHub Copilot’s Code Generation"
The introduction of GitHub’s new code generation tool, GitHub Copilot, seems to be the first well-established instance of an AI pair-programmer. GitHub Copilot has access to a large amount of open-source projects, enabling it to utilize more extensive code in various programming languages than other code generation tools. Although the initial and informal assessments are promising, a systematic evaluation is needed to explore the limits and benefits of GitHub Copilot. The main objective of this study is to assess the quality of generated code provided by GitHub Copilot. We also aim to evaluate the impact of the quality and variety of input parameters fed to GitHub Copilot. To achieve this aim, we created an experimental setup for evaluating the generated code in terms of validity, correctness, and efficiency. The results suggest that GitHub Copilot was able to generate valid code with a 91.5% success rate. In terms of code correctness, out of 164 problems, 28.7% were correct, while 51.2% were partially correct, and 20.1% were incorrectly generated. Our empirical analysis shows that GitHub Copilot is a promising tool based on the results we obtained, however further and more comprehensive assessment is needed in the future.