Welcome to the groundbreaking experiment that evaluates ChatGPT's prowess in bug detection within Python code! Our script, test_gpt.py, pushes the boundaries of AI, challenging it to mimic the command-line interface's output analysis.
test_gpt.py: The heart of our experiment, automating the comparison between ChatGPT's inference and CLI execution.output.json: The results are in! See how well ChatGPT scored in our rigorous testing.
- Clone the repo:
git clone https://github.com/YourRepoLink - Navigate to the repo directory:
cd repo-name - Run the script:
python test_gpt.py - Marvel at the AI's abilities and our detailed scoring system!
Our experiment isn't just about scores; it's a step forward in understanding AI's potential in programming. Check out our research paper linked within for an in-depth analysis.
Join us in refining the edge of AI in coding! Contributions, issues, and discussions are highly welcomed.
This experiment is open-sourced under the MIT License. Explore, tweak, and innovate!
Note: The links and repository details are placeholders and should be replaced with the actual ones.