How To Automate Unit Test Creation And Code Improvements With CodiumAI

Overview and Audience

• Overview: In this comprehensive workshop, participants will learn how to harness the power of CodiumAI to generate meaningful unit tests, understand code behavior, and suggest code improvements. Through hands-on activities and guided instruction, attendees will explore CodiumAI's integration with Visual Studio Code, delve into its AI-driven test generation, and leverage its capabilities to enhance code quality and robustness.
• Audience: Software developers who regularly use Python, Javascript, or Python seeking to become more efficient at testing or improving their code.

Objective

By the end of this workshop you will be able to use CodiumAI to:

• Create, debug, and modify unit tests
• Quickly understand the intended behavior of code
• Make both stylistic and behavioral modifications to enhance code

About the tool

• CodiumAI analyzes your code and generates meaningful tests to catch bugs before you ship.
• CodiumAI leverages a customized large language model, TestGPT, as well as extensive prompt engineering to provide an in-depth understanding of test design and intended code behavior.
• CodiumAI produced unit tests – rooted in inference of desired code behavior and coverage.
• Codium currently supports test generation in Python, Javascript, and Typescript, as well as code suggestions in any language.
• Developers spend 31% of their time on testing and code maintenance.This means that assuming 10% efficiency improvement at testing and maintenance, in a 100 person developer organization with average salaries, Codium can generate $400k+ in value.

Prerequisite

• Visual Studio Code
• Python
• pytest (pip install pytest) 
  ‍

Section 1 - Getting Started with Calculator Example in Python - 20 minutes

Step 1- Installing

Installation:

1. Click the “Extensions” button to open the Extensions Panel in VSCode

‍

2. Search for “CodiumAI” and click install 

3. Click “Sign In”, and Codium will take open up a page in your web browser where you can sign up/sign in with your preferred method (e.g., GitHub/Email)

Step 2 - Creating your first test

1. Click “Explorer” in VSCode’s side panel, then click “Open Folder”. Select the “codium-ai-tutorial” project which has been shared with you.
2. Click “Calculator.py” in the explorer sidebar to open this class.
3. Click the “Test this class” method.

Step 3 - View first Codium results and inspect configuration

You will now see an output containing three tabs:

• “Test Suite” – containing our generated tests
• “Code Explanation” – which shows an interpretation of our code
• And “Code Suggestions” – which includes ideas for how to improve our code

Click on the “Configuration” dropdown. We can see that our tests have been configured to use either the “Pytest” or “Unittest” frameworks.

Note that if we would like Codium to reflect the style of some existing test, we can upload a “Reference test” for it to emulate, though we will skip this for the purposes of this exercise.

Step 4 - Improving Test Coverage

Click on the numbers next to “Current Coverage”.

We can now see the desired behaviors for this class which Codium has detected. Note that on the pictured run, the figure “19/20” suggests that Codium detected 20 desired behaviors for this class, but only 19 were covered by the tests which it initially generated. Let’s fix this.

Scroll down to find an uncovered behavior and click “Generate Test”.

Step 5 - Running our Codium Tests

Click the “Run All Tests” button within the Test Suite tab.

Step 6 - Inspecting Test Results

In the event of a failed test, Codium provides potential explanations of why the test may have failed, either citing an issue with the code or with the test itself.

If there were any failed tests in your run, scroll down within the “Test Suite” tab to find the failed test, and click “Possible Code Issue” to view Codium’s explanation for one explanation of why the test may have failed.

Step 7 - Fix Code Based on Test Results

After inspecting the potential code/test errors, we can apply either the suggested test fix or code fix, and apply it to our project.

Click “Fix Code” to generate a suggested code fix, and then click “Review Fix” to see CodiumAI’s suggestion. Finally, click “Apply to my code” to update our Calculator class.

‍

Step 8 - CodiumAI’s Code Explanations

Now, we will explore CodiumAI’s code explanation features.

Click on Codium’s Code Explanation tab in the CodiumAI pane.

Note that Code Explanations include the following features:

• Summary which gives a high level explanation of the code
• Example Usage which shows example usage of the code in question
• Code Analysis which explains the functionality and methods of the code in more detail

Step 9 - Code Explanations: Enhance Code

Now we will see how Codium can automatically generate enhancements to our code.

Scroll to the bottom of the Code Explanation tab. Next click “Enhance Code”.

In the example shown, CodiumAI has suggested the addition of docstrings and type hints.

Click “Apply to Code” at the bottom of the pane to apply these changes.

‍

Step 10 - Code Suggestions

Next click the Code Suggestions tab. Here, CodiumAI will suggest changes to your code to help align with best practices.

To see an example of this in practice, click the checkbox next to CodiumAI’s first suggestion, then click “Prepare Code Changes”.

‍

After reviewing Codium’s suggested changes, click “Apply to my Code” to incorporate these suggestions.
‍

Section 2 - Testing and Improving Bookshelf.py (15 minutes)

‍Step 1 - Open Bookshelf.py

We will test these features of CodiumAI, as well as several new ones, on a separate class simulating a Bookshelf.

Click “Explorer” in the VS Code  sidebar, then click “Bookshelf.py” in the sidebar to open the Bookshelf class.

‍

Step 2 - Create tests with our “reference code style”

Click “Test this class” button above the Bookshelf class. However, ignore the results for now.

Instead, we will provide a reference test file. In the Test Suite tab, click Configuration to open this dropdown. Then, click the dropdown next to “Reference test example”, and then click “Add reference test examples.”

With the reference test menu open, click on the Title bar and enter “My Test”. Then click in the Test contents input, and copy and paste the contents of the my-tests.py file provided. Then click “Save”.

Finally, click “Regenerate” to create new tests that leverage this reference test.

Step 3 - Improve test coverage

To improve our code coverage, click on the fraction next to code coverage (note that the numbers in your run may differ slightly from the numbers in the screenshot below).

Scroll down to find an uncovered behavior, indicated by the presence of a “Generate test” button, and click the “generate test” button.

Step 4 - Modify a test using natural language

We will now experiment with CodiumAI’s ability to modify tests with natural language.

Scroll down to the Test sections of this tab, and locate a test with potential to be modified. Click on the input to modify a test with natural language, and enter a prompt to modify it. Click the paper plane icon to complete modifying the test.

Note that in the example above, we used the prompt “change the ISBN to 4567”, but you may need to use a different prompt that is appropriate to the tests that CodiumAI’s model generated on your run

Step 5 - Apply CodiumAI’s Code Suggestions

Now, we will explore CodiumAI’s ability to generate more complex code changes. 

Click on the “Code Suggestions'' tab. Please read through some of Codium’s proposed changes to this class as well as its provided reasons for doing so. You may notice a mix of changes that fix documentation, coding style, and performance.

Click on the checkmark next to each of Codium’s suggestions, then click on the “Prepare Code Changes” button.

You will now see several proposed code changes from CodiumAI. Click “Apply to my Code” to incorporate these changes.

Conclusion & Next Steps

Here is a recap of what you can do now with your own codebase in just a few clicks:

• Automate unit test creation
• Modify tests or improve test coverage
• Summarize and interpret code behavior with AI
• Refactor code and generate improvements

If you want to go further here are the next steps you can follow to be more advanced.

• Review the Tutorials on CodiumAI’s Resources Page
• Join CodiumAI’s Discord
• Explore CodiumAI’s PR Agent feature

Homework:

Use CodiumAI with your own code, and do the following:

• Generate unit tests
• Use “Code Explanations” to help interpret and generate documentation for dense or undocumented classes within your code base
• Use “Code Suggestions” to help refactor a class within your codebase that you think is in need of improvement

List of References and Links

• Tutorial: Boost your development workflow with CodiumAI
• CodiumAI’s Github

Additional Resources

• CodiumAI fundraising announcement
• CodiumAI YouTube channel
• Javarvisited: 10 Reasons to Use CodiumAI

‍