How to build LLM agent to automate your code review workflow using crewAI?

Read Time:

👋 Introduction

Imagine you are an entrepreneur who have a product idea but it require extra 1-2 persons to help you achieve that task or you are a manager who got an feature idea but you have to hire more 2-3 employees for that task. But what if i say you can hire a group of AI which can work for you as an employee. sounds good right? 👀

💡You can get all of the code shown in this blog from here.

You can achieve this using AI agents. These agents can automate any task for you and they can be any personality in your company like software developer, content writer or travel agency expert. You have full command on these agents and these agent will only listen to you and do the tasks which you will provide to them. So basically you are their boss 😎.

You can create such agents using crewAI which is a framework that allows you to create such autonomous AI agents. crewAI works on the top of Langchain framework which is one major framework to connect any LLM with external sources and create applications on top of these LLMs.

To learn more about Langchain and Langchain agents, take a look at my blog where i automated a meeting workflow using Langchain Agent.

In this blog, we will explore crewAI and its functionalities and also we will create one crew which will automate a code review workflow for us.

This crew will do following tasks

  • It will take url of github repository of a project as an input and a folder or file name which you want to review
  • It will get the content of specified files using github API
  • Review the given file content and make changes in a code and provide feedbacks
  • Store the review in a notion document so that it can be reviewed later by a human (You)

Here is the sneak peek of our crew 👀

So let’s start! 🚀

👥 What is crewAI?

CrewAI is an AI framework which is designed to orchestrate role-playing autonomous AI agents. This framework empowers agents to share information among each other, assign tasks to each other and delegate tasks to each other to get the final result.

One of the key feature of crewAI is that every agent have a specific task assigned to it so that it can focus on the specific task and try to reach the end goal which is assigned to it by using built in or custom tools provided to it.

What is an agent?

Agents are the employees or little helpers within the CrewAI ecosystem, bringing the magic of automation to life. These agents are essentially intelligent entities, programmed to understand and respond to human language, allowing them to perform a myriad of tasks autonomously.

In CrewAI, every agent have following parameters:

  • Role: The role or personality of an agent, for example software engineer or content writer
  • Goal: Goal specifies the final goal which agent have to achieve, for example perform maths calculations
  • Backstory: Provides context to the agent's role and goal, enriching the interaction and collaboration dynamics.
  • Tools: The tools which agent can use to achieve the given goal. These tools can be default tools provided by CrewAI or Langchain, or it can be custom tools created by you.

A sample agent will look like this in code:

mathsAgent = Agent(
    role='Maths expert',
    goal='Perform the calculations on the numbers given to you',
    backstory='An maths expert working with numbers who have experience solving complex and basic maths problems',

What is task?

In the CrewAI framework, tasks are individual assignments that agents complete. For example, one agent might gather data while another analyzes it. This collaborative approach can be defined within the task properties and managed by the Crew's process.

Every task have following parameters:

  • description: A detailed description about task which includes information about end goal, what needs to be achieved and how to achieve it
  • agent: An agent which is going to complete this task
  • expected_output: Clear and detailed definition of expected output for the task.

A sample task will look like this:

first = 100
second = 20
operation = "add"
MathsTask = Task(
          You are given 2 numbers and you have to convert the operation text into maths syntax based on below conditions
          For example,
          add: a+b
          sub: a-b
          div: a/b
          mul: a*b
          Here are 2 numbers
          {first} {second}
          Here is the operation
          Return mathematical expression as a response.
			expected_output="Mathematical expression as a string"

What is crew?

A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks. Each crew defines the strategy for task execution, agent collaboration, and the overall workflow.

You can think of crew as your own organization where agents are your employees and tasks are the roles provided to them.

You can create 2 types of crew flows:

  • sequential: Where output of first agent is used as input for next agent which creates a chain of agents which is working sequentially.
  • hierarchical: Where these agents can work independently without being dependent on any other agent. There will be one manager LLM which manages all these agents and shares information among agents to reach the end goal.

Let’s create our first crew

Let’s take a basic example, you want to get the latest value of any currency in INR and then you want to perform some mathematical operations on that value. Let’s create a crew for that!

We will need 2 agents for this task:

  • Search Expert: A google search agent which will give us the latest price of given currency in INR
  • Maths Expert: An agent which can perform mathematical operations on given numbers

We will use serper API to get data from google search and we will create custom tools for both search and calculation.

🛠️ Prerequisites:

  • OpenAI API key: we will use gpt-4 as our LLM for this task. Get your api key from openai dashboard and add it as a environment variable in your code as it’s recommended to keep it private.
  • Serper API key: It will be used to search internet. Get your API key from serper website (it provides free tokens for new users without credit card!).

Let’s first install all required modules and dependencies

!pip install crewai requests 'crewai[tools]'

Let’s import the methods and modules which we are going to use

# Import statements
from crewai import Crew, Agent, Task
from textwrap import dedent
import os
import json
import requests

Don’t forget to store your openai API key in environment variables with name OPENAI_API_KEY otherwise it won’t work!

os.environ["OPENAI_API_KEY"] = openai_secret;

Now let’s create our custom tools, one for mathematical calculation and one for internet search

Here we will use @tool decorator from Langchain to specify a tool name and description so that our agents can use it. These tools are just simple functions where we will write our business logic

from import tool
class Tools:

    @tool("Make a calculation")
    def calculate(operation):
        """Useful to perform any mathematical calculations,
        like sum, minus, multiplication, division, etc.
        The input to this tool should be a mathematical
        expression, a couple examples are `200*7` or `5000/2*10`
            return eval(operation)
        except SyntaxError:
            return "Error: Invalid syntax in mathematical expression"

    @tool("Search the internet")
    def search_internet(query):
      """Useful to search the internet 
      about a a given topic and return relevant results"""
      top_result_to_return = 2
      url = ""
      payload = json.dumps({"q": query})
      headers = {
          'X-API-KEY': serper_secret,
          'content-type': 'application/json'
      response = requests.request("POST", url, headers=headers, data=payload)
      results = response.json()['organic']
      string = []
      for result in results[:top_result_to_return]:
              f"Title: {result['title']}", f"Link: {result['link']}",
              f"Snippet: {result['snippet']}", "\n-----------------"
        except KeyError:

      return '\n'.join(string)

Now let’s create our agents aka AI employees

class Agents:
  def mathsAgent():
    return Agent(
    role='Maths expert',
    goal='Perform the calculations on the numbers given to you',
    backstory='An maths expert working with numbers who have experience solving complex and basic maths problems',
    allow_delegation=False # if it's true then agent will be able to delegate task to another agent
  def searchAgent():
    return Agent(
    role='Financial Expert',
    goal='Search internet and give the current value of given currency in INR',
    backstory='A stock market expert who have very good knowledge about every currency and have given accurate currency values',

And now let’s create the tasks which we will give to these agents

class Tasks:
  def mathsTask(agent,operation):
    return Task(
          You are given a currency value and you have to perform given operation on that number
          Here is the operation:
      expected_output="Final answer after performing mathematical operation"
  def searchTask(agent,stock):
    return Task(
        you have given a currency name and you have to search internet and find the current value of this currency in INR
        Here is the currency name:
      expected_output="currency value in INR as a string"

Let’s create our crew class, we will call it MathsCrew for now but you can give any name you want because it’s your company 😎

Remember that the sequence of tasks matter here because it is a sequential process.

class MathsCrew:
  def __init__(self,stock,operation):
    self.stock = stock
    self.operation = operation
  def run(self):
		# Agents
    mathsAgent = Agents.mathsAgent()
    searchAgent = Agents.searchAgent()
		# Tasks
    searchTask = Tasks.searchTask(agent=searchAgent,stock=stock)
    mathsTask = Tasks.mathsTask(agent=mathsAgent,operation=operation)
		# Create crew
    crew = Crew(
      verbose=2, # You can set it to 1 or 2 to different logging levels
		# Run the crew
    result = crew.kickoff()

Lastly, add a code to take input from user and run the crew

print("Welcome to currency agent!")
stock = input("Enter currency name:")
operation = input("Enter operation:")

mathsCrew = MathsCrew(stock,operation)
result =

Let’s run the above code with these 2 inputs:

  • stock - USD
  • operation - add 10 into it

After running it, you will get this output from crew:

As you can see, we can see the whole thinking process of both agents and how data is getting passed between these 2 agents in crew and finally we are getting our expected output!

So now we know how to create a basic crew so let’s make one mini project to automate code review workflow of any organization or repository 🚀

💻 Mini Project: Automating code review workflow

Imagine you are a CEO of a company which works with information technology industry and you have a team of software developers but as a good CEO you want to maintain the quality of code which these developers have written to match the industry standards and to get a good reputation among your clients. That’s why you need someone senior who can review codes written by these developers.

Now the main problem is that you need to hire a senior software developer who can review code of these developers but you don’t want to pay him just for a code review and doing this manually might take some time. So instead of doing this manually you can create your own crew of agents which will do this task for you. 👀


We will create a crew which can do following tasks

  • Get the tree structure of github repo
  • Get the content of the files which you want to review
  • Review the given file
  • Add the review and updated code in notion document

The workflow of our crew will look like this

We will first take a url of github repository which we have to review and then we will take a file or folder name as an input from a user. if user provides a folder name then we will review every file inside that folder and if user provides a filename then we will only review that file.


  • OpenAI API key: we will use gpt-4 as our LLM for this task. Get your api key from openai dashboard and add it as a environment variable in your code as it’s recommended to keep it private.
  • Github personal access token: we will use this token to make requests to github API as an authenticated user because you can’t access your private repositories without this token and we are assuming that as an organization, you like to keep your repository private. Get your personal access token (PAT) from github settings.
  • Notion API key: we will use this notion key to create notion pages and to add data into a notion document using notion API. Create your integration from here and add it in your document by clicking on 3 dots icon → connect to → your integration name.

So now we have all the things ready so let’s code it! 🚀

Let’s code it

Start by installing the required modules and dependencies

!pip install crewai requests 'crewai[tools]' notion_client

Import all the modules and dependencies

# Import statements
from crewai import Crew,Agent,Task
from textwrap import dedent
from import tool
import os

Make sure to keep all of your API keys private and store it as an environment variable

secret = os.environ["OPENAI_API_KEY"]
github_secret = os.environ['GITHUB_KEY'] 
notion_key = os.environ['NOTION_KEY']

We will create one notion page and we will keep pushing the review in that notion file instead of creating separate notion pages for every file. we will use notion_client module to interact with notion API.

Let’s create a helper function to create a notion page which will take project name as a parameter which will be used as a file name. Also get the page_id of a notion page in which you want to add all these documents.

# Function to create a notion page with given title
def createNotionPage(projectName):
  parent = {"type": "page_id","page_id": notion_page_id}
  properties = {
      "title": {
          "type": "title",
          "title": [{ "type": "text", "text": { "content": f"Code review of {projectName}" } }]
  create_page_response = notion.pages.create(
  return create_page_response['id']

Now let’s create a helper method to get the tree structure of given github repository

# Our final path will be stored in this global variable
global_path = ""
# Function to get tree structure of given github repo
def getFileTree(owner,repo,path='',level=0):
    Fetch and print the tree structure of a GitHub repository, ignoring specific folders.

    - owner: The username of the repository owner.
    - repo: The name of the repository.
    - path: The path to fetch. Leave empty to fetch the root directory.
    - level: The current depth in the tree structure.
    # Directories to ignore
    ignore_dirs = {'public', 'images', 'media', 'assets'}
    global global_path

    api_url = f"{owner}/{repo}/contents/{path}"
    # Add the Authorization header with the token
    headers = {'Authorization': f'token {github_secret}'}
    # Make the request
    response = requests.get(api_url,headers=headers)
    items = response.json()

    if isinstance(items, list):
        for item in items:
            # Skip ignored directories
            if item['name'] in ignore_dirs:

            global_path += f"{' ' * (level * 2)}- {item['name']}\n"
            if item['type'] == 'dir':
                getFileTree(owner, repo, item['path'], level + 1)

Define all of your tasks for your agents

# Tasks
from crewai import Agent,Task
import requests
import base64

class Tasks:
  def ReviewTask(agent,repo,context):
    return Task(
                  Review the given file and provide detailed feedback and reviews about the file if it doesn't follow the industry code standards
                  Take the file path and file contents from contentAgent
                  Make changes in file content to make it better and return the changed content as updated_code in response
                  Return the below values in response
                  project_name: {repo}
                  file_path: file path
                  review: review_here
                  updated_code: updated content of file after making changes

                  Return the output which follows the below array structure and every element must be wrapped in multi-line string
                  In case of updated_code, add the full code as a multi-line string
                  Only return file content which got changed in updated_code, there are multiple changes in file content then send whole file content

                  Every array should follow this format:

                  Don't return anything except array in above format
                expected_output="An array of 4 elements in a format given in description"

  def NotionTask(agent,context,page_id):
    return Task(
        You are given an array of 4 elements and a page id and you will have to add this data in notion
        Here is the id of notion page
        Say 'Data is added successfully into notion' in case of success else return given array.
        expected_output="Text saying 'Data is added successfully into notion' in case of success and 'Could not add data in notion' in case of failure"
  def getFilePathTask(agent,filetree,userInput):
    return Task(
        You are given a tree structure of folder and userInput and first you have to decide whether it is a folder or file from given tree structure of a folder
        Follow this approach
        - If it's a file then return array with 1 element which contains the full path of that file in this folder structure
        - If it's a folder then return array of paths of sub files inside that folder, if there is a subfolder in given folder, then return paths for those files as well
        - If userInput is not present in given tree structure then just return empty array
        Please return FULL path of a given file in given folder tree structure
        For example if tree structure looks like this:
        - src
          - components
            - Login.jsx
            - Password.jsx
        - backend
          - api
        Then the full path of Login.jsx will be src/components/Login.jsx
        DON'T send every file content at once, send it one by one to reviewAgent
        here is the tree structure of folder:
        here is user input


        ONLY an array of paths
        For example:

  def getFileContentTask(agent,owner,repo,path):
    return Task(
        You are given a file path and you have to get the content of file and file name using github API
        here is the file path
        here is owner name
        here is repo name
        Don't return anything except filename and content
        expected_output="filename and content of given file"

Let’s create custom tools which our agents will use.

We will create 2 custom tools here:

  • addToNotion: It will add the given content into notion document with given page_id
  • getFileContents: It will return the content of given file using github API

We will @tool decorator from Langchain to define the custom tool

# Custom Tools
from pprint import pprint
import json
import ast
from import tool
class Tools():

    @tool("Add data to notion")
    def addToNotion(output,page_id):
      Used to add an data given as input in notion document.
      children = [
              "object": "block",
              "type": "heading_2",
              "heading_2": {
                "rich_text": [{ "type": "text", "text": { "content": "🚀 File Name" } }]
              "object": "block",
              "type": "paragraph",
              "paragraph": {
                "rich_text": [{ "type": "text", "text": { "content": output[1] } }]
              "object": "block",
              "type": "heading_2",
              "heading_2": {
                "rich_text": [{ "type": "text", "text": { "content": "📝 Review" } }]
              "object": "block",
              "type": "paragraph",
              "paragraph": {
                "rich_text": [{ "type": "text", "text": { "content": output[2] } }]
              "object": "block",
              "type": "heading_2",
              "heading_2": {
                "rich_text": [{ "type": "text", "text": { "content": "💡 Updated code" } }]
              "object": "block",
              "type": "code",
              "code": {
                "caption": [],
                "rich_text": [{
                  "type": "text",
                  "text": {
                    "content": output[3]
                "language": "markdown"
      add_data_response = notion.blocks.children.append(

    @tool("get file contents from given file path")
    def getFileContents(path,owner,repo):
        used to get the content of given file using the given path, owner of repository and repository name
        url will look like this{owner}/{repo}/{path}
      if path.startswith("https://"):
        api_url = path
        api_url = f"{owner}/{repo}/contents/{path}"

      # Add the Authorization header with the token
      headers = {'Authorization': f'token {github_secret}','X-GitHub-Api-Version': '2022-11-28'}
      # Make the request
      response = requests.get(api_url,headers=headers)

      # Check if the request was successful
      if response.status_code == 200:
          file_content = response.json()

          # Check the size of the file
          if file_content['size'] > 1000000:  # 1MB in bytes
              return "Skipped: File size is greater than 1 MB."

          # The content is Base64 encoded, so decode it
          content_decoded = base64.b64decode(file_content['content'])

          # Convert bytes to string
          content_str = content_decoded.decode('utf-8')

          # Check the number of lines in the file
          if len(content_str.split('\n')) > 500:
              return "Skipped: File contains more than 500 lines."
          return content_str
          # Handle errors (e.g., file not found, access denied)
          return f"Error: {response.status_code} - {response.reason}"

We will require 4 agents in our crew:

  1. ReviewAgent: It will review given file based on filename and file content
  2. NotionAgent: It will add the given data into given notion document
  3. ContentAgent: It will return the content of given file using github API
  4. PathAgent: It will return the array of full paths of files from the given tree structure so that we can build the API url for that file

Let’s create our agents!

# Agents
class Agents:
  def ReviewAgent():
    return Agent(
            role='Senior software developer',
            'Do code reviews on a given file to check if it matches industry code standards',
            "You're a Senior software developer at a big company and you need to do a code review on a given file content.",
  def NotionAgent():
    return Agent(
        role = "Notion api expert and content writer",
        goal = "Add given array data into notion document using addToNotion tool",
            "You're a notion api expert who can use addToNotion tool and add given data into notion document",
  def PathAgent():
    return Agent(
        role="File path extractor",
        goal = "Get the tree structure of folder and return full paths of the given file or files of given folder in array format",
        backstory = "You're a file path extractor who have created several file paths from given tree structure",
  def ContentAgent():
    return Agent(
        role="github api expert",
        goal="Get the content of given file using github API",
        backstory="You're github api expert who have extracted many file contents using github's api",

Now let’s create our crew class, we will call it ReviewCrew in which we will have a run method which will start the crew execution

ReviewCrew will accept 4 parameters:

  • owner: name of github repository owner
  • repo: name of github repository
  • page_id: page id of the notion page in which we want to add the review
  • path: full path of file which needs to be reviewed

from crewai import Process
class ReviewCrew:
  def __init__(self,owner,repo,page_id,path):
    self.owner = owner
    self.repo = repo
    self.page_id = page_id
    self.path = path
  def run(self):
    # Agents
    reviewAgent = Agents.ReviewAgent()
    contentAgent = Agents.ContentAgent()
    notionAgent = Agents.NotionAgent()
    # Tasks
    contentTask = Tasks.getFileContentTask(agent=contentAgent,owner=owner,repo=repo,path=path)
    reviewTask = Tasks.ReviewTask(agent=reviewAgent,repo=repo,context=[contentTask])
    notionTask = Tasks.NotionTask(agent=notionAgent,page_id=page_id,context=[reviewTask])
    # Crew
    crew = Crew(
      verbose=2, # You can set it to 1 or 2 to different logging levels
		# Run the crew
    result = crew.kickoff()

Now we are only one step close to our final crew!

Let’s add the code to do following tasks:

  • Take input from user
  • Get tree structure of given github repository
  • Get array of file paths which needs to be reviewed
  • Traverse these paths one by one and review them one by one using our crew

import ast
# Take input from user
github_url = input("Provide github repo URL:")
userInput = input("Provide file/folder name you want to review:")
# Get owner and repo name from github url
split_url = github_url.split('/')
owner = split_url[3]
repo = split_url[4]
# Get the tree structure of github repository
# Get array of full paths of given files
pathAgent = Agents.PathAgent()
pathTask = Tasks.getFilePathTask(agent=pathAgent,filetree=global_path,userInput=userInput)
paths = pathTask.execute()
global_path = ""
# Convert agent output into an array
page_id = createNotionPage(projectName=repo)
paths = ast.literal_eval(paths)
# Traverse the paths one by one and review them using ReviewCrew
for path in paths:
  # Run crew
  reviewCrew = ReviewCrew(owner=owner,repo=repo,page_id=page_id,path=path)

Now we are ready to launch our crew 🚀

After running the above code, you will be able to see the thought process of agents and how they are sharing information with each other in your console. Once the process is completed you will see the newly created notion document in your workspace which looks like this:

After opening the document you can see the detailed review about the files which you specified 🤩

And now we have successfully created a crew of autonomous agents which can review your code without a need of human interaction 🚀

💡You can get all of the code shown in this blog from here.

👀 Challenges

Let’s discuss the challenges which I faced while making this mini project 👀

Problem in reviewing the github repository

The first question i came in my mind was that “How do i review the whole github repository and if i review every file one by one then how the agent will know which file is related to which directory 🤔? “

After thinking about it, i found one way that we can create the tree structure of given github repository and send it as a context to our agent so that it can know the relationship of files and also it can review the code structure if needed.

Which file to review?

Now i had the tree structure of code but still i was unaware about the files which i need to review so after thinking for a while, i found that there are basically 2 scenarios while doing code review:

  1. You are maintainer of a repository or a person who already have knowledge about the repository which you want to review and you only want to review a commit of specific files from whole repository
  2. You have no idea about the repository and you just want to review the basic structure of repo if it follows the industry standards or not

So for this project i chose the 1st scenario because most of the time we do have idea about the project or the code which we are reviewing so i added an input field where user will provide the folder or file name which they want to review and then from that input, we will provide both tree structure and userInput to our agents and then they will only fetch those file contents using github API.

So here we solved the openAI cost and github API rate limiting issue by only reviewing files which are needed 🤧.

If you want to go with 2nd scenario then you can take the tree structure of a repository and first check if it follows the industry standards for a folder structure. For example, if its a react app then check if routing is there or redux setup is there or does it follows your company folder structure.

Choosing the files/folders to review is a crucial task and also depends on your way of code review. you can also create agents to only review Routing or a redux setup in an application.

Response token limit of openAI

When i was making this project, I was using all of the 4 agents in a single crew and i was running that crew only once. In current version of crew it runs for each path in paths array but the main problem was that it was reviewing every file at once and was returning it to another agents and because of that the output was not perfect and i was not getting updated code because of the output token limit of openAI.

Also for a files which have more than 200-300 lines of code was not getting reviewed properly because of the context length because in previous version every file content was getting added in a single context to openai and because of that i was not able to review more than 2-3 files.

To solve this issue, I used the pathAgent separately and then i iterated through the file paths one by one and for each path I ran the ReviewCrew so that it can review one file at a time and can review more bigger files. It increased the openAI API calls but it made the code review process more smooth, scalable and accurate.

📝 Conclusion

As we've explored in this blog, the potential of autonomous agents and CrewAI is vast. By establishing the crew of agents withing your organization you can increase efficiency and accuracy of a code review process. Integrating CrewAI into your company's operations can revolutionize how code reviews are conducted, leading to higher-quality code, faster delivery times, and a more agile development environment.

If you want to take a look at more crew examples like this then check it here.

So, whether you're a small team looking to optimize your code review workflows or a large organization seeking to revolutionize your workflow, autonomous agents are a very good consideration to automate your tasks with the use of Artificial Intelligence.

If you are looking to build custom AI agents to automate your workflows then kindly book a call with us and we will be happy to convert your ideas into reality and make your life easy.

Thanks for reading 😄.

Book an AI consultation

Looking to build AI solutions? Let's chat.

Schedule your consultation today - this not a sales call, feel free to come prepared with your technical queries.

You'll be meeting Rohan Sawant, the Founder.
Book a Call

Let us help you.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Behind the Blog 👀
Shivam Danawale

Shivam is an AI Researcher & Full Stack Engineer at Ionio.

Rohan Sawant

Rohan is the Founder & CEO of Ionio. I make everyone write all these nice articles... 🥵