DEV Community

Cover image for Battle of the Giants: GitHub Copilot vs ChatGPT ⚔️⚔️
Tapajyoti Bose
Tapajyoti Bose

Posted on • Updated on

Battle of the Giants: GitHub Copilot vs ChatGPT ⚔️⚔️

The release of ChatGPT sent ripples through the programming community as it could come up with incredible code snippets.

How does it compare to GitHub Copilot released last year? Is AI coming after your job? Let's find out.

If you haven't read my previous article on GitHub Copilot, you can check it out here.

Testing ChatGPT

Let's briefly go through the same tests we put GitHub Copilot through.

Test 1: Stock Price Fetch

Sadly, the first time it generated incorrect code.

stock-price-fail

The function wasn't defined as async, and the promises weren't handled properly.

The next attempt was better

stock-price-fetch

Weirdly it works without the API key (as long the query params contain the apikey key with any random value).

Result: GitHub Copilot leaves ChatGPT in the dust.

Test 2: BFS Traversal

This time, ChatGPT nailed it in one go, sadly it was so long that a screenshot didn't fit in the screen, so here is the generated code:

class Node:
  def __init__(self, value):
    self.value = value
    self.left = None
    self.right = None

class BinarySearchTree:
  def __init__(self):
    self.root = None

  def insert(self, value):
    new_node = Node(value)

    if self.root is None:
      self.root = new_node
      return

    current_node = self.root
    while True:
      if value < current_node.value:
        if current_node.left is None:
          current_node.left = new_node
          return
        current_node = current_node.left
      else:
        if current_node.right is None:
          current_node.right = new_node
          return
        current_node = current_node.right

  def inorder_traversal(self, node, numbers):
    if node is None:
      return

    self.inorder_traversal(node.left, numbers)
    numbers.append(node.value)
    self.inorder_traversal(node.right, numbers)

  def get_numbers(self):
    numbers = []
    self.inorder_traversal(self.root, numbers)
    return numbers

bst = BinarySearchTree()
bst.insert(10)
bst.insert(5)
bst.insert(15)
bst.insert(3)
bst.insert(7)
bst.insert(13)
bst.insert(17)

numbers = bst.get_numbers()
print(numbers)  # [3, 5, 7, 10, 13, 15, 17]
Enter fullscreen mode Exit fullscreen mode

The entire code was generated by ChatGPT along with comments on how it works.

Result: ChatGPT wins this one.

Test 3: Array of 100 ones

ChatGPT generated multiple suggestions for this one

array-of-100-ones

When GitHub Copilot was launched, it generated a bit wonky suggestion, but now it generates the same code as ChatGPT.

Result: Tie (depending on which version of the software you compare)

Creating a complete TODO project

To push the tests further, I decided to create a complete TODO project using both the software.

ChatGPT

  1. How to create a TODO project?

    The steps mentioned by ChatGPT are quite comprehensive

    create-todo-project-gpt

  2. How to create a React project?

    create-react-app

  3. Create a Todo component

    todo-component

    This is the part I didn't like. The generated code was using the legacy Class Component syntax. It wasn't handling the addition of todos either

  4. Optimize the Todo component to handle the addition and deletion of todos as well as mark them as done.

    Here is the code it generated (too long for a screenshot)

    import React from "react";
    
    // TodoItem component
    const TodoItem = ({
     item,
     onDelete,
     onComplete,
    }) => (
     <li>
       {item.text}
       <button onClick={onDelete}>Delete</button>
       <button onClick={onComplete}>
         {item.isComplete ? "Undo" : "Complete"}
       </button>
     </li>
    );
    
    // TodoApp component
    class TodoApp extends React.Component {
     // Define the initial state of the component
     state = {
       todoItems: [],
     };
    
     // Method to handle adding new items to the todo list
     addTodo = (text) => {
       // Add the new item to the todo list
       this.setState({
         todoItems: [
           ...this.state.todoItems,
           { text, isComplete: false },
         ],
       });
     };
    
     // Method to handle deleting items from the todo list
     deleteTodo = (index) => {
       // Delete the item at the given index
       this.setState({
         todoItems: this.state.todoItems.filter(
           (_, i) => i !== index
         ),
       });
     };
    
     // Method to handle completing/undoing a todo item
     completeTodo = (index) => {
       // Update the "isComplete" property of the item at the given index
       this.setState((state) => {
         const todoItems = [...state.todoItems];
         todoItems[index].isComplete =
           !todoItems[index].isComplete;
         return { todoItems };
       });
     };
    
     // Method to render the component
     render() {
       return (
         <div>
           {/* Form to add new items to the todo list */}
           <form>
             <input
               type="text"
               placeholder="Add a todo item"
             />
             <button type="submit">Add</button>
           </form>
    
           {/* List of todo items */}
           <ul>
             {this.state.todoItems.map(
               (item, index) => (
                 <TodoItem
                   key={index}
                   item={item}
                   onDelete={() =>
                     this.deleteTodo(index)
                   }
                   onComplete={() =>
                     this.completeTodo(index)
                   }
                 />
               )
             )}
           </ul>
         </div>
       );
     }
    }
    

    The generated code was a bit better this time with code splitting & handling marking todos as done. It was still using the legacy Class Component & didn't handle the addition of todos even after specifically asking for it.

  5. Last try by mentioning to create Functional Components and to handle addition, deletion & completion of todos.

import React from "react";

// TodoItem component
const TodoItem = ({ item, onDelete, onComplete }) => (
  <li>
    {item.text}
    <button onClick={onDelete}>Delete</button>
    <button onClick={onComplete}>
      {item.isComplete ? "Undo" : "Complete"}
    </button>
  </li>
);

// TodoApp component
const TodoApp = () => {
  // Define the initial state of the component
  const [todoItems, setTodoItems] = React.useState([]);

  // Method to handle adding new items to the todo list
  const addTodo = (text) => {
    // Add the new item to the todo list
    setTodoItems([
      ...todoItems,
      { text, isComplete: false },
    ]);
  };

  // Method to handle deleting items from the todo list
  const deleteTodo = (index) => {
    // Delete the item at the given index
    setTodoItems(
      todoItems.filter((_, i) => i !== index)
    );
  };

  // Method to handle completing/undoing a todo item
  const completeTodo = (index) => {
    // Update the "isComplete" property of the item at the given index
    setTodoItems((state) => {
      const todoItems = [...state];
      todoItems[index].isComplete =
        !todoItems[index].isComplete;
      return todoItems;
    });
  };

  // Return the component
  return (
    <div>
      {/* Form to add new items to the todo list */}
      <form>
        <input
          type="text"
          placeholder="Add a todo item"
        />
        <button type="submit">Add</button>
      </form>

      {/* List of todo items */}
      <ul>
        {todoItems.map((item, index) => (
          <TodoItem
            key={index}
            item={item}
            onDelete={() => deleteTodo(index)}
            onComplete={() => completeTodo(index)}
          />
        ))}
      </ul>
    </div>
  );
};
Enter fullscreen mode Exit fullscreen mode

Even this time the addition wasn't handled. On the bright side, Functional Components were used

GitHub Copilot

Sadly Copilot cannot generate a React project. It can only suggest code snippets in a given context, so let's start by creating a react app ourself

  1. Create a React project

    npx create-react-app react-copilot
    
  2. Remove the redundant files & create TodoApp.js, TodoItem.js

  3. Finally it's time for Copilot to work its magic while we sit back and sip a mojito.

    A comment was added at the top of the file, on the task to be done

    Todo Item

    // Functional TodoItem component which takes onDelete,
    // onComplete, and item ({ todo: string; completed: boolean })
    // as props
    

    Todo App

    // Functional TodoApp component which handles addition,
    // deletion & completion of todos. The TodoItem component
    // is used to render each todo item
    
  4. Generated TodoItem

    export default function TodoItem({
     onDelete,
     onComplete,
     item,
    }) {
     return (
       <div>
         <input
           type="checkbox"
           checked={item.completed}
           onChange={() => onComplete(item)}
         />
         <span
           style={{
             textDecoration: item.completed
               ? "line-through"
               : "none",
           }}
         >
           {item.todo}
         </span>
         <button onClick={() => onDelete(item)}>
           Delete
         </button>
       </div>
     );
    }
    

    The generated code used Functional Components and visual indicators for the completed todos with additional styling (it also used a checkbox, which was a personal preference of mine)

  5. Generated TodoApp

    import React, { useState } from "react";
    import TodoItem from "./TodoItem";
    
    export default function TodoApp() {
     const [todos, setTodos] = useState([]);
     const [todo, setTodo] = useState("");
    
     const addTodo = () => {
       setTodos([...todos, { todo, completed: false }]);
       setTodo("");
     };
    
     const deleteTodo = (item) => {
       setTodos(todos.filter((todo) => todo !== item));
     };
    
     const completeTodo = (item) => {
       setTodos(
         todos.map((todo) =>
           todo === item
             ? { ...todo, completed: !todo.completed }
             : todo
         )
       );
     };
    
     return (
       <div>
         <h1>Todo App</h1>
         <input
           type="text"
           value={todo}
           onChange={(e) => setTodo(e.target.value)}
         />
         <button onClick={addTodo}>Add</button>
         {todos.map((item, index) => (
           <TodoItem
             key={index}
             onDelete={deleteTodo}
             onComplete={completeTodo}
             item={item}
           />
         ))}
       </div>
     );
    }
    

Results

Even though we were required to do additional work to set up the project, Copilot was able to generate better code for the Todo App compared to ChatGPT.

At the end of the day, Copilot is supposed to be a tool to help developers write code faster, while ChatGPT is a general purpose chatbot, yet it still can streamline the development process, but GitHub Copilot wins hands down when the task is coding focused! This brings us to the question...

Who will take your job?

(In your mind) Developers of the AI: * evil laugh *

evil-laugh

Before you take out your pitchforks to burn down GitHub HQ, let's take a step back and look at the bigger picture. These are tools to streamline the development process, and they are NOT meant to replace developers.

We still need developers to write the code. Sure, these tools can help us code faster, but without human input they still can't come up with everything on their own.

The news of layoffs spreading like wildfire due to the recession might sound extremely scary, but as long as you keep updating & polishing your skills, you'll be fine!

Take a chill pill, and happy upskilling!

Finding personal finance too intimidating? Checkout my Instagram to become a Dollar Ninja

Thanks for reading

Need a Top Rated Front-End Development Freelancer to chop away your development woes? Contact me on Upwork

Want to see what I am working on? Check out my Personal Website and GitHub

Want to connect? Reach out to me on LinkedIn

I am a Digital Nomad and occasionally travel. Follow me on Instagram to check out what I am up to.

Follow my blogs for bi-weekly new tidbits on Dev

FAQ

These are a few commonly asked questions I get. So, I hope this FAQ section solves your issues.

  1. I am a beginner, how should I learn Front-End Web Dev?
    Look into the following articles:

    1. Front End Development Roadmap
    2. Front End Project Ideas
  2. Would you mentor me?

    Sorry, I am already under a lot of workload and would not have the time to mentor anyone.

Top comments (12)

Collapse
 
tnypxl profile image
tnypxl

AI at large has mostly been an aid or accessory to work that we still have to mostly do ourselves.

ChatGPT and Copilot are really good at accelerating the research-evaluate-implement workflow. It does it by allowing us to inject context and perspective. Something that search engines are not real designed to allow. At least current day.

I love that I can ask ChatGPT to just try again using a different library or a specific design pattern.

Collapse
 
lorenz1989 profile image
lorenz1989

Personally, I prefer ChatGPT. The ChatGPT tool is currently being offered for free with many standout features. It is a valuable resource for anyone looking for quick and accurate answers to their questions. Whether you are a content creator, researcher, or just want to learn more about a particular topic, ChatGPT can help you find the information you need.

ChatGPT

ChatGPT

ChatGPT

Collapse
 
sarthology profile image
Sarthak Sharma

Great Article !!

But the fun fact is Github is owned by Microsoft now.
ChatGPT, which is owned by Open AI, is also funded by Microsoft. So I assume that the only giant who's winning this is Microsoft. 😄

Collapse
 
mokdev profile image
MokDevelopment • Edited

Good Point!...

So bottom line as always... When Microsoft buys the competition the creativity and the innovation will loose its drive until nobody is interested any longer :)

There is really only one good thing on earth that Microsoft did well and that is "VSC" and that is only because Adobe and Jetbrains are still there.

However... lets hope that Microsoft is not the worst future for AI - hopefully Apple, Google and Co develop other systems so that Microsoft needs further development to gain profits from the whole thing.

Collapse
 
kayis profile image
K

I asked ChatGPT for an integration of two libraries and it said it didn't know them so it couldn't help me.

Guess, my job is save.

Collapse
 
ergrato profile image
ergrato

It is good that we feel protected our work as developers. The question is: for how long?

Collapse
 
sardiusjay profile image
SardiusJay

Nice Wrteup, My understanding of ChatGPT leads me to believe that it is only used for text generation. I appreciate the article.

Collapse
 
evanilukhin profile image
Ivan Iliukhin

Technically, in the examples it's used for text generating. The main part of work: what to do? and why it does not work? was made by the author.

Collapse
 
csituma profile image
Clara Situma

Nice read,

There is alot of fear mongering about AI replacing our developer jobs for sure....
but i, like you, see it as JUST a tool to make work faster

Collapse
 
knowlengr profile image
Mark Underwood

ChatGPT as a developer aid wouldn't seem revolutionary if it were not for the primitive state of automatic code generation tools over past decades. Developers gave up in favor of DIY. This tendency still afflicts slow adoption of reuse.

A cautionary note: ChatGPT isn't Watson. It's not designed to answer questions and mine knowledge. It can do that in as much as it finds what Geoffrey Hinton calls "vectors" in text. I wouldn't rely on it to guide a surgeon's knife, but it could well help train future cardiologists and improve diagnostic results when used in conjunction with human expertise.

Collapse
 
atharvashankar profile image
Atharva Shankar avhad

Chat-gpt is not focused on the programming. its made for just text genration. *I think

so, You can type "write story of 10 pages", or "write a poem"

Collapse
 
rostiq profile image
Rostik Natkha

article was written by chatGPT for sure