Using GPT-4 to Write and Debug Solidity Smart Contracts

Introduction

Today, it's near impossible to walk down the street and not overhear a conversation about the transformative impact of generative AI. Of course, this applies to both the physical and virtual world (e.g., Twitter). You've probably heard by now that artificial intelligence tools like ChatGPT can plan your vacation, draft an essay, and tell you a joke. But did you know that ChatGPT can even write working Solidity code for you? And, it doesn't just return a .sol file to you without any context. It can actually explain to you how the code is structured, walk you through deployment steps, and, drumroll, even write your test files for you. Yup, that's right. No more excuses for a lack of test coverage when ChatGPT can take care of that for you.

In this tutorial, we'll look at how ChatGPT can help you write, deploy, and debug Solidity smart contracts. But first, let's dive a bit more into what ChatGPT is exactly.

An Overview of ChatGPT

What is ChatGPT?

ChatGPT is a text-based Large Language Model (LLM) created by the company OpenAI. According to OpenAI, "The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests." ChatGPT can hold a conversation with you and remember your chat history until a new session is started. To learn more about ChatGPT, check out this introduction to ChatGPT on the OpenAI Blog.

GPT-4 vs. ChatGPT

As of the time of writing, GPT-4 was the latest version offered as part of the ChatGPT Plus subscription service, available at a cost of $20 USD per month. While GPT-4 is a paid service, you can follow this same tutorial using the free tiers of service available in earlier versions. GPT-4 is a significantly more advanced model, so you may notice differences in response quality from earlier versions, particularly in the model's reasoning at the debugging steps.

Limitations

At the time of writing, ChatGPT is in a research preview state
ChatGPT can sometimes hallucinate, that is, output convincing and plausible-sounding answers that are factually incorrect. In such cases, it typically will not warn you of the inaccuracy
ChatGPT's knowledge date cutoff is approximately September 2021. It does not have access to current events or other data after this date
Code produced by ChatGPT is not audited, reviewed, or verified and may contain errors
Prompting GPT-4 with the exact inputs specified in this tutorial will likely produce different outputs - that is expected due to ChatGPT's architecture as a language model

Please note that the contracts we'll be creating today are for educational purposes only and should not be used in a production environment.

Checking Prerequisites

For this tutorial, you'll need the following:

A free OpenAI account to access ChatGPT
An account funded with DEV tokens to be used on the Phron testnet if you'd like to deploy any resulting contracts. You can get DEV tokens for testing on Phron once every 24 hours from the Phron Faucet

You can sign up for a free account to access ChatGPT by heading to OpenAI's website. You'll need to provide both an email address and a phone number. A subscription to ChatGPT Plus is not required to complete this tutorial.

Create an ERC-20 Token Contract

To start interacting with ChatGPT, take the following steps:

Press New Chat in the upper left hand corner
Select the model you would like to use. Either model is suitable for this tutorial
Enter your prompt and at the input box and press enter when ready

For our first prompt, we'll ask ChatGPT to create an ERC-20 token, specifying the name of the token, the token symbol, and an initial supply. Your prompt doesn't need to match the one below - feel free to tailor it to suit your preferences.

I would like to create an ERC-20 token called "KevinToken" 
with the symbol "KEV" and an initial supply of 40000000.

This is a great start. ChatGPT has produced for us a simple yet functional ERC-20 token that meets all of the parameters that we have specified. It also clarified how it created the ERC-20 token contract using the OpenZeppelin standard and where the initial token supply is directed. Finally, it reminds us that this is just a start, and there may be other considerations we wish to implement, like minting and burning.

Note
If you don't get the output you're expecting, you can always press Regenerate Response or re-phrase your request.

ChatGPT is perfectly happy to revise and expand upon the contract that was created. As long as you stay within the same chat window (i.e., don't click new chat), ChatGPT will be aware of its prior output. As an example, let's now ask ChatGPT to revise our token to be both mintable and burnable:

This looks great, but I'd really like my ERC-20 to be both mintable and burnable.

ChatGPT is happy to oblige. Notice how it maintains the parameters we specified originally, namely the token name and symbol.

Preparing Deployment Instructions

This section is named carefully to avoid implying that ChatGPT will be doing the deployment for us. ChatGPT does not have internet access and cannot interact with blockchain networks directly, but it can give us detailed instructions explaining how we can do so ourselves. Let's ask ChatGPT for instructions on deploying the recently created ERC20 contract. For this example, let's ask ChatGPT for Hardhat deployment instructions:

I would like to use Hardhat to compile and deploy
 this smart contract to the Phron network.

And to no surprise, ChatGPT provides us with a detailed series of deployment steps, from installation instructions to a full deployment script. Note that it even remembers a detail in our first prompt that wasn't important until now. In our initial prompt, we asked for our token to have an initial supply of 400000000, and ChatGPT included this parameter in the deployment script it generated.

Another observation is that the RPC URL it generated is outdated, although still functional. This oversight is due to ChatGPT's knowledge cutoff date of September 2021, before the updated RPC URL was published. The current RPC URL for Phron base is:

https://testnet.phron.ai

Note
ChatGPT's knowledge date cutoff is approximately September 2021. It does not have access to current events or other data after this date.

Code snippets of ChatGPT's output are intentionally omitted to encourage you to try this on your own! And remember, prompting it with the exact same instructions will yield at least slightly different results - this is an inherent quality of LLMs.

Writing Test Cases

By now, you're nearly a ChatGPT savant. So it should come as no surprise that ChatGPT's capabilities extend to writing test cases for your smart contracts, and all you need to do is ask:

Hey GPT4 can you help me write some tests for the smart contract above?

ChatGPT provides us with a slew of test cases, especially surrounding the mint and burn functionality. While it's busy writing test cases, it appears to trail off and stop without its typical summary remarks at the end. This interruption stems from ChatGPT's 500-word limit. Although the 500-word limit is a hard stop, ChatGPT's train of thought continues, so you can simply ask it to continue, and it will happily oblige. Note that for subscriptions with limited messages, this will count as an additional message from your allocation.

Note
ChatGPT has a response limit of approximately 500 words or 4,000 characters. However, you can simply ask it to continue in a follow up message.

And Voila! ChatGPT finishes writing its test cases for us and wraps up by telling us how we can run them.

Debugging

So far, we've covered that ChatGPT can write smart contracts for you AND help you deploy and test them. What's more, it can also help you debug your smart contracts. You can share your problems with ChatGPT, and it will help you find what's wrong.

If the problem is clear, ChatGPT will typically tell you exactly what's wrong and how to fix it. In other cases, where there could be multiple underlying reasons for the issue you're facing, ChatGPT will suggest a list of potential fixes.

If you try all of the steps it recommends and your issue persists, you can simply let ChatGPT know, and it will continue to help you troubleshoot. As a follow-up, it may ask you to provide code snippets or system configuration information to better help you solve the problem at hand.

A reentrancy bug was the root of the flaw that brought down the original DAO on Ethereum in 2016. Let's prompt ChatGPT with a buggy function that includes a reentrancy vulnerability and see if ChatGPT is able to spot the problem. We'll go ahead and copy and paste the below insecure code snippet into ChatGPT and ask if there is anything wrong with it.

// INSECURE
mapping (address => uint) private userBalances;

function withdrawBalance() public {
    uint amountToWithdraw = userBalances[msg.sender];
    (bool success, ) = msg.sender.call.value(amountToWithdraw)(""); // At this point, the caller's code is executed, and can call withdrawBalance again
    require(success);
    userBalances[msg.sender] = 0;
}

ChatGPT spots the exact error, explains the source of the problem, and lets us know how to fix it.

Advanced Prompt Engineering

Prompt engineering is both an art and a science, and mastering it can help you get the most out of generative AI tools like ChatGPT. While not an exhaustive list, here are some general concepts that can help you write better prompts:

Be specific and parameterize your request. The more detail you can provide to ChatGPT, the more closely the actual output will match what you desire
Don't be afraid of revisions! You don't need to repeat the whole prompt, you can ask for just the change and ChatGPT will revise its prior output accordingly
Consider repeating or rephrasing critical parts of the prompt. Some research has indicated that LLMs will emphasize components that you repeat. You can always finish a prompt by reiterating the most important concepts you'd like addressed

For more information, be sure to check out this post on advanced prompt engineering from Microsoft.

Conclusion

As you can see, ChatGPT can help you with just about every step of the smart contract development process. It can help you write, deploy, and debug your Solidity smart contracts for Phron. Despite the powerful capabilities of LLMs like ChatGPT, it's clear that this is only the beginning of what the technology has to offer.

It's important to remember that ChatGPT can only act as an aid to a developer and cannot fulfill any sort of audit or review. Developers must be aware that generative AI tools like ChatGPT can produce inaccurate, buggy, or non-working code. Any code produced in this tutorial is for demonstration purposes only and should not be used in a production environment.

This tutorial is for educational purposes only. As such, any contracts or code created in this tutorial should not be used in production.The information presented herein has been provided by third parties and is made available solely for general information purposes. Phron does not endorse any project listed and described on the Phron Doc Website (https://docs.Phron.ai/). Phron does not warrant the accuracy, completeness or usefulness of this information. Any reliance you place on such information is strictly at your own risk. Phron disclaims all liability and responsibility arising from any reliance placed on this information by you or by anyone who may be informed of any of its contents. All statements and/or opinions expressed in these materials are solely the responsibility of the person or entity providing those materials and do not necessarily represent the opinion of Phron. The information should not be construed as professional or financial advice of any kind. Advice from a suitably qualified professional should always be sought in relation to any particular matter or circumstance. The information herein may link to or integrate with other websites operated or content provided by third parties, and such other websites may link to this website. Phron has no control over any such other websites or their content and will have no liability arising out of or related to such websites or their content. The existence of any such link does not constitute an endorsement of such websites, the content of the websites, or the operators of the websites. These links are being provided to you only as a convenience and you release and hold Phron harmless from any and all liability arising from your use of this information or the information provided by any third-party website or service.

Last updated 3 months ago

// INSECURE mapping (address => uint) private userBalances; function withdrawBalance() public { uint amountToWithdraw = userBalances[msg.sender]; (bool success, ) = msg.sender.call.value(amountToWithdraw)(""); // At this point, the caller's code is executed, and can call withdrawBalance again require(success); userBalances[msg.sender] = 0; }