Skip to content
Cracking the Code: ChatGPT's Mathematical Missteps (Technical Insights) episode artwork

EPISODE · Mar 17, 2024 · 7 MIN

Cracking the Code: ChatGPT's Mathematical Missteps (Technical Insights)

from Hard Fork AI

In this episode, we crack the code of ChatGPT's mathematical missteps, providing technical insights into the challenges of integrating numerical reasoning into AI language models and uncovering potential solutions. Get on the AI Box Waitlist: ⁠⁠⁠https://AIBox.ai/⁠⁠⁠ Join our ChatGPT Community: ⁠⁠⁠⁠https://www.facebook.com/groups/739308654562189/⁠⁠⁠⁠ Follow me on Twitter: ⁠⁠⁠⁠https://twitter.com/jaeden_ai⁠ See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Episode metadata supplied by the publisher feed · Published Mar 17, 2024

Embed this episode

Ready to play

Cracking the Code: ChatGPT's Mathematical Missteps (Technical Insights)

0:00 7:34
of MATCHES

TRANSCRIPT · AUTO-GENERATED

can chat JPT do math. This is a question a lot of people have had, especially people in school, or even just if you're trying to get some quick back of the envelope math done, right on the fly. A lot of people want to know, you know, can chat JPT do math. They can do coding, they can do writing, they can do a lot of things, but is it good at math?

And this is actually one area that chat JPT is not good at, I would say. In fact, to the point that it's pretty well recognized, it is pretty notoriously bad at math. And the reason for this is because the way chat JPT works. Chat JPT is a predictive language model, meaning all it's doing is typically predicting the next word that should come in a sentence.

And so it's not really sitting there and computing a lot of what's going on. And again, we also see that problem with fact checking. You know, it'll come up with lots of different facts that are not accurate. Typically when I find it writing stories, it's going to get the dates wrong.

It's going to get the amounts wrong. It's going to get the dollars wrong. And so it's definitely not very good at math. But the problem is that it can be very convincing in its responses.

So today we're going to dive into a little bit about how chat JPT works in regards to the math side of things, looking at some strengths, looking at some weaknesses, and overall at the end of the day, what it can do. And then I'm going to run an actual test through chat JPT to see if it's able to solve my math problem test. You guys can follow along, see if you guys can get it in the back of your head. So I guess the short answer is yes, chat JPT can perform basic arithmetic operations, such as addition, subtraction, multiplication, and division.

However, its mathematical abilities are pretty limited to just simple arithmetic. And it does not extend to more advanced mathematical concepts such as calculus or linear algebra. That being said, if you ask it a question about calculus or linear algebra, it will typically try to answer it. And it's pretty interesting.

Just recently, chat JPT came up with a new update, which they said their goal on that new version was to make it better at doing math. Exclusively, that's pretty much the only thing that happened in the update. And yet, even with that, I'm still finding it not doing an amazing job. So that is definitely downside.

But one of the other strengths of chat JPT is that when it comes to math, its ability to understand natural language inputs and provide an output in a conversational format is kind of useful. So for example, if you were to ask it, what is 2 plus 2? Chat JPT would respond 2 plus 2 is equal to 4, right? So you don't necessarily have to put the symbol to a plus symbol and 2.

You could just say the word. And you could say similar things, right? You can explain the situation. And theoretically, it's supposed to try and solve that using simple human English.

That being said, it's not always very good of that, to be honest. The other thing that they've tried to build into this is the ability to handle multi-step calculations and word problems. So for instance, if John has five apples and he gives way two, how many does he have left? Chat JPT should respond.

John would have three apples left because it can understand the text and you can make some multiple step things. That being said, I have seen cases where it was unable to do those types of problems. Something that's very interesting is if you literally ask Chat JPT, what is your weakness in regard to math? It's going to say something like, one major weakness of Chat JPT is its lack of understanding of mathematical notations and symbols.

Well, it can handle basic arithmetic and operations when expressed in natural language. It struggles to understand mathematical expressions when in symbolic form. So for example, if you use a word to ask what is the value of 2x plus 3y when x equals 2 and y equals 4, Chat JPT would not be able to provide a correct answer. So it literally, it's explicitly states, it's not able to solve problems that are using symbols in them.

But if you did ask that question, it will give you an answer and it will break down the whole answer, whether that's correct or not. I think it's usually not correct. But it literally is trying. So I guess the problem there is there's a much of a disclaimer on math on Chat JPT, and it's typically getting it wrong.

So another one of its big drawbacks is it does not have a good understanding of higher level mathematical concepts. So it can perform basic operations in math, but it does not have the ability to solve more complex things such as different equations or vector calculus. So it kind of limits its usefulness. But all in all, I think it's got a long way to go before it's going to be replacing calculators for people to just talk to.

So I actually went ahead and tried giving it a math problem that I heard someone online in a forum say struggled with. And my math problem was if it takes five machines, five minutes to make five devices, how long would it take? 100 machines to make 100 devices. And this is kind of like a silly word problem that if someone wasn't really paying attention, they'd say five machines, five minutes, five devices, 100 machines, 100 minutes, 100 devices.

I guess that's like the trick you're trying to get someone to say. In reality, five machines, in five minutes, making five devices just mean that each machine takes five minutes to make a device. So obviously, that would mean that it would take 100 machines, five minutes as well to make 100 devices. Nothing would change.

But if you ask Chat JPT, this is the response it gave me. It says if five machines can make five devices in five minutes, then we calculate the rate of production for each machine. Five devices divided by five minutes equals one device per minute. Since the rate of production is constant, for each machine, we can use this value to calculate the total production rate of 100 machines, 100 machines times.

And then it's gotten brackets, one device divided by a minute equals 100 devices per minute. So if 100 machines were working together, they'd be able to produce 100 devices in one minute, which is not true. It would actually take five minutes. So it's not like a super complex problem.

So I guess I just kind of give this a little bit of an overview of Chat JPT and math. It can do basics. If I said what's 100 plus 234, and it said the results 100 plus 234 is 334. But honestly, that's the most basic math problem you could get.

You type the same thing in Google, and it would be able to give it to you just with their in-search calculator function. So honestly, obviously not very impressive. So I guess, in conclusion on this whole thing, Chat JPT has some mathematical abilities. They're pretty limited to basic arithmetic operations and very simple word problems.

But it lacks the ability to understand high level stuff. It lacks the ability to honestly to even do a lot of civil word problems. And so it's really not a good tool for math, all in all, I would say they've got a long ways to go. They obviously are aware of it.

They just did a big update. So it will be to be determined how they improve this in the future.

No similar episodes found.

Frequently Asked Questions

How long is this episode of Hard Fork AI?

This episode is 7 minutes long.

When was this Hard Fork AI episode published?

This episode was published on March 17, 2024.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this Hard Fork AI episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.
URL copied to clipboard!