Blog - ChatGPT Is Pretty Stupid

After some further development I have realised that ChatGPT is pretty stupid.

In short, given a set of coordinates for a hexagonal grid, it is unable to follow logic and compute around the grid. This is after its given an entire complete array with all the coordinates.

Essentially the top left hex is 0a with a flat side to the top. To the lower left is 0b and below that is 0c all the way down to 0n where n is the final limit.

The next column is the same, starting 1a, 1b, 1c etc to 1n.

Once given this information, you might have thought that ChatGPT could tell you all the hexes surrounding 5h. Wrong. It simply can’t do it. In fact, you can tell it the correct answer, and then as for the hex above 5h and it will get that wrong.

After I explained the map, again, ChatGPT said this:

Yes, I understand the hex map you're describing. It is a grid of hexagons arranged in columns and rows, where columns are labelled with numbers and rows with letters. The top-left hexagon is labelled as "0a," the one below it is "0b," and below that is "0c." To the top-right of "0b" is "1a," and so on. This layout creates a grid where each hexagon has six neighbouring hexagons.

So I asked: what hexes surround 3m

ChatGPT said:

The hexes surrounding "3m" in the hex map are:

• 2m (to the upper left)
• 4m (to the upper right)
• 3l (to the left)
• 3n (to the right)
• 2n (to the lower left)
• 4n (to the lower right)

These six hexagons are the immediate neighbours of "3m."

Which is incorrect. To start off with, there is not left or right, only a top and bottom, that s the orientation of the hex.

I prompted: wriong/ ChatGPT replied:

Apologies for the confusion. Upon re-examining the hex map, the hexes surrounding "3m" are as follows:

• 2m (to the left)
• 4m (to the right)
• 2l (to the upper left)
• 3l (to the upper right)
• 2n (to the lower left)
• 3n (to the lower right)

These six hexagons are the immediate neighbours of "3m."

Which is wrong again.

I gave ChatGPT the correct answer:

3k (to the top) 3o (to the bottom) 2l (to the upper left) 3l (to the upper right) 2n (to the lower left) 3n (to the lower right)

ChatGPT replied:

Thank you for providing the correct layout. I understand now. The hexes surrounding 3m in the hex map are:

• 3k (to the top)
• 3o (to the bottom)
• 2l (to the upper left)
• 3l (to the upper right)
• 2n (to the lower left)
• 3n (to the lower right)

I appreciate your patience and clarification. If you have any further questions or need assistance, please feel free to ask.

Based on that I asked for the surrounding hexes for 6k and it got that wrong and reorientated the hexs.

To cut a long story short, no amount of correcting ChatGPT resulted in it getting the answer right at all.

It clearly does not calculate. this is such an outrageous failure that its clear that ChatGPT clearly cannot carry out computational instructions. That’s not to say AI can't, and maybe ChatGPT isn't designed to do that, but if you want to use a ChatGPT API for any php computations you are out of luck. It can't even do the basics.

However, it can complete this:

cat is to dog as water is to

fire

Whilst ChatGPT is great for the donkey work on basic blurb generation, it clearly has limits and I suspect that that is built in as otherwise it could be a very powerful system.

TWS is involved in some fairly advanced php based spatial modelling systems and the use of hex grids is fairly well developed and it is complex but the systems that TWS has developed are effectively fool-proof and so it was only natural to see how ChatGPT could read and understand a simple map. Basically it cant, and so the prospect of further development in that avenue is mute.