AI Writing Detection Tools are Faulty and Here's Why

AI Writing Detection Tools are Faulty and Here's Why

📟 Tech

Why AI-detection tools are error-prone


Have you ever wondered how accurate AI writing detection tools really are? Recently, I tested several texts, from simple to more complex, to see if they were flagged as AI-generated. The results were surprising, and it raised questions about the effectiveness of AI detection tools (AIDTs) and the extent to which we should rely on them.

🙊 Caveman English OK, Formal English No OK

Something as simple as stating the Pythagorean Theorem is automatically detected as AI-generated.

In order to determine the length of a side in a right-angled triangle, algebraic techniques must be applied to rearrange the variables in such a way that the square of the hypotenuse is expressed as the sum of the squares of the other two sides. By performing these mathematical manipulations, the resulting equation can be solved by taking the square root of both sides, thereby obtaining the length of the desired side. This process comes from the Pythagorean theorem, which states that the square of the hypotenuse is equal to the sum of the squares of the other two sides in a right-angled triangle.

To verify this, I tried to dumb down my language and make it sound casual by using this prompt:

If you want to get a length of a side in a right-angled triangle, rearrange the variables so that the square of the hypotenuse is shown as the sum of the squares of the other two sides. Doing this gives you an equation that can be solved by taking the square root of both sides. After this, you can get the length of the side that you want. This universal fact is called the "Pythagorean theorem". It means that the square of the hypotenuse is equal to the sum of the squares of the remaining two sides.

I literally couldn't make this sound any simpler, so I asked ChatGPT to say what I just said, but in a way that sounds like a "caveman".

After running it through AI, it detected it as ZERO percent AI-generated.