AI for Everybody - Lesson 6
What Is This Thing? Why "It Feels Like It Understands" Is the Wrong Question
GPT-4 passed the bar exam at the 90th percentile of human test-takers (Katz et al., 2024). GPT-4 has, in well-documented public sessions, confidently insisted that 9.11 is larger than 9.9. The same system does both things. If you go looking for evidence that it understands, you can find it. If you go looking for evidence that it does not, you can find that too.
This is the question Part 1 cannot answer, and this lesson is going to be careful with it. We will close Part 1 by naming the question, drawing the map of the field we have been crossing, and looking honestly at how the smartest people in the field disagree about whether anything inside a chatbot deserves to be called comprehension. The point of the capstone is not to give you the answer. The point is to make sure you do not borrow somebody else’s answer without realizing you have done it.
