The Game's Structure

Consider a scenario where you are a player alongside an advanced Artificial General Intelligence (AGI) that possesses far greater intelligence and computational power than any human.

You are faced with two boxes: Box A and Box B.

Box A is clear and contains $1,000, while Box B is opaque, hiding either $1,000,000 or nothing. The content of Box B depends on the AGI's prediction of your choice.

Your options are:

1. Select both Box A and Box B.
2. Choose Box B alone.

The AGI's prediction does not influence the contents of Box A. If it anticipates you will take both boxes, it places $0 in Box B; if it expects you to select only Box B, it fills it with $1,000,000.

With the rules laid out, would you like to play a round?

What’s Your Choice?

Upon first encountering this game, I thought, “The choice is clear!”

If you're anything like me, you likely have a predetermined preference. However, after reviewing research and survey data surrounding this game, I realized my initial thought was misguided. The data indicated an almost equal split in choices among participants.

For instance, a 2020 survey of 714 professional philosophers revealed a slight majority favoring Choice 1 over Choice 2 (39.03% vs. 31.19%).

What’s happening here? To delve deeper, let's examine the game through a game-theoretic perspective.

Newcomb’s Paradox and Game Theory

To understand this paradox, we need to analyze the possible outcomes:

We can approach this scenario using two game theory strategies: Expected Utility Theory and the Strategic Dominance Principle.

According to the expected utility theory, decisions should be made based on probabilities. If we view the AGI's predictive capability as nearly perfect (probability nearing 1), opting for both boxes leads to an expected return of about $1,000. This suggests that selecting Box B alone (Choice 2) would yield a higher expected reward of approximately $1,000,000.

Conversely, the strategic dominance principle disregards the AGI's capabilities, recommending the strategy that consistently offers better outcomes. Here, choosing both boxes (Choice 1) guarantees $1,000, which is more than just selecting Box B (Choice 2).

To summarize:

1. Expected Utility Theory suggests choosing Box B only (Choice 2).
2. Strategic Dominance Principle advocates for selecting both boxes (Choice 1).

Thus, game theory presents a split perspective. Depending on your viewpoint, one choice may seem more "obvious." Yet, humanity remains divided on the optimal strategy, exemplifying the paradox.

Now that we have explored the paradox's intricacies, let's consider its implications for deeper issues.

Newcomb’s Paradox: Free Will and Consciousness

The core issue with Newcomb's paradox lies in the existence of the advanced AGI. If we generalize this intelligence, it suggests the possibility of accurately predicting human behavior.

This notion raises an important question: "Is free will a genuine phenomenon?"

If free will exists, your choices should be unpredictable enough to defy the AGI's predictions. However, if the AGI can accurately foresee your decisions, a causal relationship emerges between your actions and the AGI’s predictions, invoking classic philosophical dilemmas like the “chicken and egg” problem.

While we won’t dive deeper into these complexities, it’s critical to acknowledge that such philosophical ramifications arise when investigating Newcomb's paradox further.

Consciousness Considerations

Newcomb's paradox also challenges our understanding of machine consciousness. For the AGI to make accurate predictions, it must somehow replicate human consciousness. If it succeeds, you might question whether you are genuinely engaging with the boxes or merely experiencing a simulation within the AGI's environment.

In this hypothetical, as the "virtual" chooser, you might effectively communicate your choice to the AGI, leading you to consistently opt for Choice 2 (Box B only) in each round of the game.