### State-Observable Duality (Part 3)

#### Posted by John Baez

This is the third and final episode of a little story about the foundations of quantum mechanics.

In the first episode, I reminded you of some basic facts about the real numbers $\mathbb{R}$, the complex numbers $\mathbb{C}$, and the quaternions $\mathbb{H}$.

In the second episode, I told you how Jordan, von Neumann and Wigner classified ‘formally real Jordan algebras’, which can serve as algebras of observables in quantum theory. Apart from the ‘spin factors’ $\mathbb{R}^n \oplus \mathbb{R}$ and the Jordan algebra of $3 \times 3$ self-adjoint octonionic matrices, $\mathrm{h}_3(\mathbb{O})$, these come in three kinds:

- The algebra $\mathrm{h}_n(\mathbb{R})$ of $n \times n$ self-adjoint real matrices with the product $a \circ b = \frac{1}{2}(a b + b a)$.
- The algebra $\mathrm{h}_n(\mathbb{C})$ of $n \times n$ self-adjoint complex matrices with the product $a \circ b = \frac{1}{2}( a b + b a)$.
- The algebra $\mathrm{h}_n(\mathbb{H})$ of $n \times n$ self-adjoint quaternionic matrices with the product $a \circ b = \frac{1}{2}(a b + b a)$.

In every case, even the curious exceptional cases, there is a concept of what it means for an element to be ‘positive’, and the positive elements form a cone.
In this episode we’ll explore that further: we’ll meet the Koecher–Vinberg classification of convex homogeneous self-dual cones, and see how it’s really all about *state-observable duality*.

Last time we talked about Jordan algebras and their role in the foundations of quantum mechanics. But the formalism of Jordan algebras seems rather removed from the actual practice of physics. After all, physicists hardly ever take two observables $a$ and $b$ and form their Jordan product
${1\over 2}(a b + b a)$. As I hinted last time, it is better to think of this operation as derived from the process of *squaring* an observable, which is something physicists actually do. But still, I can’t help wondering: can we see the classification of finite-dimensional formally real Jordan algebras, and thus the special role of normed division algebras, as arising from some axioms more closely tied to quantum theory as physicists usually practice it?

One answer involves the duality between states and observables. To understand this, you need to know about ‘mixed states’. A ‘state’ describes your knowledge about a physical system. If you know as much as possible we call it a ‘pure state’, but more generally, you may not know as much as possible, and then we speak of ‘mixed states’. It’s usually best to consider pure states as a special case of mixed states.

How does this play out in ordinary quantum theory? If a quantum
system has the Hilbert space $\mathbb{C}^n$, observables are described by
self-adjoint $n \times n$ complex matrices: elements of the Jordan
algebra $\mathrm{h}_n(\mathbb{C})$. But matrices of this form that are nonnegative and have trace 1 also play another role. They are called ** density matrices**, and they describe mixed states of our quantum system. The idea is that any density matrix $\rho \in \mathrm{h}_n(\mathbb{C})$ lets us define expectation values of observables $a \in \mathrm{h}_n(\mathbb{C})$ via
$\langle a \rangle = \mathrm{tr}(\rho a) .$
The map sending observables to their expectation values is
real-linear. The fact that $\rho$ is nonnegative is equivalent to
$a \ge 0 \; \implies \; \langle a \rangle \ge 0$
and the fact that $\rho$ has trace 1 is equivalent to
$\langle 1 \rangle = 1 .$

We might call this relationship between states and observables
‘state-observable duality’. By this, we are not merely referring to
the fact that states live in the dual of the vector space of
observables: that much is obvious, given that the expectation value of
an observable should depend linearly on that observable. The
nontrivial thing is that we can identify the vector space of
observables with its dual:
$\begin{aligned}
\mathrm{h}_n(\mathbb{C}) &\stackrel{\sim}{\longrightarrow}& \mathrm{h}_n(\mathbb{C})^* \\
a &\mapsto& \langle a, \cdot \rangle
\end{aligned}$
using the trace, which puts a real-valued inner product on the
space of observables:
$\langle a, b \rangle = \mathrm{tr}(ab) .$
Thus, *states can be identified with certain special observables!*

All this generalizes to an arbitrary finite-dimensional formally real
Jordan algebra $A$. Every such algebra automatically has an identity
element. This lets us define a ** state** on $A$ to be a
linear functional $\langle \cdot \rangle : A \to \mathbb{R}$ that is **
nonnegative**:
$a \ge 0 \implies \langle a \rangle \ge 0$
and ** normalized**:
$\langle 1 \rangle = 1 .$
But in fact, there is a one-to-one correspondence between
linear functionals on $A$ and elements of $A$. The reason is that
every finite-dimensional Jordan algebra has a ** trace**
$\mathrm{tr} : A \to \mathbb{R}$
defined so that $\mathrm{tr}(a)$ is the trace of the linear operator ‘multiplication by $a$’. Such a Jordan algebra is then formally real if and only if
$\langle a, b \rangle = \mathrm{tr}(a \circ b)$
is a real-valued inner product. So, when $A$ is a finite-dimensional
formally real Jordan algebra, any linear functional
$\langle \cdot \rangle : A \to \mathbb{R}$ can be written as
$\langle a \rangle = \mathrm{tr}(\rho \circ a)$
for a unique element $\rho \in A$. Conversely, every element $\rho
\in A$ gives a linear functional by this formula. While not obvious,
it is true that the linear functional $\langle \cdot \rangle$ is
nonnegative if and only if $\rho \ge 0$ in terms of the ordering on
$A$. More obviously, $\langle \cdot \rangle$ is normalized if and
only if $\mathrm{tr}(\rho) = 1$. So, *states can be identified with certain
special observables*: namely, those observables $\rho \in A$ with
$\rho \ge 0$ and $\mathrm{tr}(\rho) = 1$.

In short: whenever the observables in our theory
form a finite-dimensional formally real Jordan algebra, we have
state-observable duality. But what is the physical meaning of
state-observable duality? Why in the world should *states* correspond to special *observables?* A state is a way for your system to be; an observable is something you can measure about it. They seem quite different!

Here is one attempt at an answer. Every finite-dimensional formally real Jordan algebra comes equipped with a distinguished observable,
the most boring one of all: the identity, $1 \in A$. This is
nonnegative, so if we normalize it, we get an observable
$\rho_0 = \frac{1}{\mathrm{tr}(1)} \, 1 \in A$
of the special kind that corresponds to a state. This state, say
$\langle \cdot \rangle_0$, is just the normalized trace:
$\langle a \rangle_0 = \mathrm{tr}(\rho_0 \circ a) =
\frac{\mathrm{tr}(a)}{\mathrm{tr}(1)} .$
And this state has a clear physical meaning: it is the ** state of maximal ignorance**! It is the state where we know as little as possible about our system — or more precisely, at least in the case
of ordinary complex quantum theory, the state where entropy is
maximized.

For example, take $A = \mathrm{h}_2(\mathbb{C})$, the algebra of observables of a spin-$\frac{1}{2}$ particle. Then the space of states is the so-called Bloch sphere, really a 3-dimensional ball.

The ball is convex, and for a good reason Suppose I flip a coin, don’t show you the result, and tell you “I made the particle’s state be $a$ if the coin landed heads up, and $b$ if it landed tails up”. Then the mixed state that describes your knowledge is halfway between $a$ and $b$. More generally any convex linear combination $p a + (1-p)b$ of mixed states is another mixed state, where the probability $p$ is between $0$ and $1$. That’s what we mean by saying the ball is convex. Indeed, for *any* formally real Jordan algebra, the space of states is convex!

So, on the surface of this ball are the pure states: the states where you know as much as possible. For any point on this surface, there’s a state where you know the electron’s spin points that way. At the center of the ball is the state of maximum ignorance. This corresponds to the density matrix $\rho_0 = \left(\begin{aligned} \frac{1}{2} &\; & 0 \\ 0 & & \frac{1}{2} \end{aligned}\right)$ In this state, when I ask you about the particle’s spin along any axis, all you can say is that there’s a $\frac{1}{2}$ chance that it’s pointing one way, and a $\frac{1}{2}$ chance of it pointing the other way.

Now, back to the general theory:

$A$ acts on its dual $A^*$: given $a \in A$ and a linear functional $\langle \cdot \rangle$, we get a new linear functional
$\langle a \circ \cdot \rangle$. This captures the idea, familiar
in quantum theory, that observables are also ‘operators’: they
*act* on states. And state-observable duality means we can
get any state from the state of complete ignorance by act on it
with a suitable observable. After all, any state corresponds
to some observable $\rho$, as follows:
$\langle a \rangle = \mathrm{tr}(\rho \circ a)$
So, we can get this state by acting on the state of
maximal ignorance, $\langle \cdot \rangle_0$, by the observable
$\mathrm{tr}(1) \rho$:
$\langle \mathrm{tr}(1)\rho \circ a \rangle_0 =
\frac{\mathrm{tr}(1)}{\mathrm{tr}(1)} \mathrm{tr}(\rho \circ a) =
\langle a \rangle .$
So, we see that the correspondence between states and special observables springs from two causes. First, there is a distinguished state, the state of maximal ignorance. Second, any other state
can be obtained from the state of maximal ignorance by acting on
it with a suitable observable.

While these ideas raise a host of questions, they also help motivate an important theorem of Koecher and Vinberg. The idea is to axiomatize the situation we we have just described, in a way that does not mention the Jordan product in $A$, but instead emphasizes:

- state-observable duality,
- the fact that ‘positive’ observables, namely those whose observed values are always positive, form a cone.

To find appropriate axioms, suppose $A$ is a finite-dimensional formally real Jordan algebra. Then seven facts are always true.

First, the set of positive observables
$C = \{a \in A \colon a \gt 0\} .$
is a ** cone**: that is, $a \in C$ implies that every positive
multiple of $a$ is also in $C$. Second, this cone is ** convex**:
if $a,b \in C$ then any linear combination $p a + (1-p)b$ with
$0 \le p \le 1$ also lies in $C$. Third, it is an open set. Fourth, it is
** regular**, meaning that if $a$ and $-a$ are both in the closure
$\overline{C}$, then $a = 0$. This last condition may seem obscure,
but if we note that
$\overline{C} = \{ a \in A \colon a \ge 0 \}$
we see that $C$ being regular simply means
$a \ge 0 \; and \; -a \ge 0 \quad
\implies \quad a = 0 ,$
a perfectly plausible assumption.

Next recall that $A$ has an inner product; this is what lets us
identify linear functionals on $A$ with elements of $A$. This also
lets us define the ** dual cone**
$C^* = \{ a \in A \colon \forall b \in A \; \;
\langle a,b \rangle \gt 0 \}$
which one can check is indeed a cone. The fifth fact about $C$ is
that it is ** self-dual**, meaning $C = C^*$. This formalizes the
notion of state-observable duality!

The sixth fact is $C$ is also ** homogeneous**: given any two points $a,b \in C$, there is a real-linear linear transformation $T :
A \to A$ mapping $C$ to itself in a one-to-one and onto way, with
the property that $Ta = b$. This says that cone $C$ is highly symmetrical: no point of $C$ is any ‘better’ than any other, at least if we only consider the linear structure of the space $A$, ignoring the Jordan product and the trace.

From another viewpoint, however, there is a very special point of $C$,
namely the identity $1$ of our Jordan algebra. And this brings us to
our seventh and final fact: the cone $C$ is **pointed**, meaning
that it is equipped with a distinguished element (in this case $1 \in
C$). As we have seen, this element corresponds to the ‘state of complete ignorance’, at least after we normalize it.

In short: when $A$ is a finite-dimensional formally real Jordan algebra, $C$ is a pointed homogeneous self-dual regular open convex cone. All the elements $a \in C$ are positive observables, but certain special ones, namely those with $\langle a, 1 \rangle = 1$, can also be viewed as states.

In fact, there is a category of pointed homogeneous self-dual regular open convex cones, where:

- An object is a finite-dimensional real inner product space $V$ equipped with a pointed homogeneous self-dual regular open convex cone $C \subset V$.
- A morphism from one object, say $(V,C)$, to another, say $(V',C')$, is a linear map $T : V \to V'$ preserving the inner product and mapping $C$ into $C'$.

Now for the payoff. The work of Koecher and Vinberg, nicely explained in Koecher’s Minnesota notes:

- Max Koecher,
*The Minnesota Notes on Jordan Algebras and Their Applications*, eds. Aloys Krieg and Sebastican Walcher,*Lecture Notes in Mathematics***1710**, Springer, Berlin, 1999.

shows that:

**Theorem:** The category of pointed homogeneous self-dual
regular open convex cones is equivalent to the category of finite-dimensional formally real Jordan algebras.

This means that the theorem of Jordan, von Neumann and Wigner, which we saw last time, also classifies the pointed homogeneous self-dual regular convex cones!

**Theorem:** Every pointed homogeneous self-dual
regular open convex cones is isomorphic to a direct sum of
those on this list:

- the cone of positive elements in $\mathrm{h}_n(\mathbb{R})$,
- the cone of positive elements in $\mathrm{h}_n(\mathbb{C})$,
- the cone of positive elements in $\mathrm{h}_n(\mathbb{H})$,
- the cone of positive elements in $\mathrm{h}_3(\mathbb{O})$,
- the future lightcone in $\mathbb{R}^n \oplus \mathbb{R}$.

Some of this deserves a bit of explanation. For $\mathbb{K} = \mathbb{R}, \mathbb{C}, \mathbb{H}$, an element $T \in \mathrm{h}_n(\mathbb{K})$ is ** positive** if and only if the
corresponding operator $T : \mathbb{K}^n \to \mathbb{K}^n$ has
$\langle v, T v \rangle \gt 0$
for all nonzero $v \in \mathbb{K}^n$. A similar trick works for defining
positive elements of $\mathrm{h}_3(\mathbb{O})$, but we do not need the details here. We say an element $(x,t) \in \mathbb{R}^n \oplus \mathbb{R}$ lies in the ** future lightcone** if $t \gt 0$ and $t^2 - x \cdot x \gt 0$. This
of course fits in nicely with the idea that the spin factors are Minkowski spacetimes. Finally, there is an obvious
notion of direct sum for Euclidean spaces with cones, where the
direct sum of $(V,C)$ and $(V',C')$ is $V \oplus V'$ equipped with
the cone
$C \oplus C' = \{(v,v') \in V\oplus V' \colon \;
v \in C, v' \in C' \} .$

In short: self-adjoint operators on real, complex and quaternionic Hilbert spaces arise fairly naturally as observables starting from a formalism where nonnegative observables form a cone, and we insist on state-observable duality.

There is a well-developed approach to probabilistic theories that works for cones that are neither self-dual nor homogeneous: see for example the work of Howard Barnum and his coauthors. This has already allowed Barnum, Gaebler and Wilce to shed new light on the physical significance of self-duality. But perhaps if we think more about state-observable duality we can better understand its meaning… and thus the appearance of normed division algebras in quantum physics!

Finally, as Urs Schreiber pointed out, it is worth comparing state-observable duality to the ‘state-operator correspondence’. This was made popular in the context of string theory, but it really applies whenever we have a $C^*$-algebra of observables, say $A$, equipped with a state $\langle \cdot \rangle : A \to \mathbb{C}$. Then the Gelfand-Naimark-Segal construction lets us build a Hilbert space $H$ on which $A$ acts, together with a distinguished unit vector $v \in H$ called the ‘vacuum state’. The Hilbert space $H$ is built by completing a quotient of $A$, so a dense set of vectors in $H$ come from elements of $A$. Thus again, some observables give us states. In particular, the vacuum state $v$ comes from the element $1 \in A$.

This is reminiscent of how in the Jordan algebra framework, the state of maximal ignorance comes from the element $1$ in the Jordan algebra of observables. But there are also some differences: for example, the Gelfand-Naimark-Segal construction requires choosing a state, and it works for infinite-dimensional $C^*$-algebras, while our construction works for finite-dimensional formally real Jordan algebras, which have a *canonical* state: the state of maximum ignorance. Presumably both constructions are special cases of something more general.

## Re: State-Observable Duality (Part 3)

I’d say if anyone is interested in exploring the state of the art of the classification of state spaces of operator algebras, a good place to start would be