Why Stuff is Hard

Why is stuff hard? That is, how can matter become solid, instead of just floating through us (or into the center of the earth) like a ghost? It might seem like a silly question, that the burdon of explaination ought to be on exceptions to the rule, such as holograms and optical illusions. But as I learned about matter from a particle physics perspective, I became increasingly perplexed that this stuff ever manages to condense itself into anything concrete. I carried this question around with me for years before finding the explaination at the end of this article; I’m surprised how rarely it is addressed in detail.

Why stuff might be soft

Fundamental particles of matter, according to Sir Isaac Newton centuries ago, or Democritus millenia ago, are hard, solid shapes that can be stacked and stuck together (with little hooks, in Democritus’s theory). News graphics of particle physics reactions suggest a similar picture, rendering electrons and quarks as shaded spheres eminating from a billards collision. For most of our history, we have concieved of matter as something which occupies space exclusively, with an inclination toward defining its reality by its impenetrability. When MacBeth saw an intangible knife floating before him,

Art thou not, fatal vision, sensible
To feeling as to sight? or art thou but
a dagger of the mind, a false creation
Proceeding from the heat-oppressed brain?

it was either imaginary or conjured by witches. When Samuel Johnson heard Berkeley’s theory that all physical objects exist only in the mind as ideas, he pronounced, “I refute it thus!” and kicked a large stone. He could not have imagined how many neutrinos, cosmic rays, and (very likely) dark matter particles were pouring through his body at that instant.

Today, we define matter using quantum field theory, the culnumation of the first 50 years of trying to understand quantum mechanics. Quantum field theory is a framework, rather than a single theory, only making predictions when given a set of fundamental fields and interactions. The goal of particle physics is to identify these inputs (or, if that doesn’t work, improve upon the framework).

A classical field, in this language, is a function from every point in space-time to a number, spinor, vector, tensor, or some other structure of numbers. The first fields discussed in earnest were the electric field and the magnetic field, both of which map points in space-time to 3-component vectors. These fields are manifestly real (they make telegraphs work) even though they fill all of space, permeating all matter, as well as each other. The crowning demonstration of the reality of these fields came when Maxwell predicted the existence of radio in 1873 (13 years early) as self-perpetuating waves in the electromagnetic field. In quantum field theory, all matter consists of self-perpetuating waves in one of several quantum fields: the up-quark field, the down-quark field, the neutrino field, etc. These fields fill all of space— an empty vacuum is simply a region without waves. In modern language, we might call Maxwell’s electromagnetic field the photon field, the field of photon particles.

A quantum field differs from a classical field in that it is a probability distribution over classical fields (plus a “phase,” an angle in an abstract space, which is not important for this discussion). This probability distribution is constrained by a differential equation called the Schrödinger equation, which often restricts energy to a discrete set of values. In particular, the energy of a standing wave, a particle at rest, is forced to be an integer multiple of the particle mass. If we want to add waves to the electron field (that is, make electrons), we can only add 0.511 MeV (one electron), 1.022 MeV (two electrons), or 1.533 MeV (three electrons), etc. The very fact that matter comes in particles of fixed mass, rather than a mushy continuum, is a consequence of quantum mechanics!

The quantum field is therefore both more free and more constrained than the classical field, as illustrated below. The energy in a quantum field can’t be any arbitrary value, but it can be several restricted values at the same time.

Derivation of quantized particles with minimal prerequisites

This is an aside, but if you’d like to see where this quantization comes from, the following derivation only requires first-year differential equations. In the simplest case of a real-valued field with no interactions, the Schrödinger equation is

where is the quantum field, (the square root of) a probability distribution over the 5-dimensional space , where is the classical field value. (We could think of the classical field as being a single point in that 5-dimensional space. That’s equivalent to a function from 4 dimensions to a real number.) The term is the potential energy: an energy cost that penalizes large values of (Einstein’s equivalence between mass and energy).

If we divide both sides by and assume that factorizes into a function of space-time multiplied by a function of classical field value, this differential equation becomes seperable (justifying the assumption). The left-hand side of the equation would then only depend on and the right-hand side would only depend on . Therefore, both sides must equal a constant, suggestively called for mass. The left-hand side is a wave equation in space and time with energy and momentum related by

and the right-hand side becomes

with being the factor of depending on only. This equation is hard to satisfy; this is what constrains the masses of particles to be integers times . The solution is

where are Hermite polynomials for integers .

Excitations of the the real-valued quantum field are therefore waves with (the solution to the left-hand side) constrained to be (to solve the right-hand side). The actual field values are spread over a continuum on both sides of zero— if the energy is single-valued, the field amplitude cannot be. This is Heisenberg’s uncertainty principle in the field theory context.

Getting back to Why stuff might be soft

Given this picture of matter as waves, it’s hard to imagine how it could ever coalesce into something solid. In fact, the above example doesn’t. These waves pass through each other, doubling the energy in the region where they superimpose, returning to their original shapes as they continue on their way. It was a simple example without interactions; a more realistic treatment would include extra terms that allow energy to flow from one field to another, in the same way that vibrational energy flows from a cello string to its sounding board to the air in a concert hall. A field without interactions vibrates in isolation, unable to be heard. This is nearly the case for the neutrino field: there are ten times as many neutrinos by mass than heavy elements like carbon and oxygen (which is like saying there are four times as many ants than humans, by mass), but they interact so weakly with our matter that a few detections per day is a good rate for a ton-scale detector. Neutrinos do not form solid structures.

Particle physicists have identified four fundamental interactions in nature:

• Electromagnetism: charged particles attract or repel each other by exciting the photon field, to which they are both coupled.
• Weak Nuclear force: particles change species by de-exciting one field and exciting two others in their place: e.g. a down quark becomes an up and a . This is how many radioactive isotopes decay.
• Strong Nuclear force: holds quarks and nuclei together with gluons, rather than photons; very short-range.
• Gravitation: the medium of exchange is the metric of space-time itself. Gravitons are virtual excitations of the curvature of space-time, treated as a quantum mechanical field.

The “contact force” that keeps solids from pushing through each other is obviously not derived from gravity, and it acts over distances which are too large to be related to the Strong Nuclear force. The Weak Nuclear force is too weak, and that would turn electrons into neutrinos anyway. So we’re left with electromagnetism.

Explaination #1: Electromagnetic force makes stuff hard

Electromagnetism is responsible for nearly all macroscopic phenomena, the major exception being gravity. It is certainly the reason small things stick together: neutral atoms can be polarized and attract each other at short distances, even though they each have zero total charge. Many molecules, like water, have permanent electric dipoles which make it bead up into drops and crawl up the edges of a glass beaker. Water’s dipole and oil’s lack of a strong dipole are together responsible for all the hydrophilic/hydrophobic mechanisms in biology, such as keeping our cells from bursting open. But it’s not clear that electromagnetism can be solely responsible for holding things apart. I have never heard a description of exactly how electromagnetism is supposed to do it, and there are some general facts about electromagnetism that seem to preclude its being responsible for the contact force.

The simplest way to hold things apart is to make them out of like charges, since like charges repel electrostatically (that is, without magnetism). Ignoring for the moment that ordinary matter is resolutely neutral, any residual charges being immediately screened by humidity or punished with an electric shock, there’s a theorem by Samuel Earnshaw which states that charged particles cannot be electrostatically trapped. Solids are in a state of stable equilibrium: the (electrostatic) attractive forces must be balanced by the repulsive contact force to keep them from collapsing to a point. The particles in a solid are trapped in Earnshaw’s sense, so electrostatic forces can’t be the reason for it.

More likely, contact forces would be due to electrically polarized atoms or molecules. I wrestled with this for a long time, trying to make a model that works. The problem is that polarized particles should attract each other, except for unusual special cases. As two neutral atoms approach, the positive parts of one lean toward the negative parts of the other, minimizing the distance between the unlike charges and maximizing the distance between the like charges, making the total force attractive. It is possible for molecules to have permanant dipoles, but then they can simply rotate themselves to minimize distance between unlike charges, becoming attractive again. In biology, huge molecules can use repulsive polarization to their advantage, largely because they can root themselves relative to the object they want to repel. But this can’t be the reason so many simple substances solidify.

Magnetism always comes in dipoles, so the same arguments apply.

I’m fairly convinced that contact forces cannot be due to electromagnetism alone, though I don’t have a proof that rules out all possible mechanisms.

The puzzling thing is that I have heard “electromagnetism” (with no further explaination) cited as the origin of contact forces in several reputable physics popularizations, one of them being Brian Greene on Nova. In our case study, we will see that electromagnetism is involved in holding metals together, but it is not responsible for the repulsive contact force.

Explaination #2: The Pauli exclusion principle makes stuff hard

Closer to the heart of the matter is Pauli’s exclusion principle, which states, roughly, that “two identical particles cannot occupy the same state at the same time.” That sounds like the solution to our problem, given as an axiom! It is the second explaination that I have heard in popular presentations of physics, always discouragingly unspecific. We have reason to be wary— this is the effect which “becomes significant” when matter is crushed in white dwarf stars. Could it also be responsible for balsa wood?

Derivation of Pauli’s exclusion principle

To see more clearly what this principle states, we should return to our formulation of matter as a quantum field. Last time, I skirted past the fact that the quantum field is the square root of a probability distribution, with a phase. The function maps classical field configurations to an “amplitude,” , which is a complex number with a normalization property when it is squared. , or , is interpreted as probability density.

(This “amplitude” is a new word. It is not the amplitude of the classical field— sorry! To use my notation from an earlier section, is , not .)The phase of the complex number is lost when is squared, but it is relevant when two waves superimpose, because their relative phase determines whether they add constructively or subtract destructively.

Pauli’s exclusion principle applies to spinor fields, not fields of real numbers or vectors. Spinors are mathematical objects which are negated by rotations. Vectors, which I assume you’re more familiar with, are unaffected by rotation by . Imagine rotating a teacup 360 degrees— if it’s a vector, you get the same teacup back, but if it’s a spinor, you get minus a teacup (which, if squared, is a teacup squared in either case).

For concreteness, we can represent spinors with matrices. A vector living in 3-dimensional space is a 3-tuple that is rotated by applying this transformation:

while a spinor living in 3-dimensional space is a 2-tuple that is rotated by applying this transformation:

Note that when for spinors. (These are both special cases of rotation around the axis, and the above spinor representation applies only to spinors in the axis. A spinor only lives in one axis, with the two components interpreted as “up” and “down.”)

The key thing about spinors is that the amplitude of two spinor-particles is negated if they are exchanged. We can construct an example of this by placing one spinor-particle at with spin and the other at with spin , which is rotated by . The amplitude of a two-particle system is the product of the amplitudes of the individual particles, because it represents the combined probability of the pair. We can interchange the two particles by rotating everything by : this swaps the positions of the two particles, and the spins become and . This is the same scenario we started with, except that one of the spinors has acquired a minus sign, so the whole amplitude has a minus sign. The general proof (of the Spin-Statistics Theorem) follows these lines.

Now imagine two particles in the same position with the same spin. Swapping them yields exactly the same state, so the amplitude is unchanged. But it is also negated, therefore . There is no amplitude for a pair of spinor-particles in the same state, so there is no probability for it, either! This is Pauli’s exclusion principle. It is not an axiom: it is derived from the rotation properties of spinors.

Consequences of Pauli’s exclusion principle

So what happens if two spinor-particles merely get close to each other? If there are no other fields whose coupling is strong enough to drain their mass-energy away, the spinor-particles can’t disappear. The total probability density must be 1, so one of them must change its state, either by changing spin from to or by entering a higher-energy state.

Imagine a row of spinor-particles, all at rest, lined up along the axis. If we crush them in a vice-grip, we will encounter no resistance until the length of the row is halved, because the particles will happly overlap each other in the remaining space, selecting opposite spins. But as we twist the crank further, the particles will be forced to overlap each other with the same spin. The only way they can do that is by climbing into higher and higher states of kinetic energy. (High-energy states are standing waves with more wiggles.) At last, when the vice-grip is one particle wide, every state from the ground state up to state number will be filled with two of the particles. We provided the energy needed to push the particles into the upper states with the handle of the vice-grip, and it felt to us like a resisting force. Force is defined , so the Pauli exclusion principle really did exert a force against our hand as we turned the crank (sometimes called the exchange force). But the Pauli exclusion principle isn’t one of the four fundamental forces! Gravity, Electromagnetism, and the Strong and Weak Nuclear interactions are put into quantum field theory by hand; the Pauli exclusion principle is a derived consequence of the rotation of spinors, independent of interactions! How can a principle exert a force?

This is not the only force-which-is-not-a-fundamental interaction. The random motions of air molecules inside a balloon exert a force on the balloon’s surface, even though they are freely-streaming particles, without interactions (ideally). Balloons and vice-grips are made of charged particles, so we might expect electromagnetism to be a distant cause, but it doesn’t need to be. Imagine, instead of a vice-grip, that the particles are enclosed in a small, toroidial universe with finite volume. If the volume shrinks for some reason, the particles will resist, even if there are no interactions in the Schrödinger equation at all.

Thus, the Pauli exclusion principle can provide a resisting force that acts qualitatively like the contact force from freshman physics. But is it big enough for balsa wood?

Case study: what makes metal hard?

I decided one day that I had gone long enough without knowing why things are hard, so I went to the library to find out. The answer lies in “that other branch of physics,” namely, everything but particle physics/cosmology. The majority of physicists start with protons, neutrons, and electrons and derive from these everything we encounter in our macroscopic lives, including tangibility. I expected “Why things are hard” to be a chapter in a standard Solid State Physics textbook. At least, I expected it to come in a “Basic Properties” section, before applications to transistors.

I never found a general answer to my question, and I suspect that the exact answer might differ in the details for every material. The Pauli exclusion principle is probably involved at some level for all of them, though it could be obfusgated by other effects. From my reading, I was able to derive the hardness of one simple, but undeniably hard, material: metal. I need to apply some approximations, and only carry out my calculation to the nearest order of magnitude, but a rough agreement with the measured hardness of metal gives me some confidence that this is the correct explaination.

Many metals can be described by the following picture: a lattice of atomic nuclei, all the same element, surrounded by several filled shells of electrons. Electrons are spinor-particles, so they obey Pauli’s exclusion principle: only one can fill a given orbital+spin energy level. The innermost shell accepts exactly 2 electrons (purely determined by spin), the next takes 8, the next 18, etc. Beyond the filled shells, the atoms may require 3 to 6 more electrons to be neutral, but this is not enough to fill a shell. These “valence” electrons are loosely bound and roam from nucleus to nucleus. The nuclei attract each other electrostatically, because although most of their charge is screened by the filled shells and the valence electrons, not all of it is. This is the attractive force that keeps metal from flying apart. The repulsive force that balances it, and resists the force of my swinging fist, derives from the dependence of the valence electrons’ energy on the size of the metal object, highly amplified by the Pauli exclusion principle.

It’s worth noting that elements with no valence electrons are all formless gasses.To derive the hardness of metal, we must consider how the internal energy responds to crushing, just like the example of the vice-grip on a line of particles. The quantity that measures three-dimensional crushing is called the bulk modulus,

also known as , inverse compressibility. We need to calculate the energy of the metal as a function of volume. Normal humans cannot crush metal to such an extent that the nuclei or their inner shells are threatened, so the valence electrons’ energy is the only component that matters.

The potential energy that the valence electrons feel is a regular lattice of wells, the Coulumb potential due to each nucleus. But do we need all of this detail? Back-of-the-envelope calculations of electron wavelengths yield 10 nm at the smallest, which are tens to hundreds of times the interatomic spacing. The valence electrons will therefore see a smoothed potential that looks remarkably like the infinite square well at the beginning of most Quantum Mechanics books. For the most part, the nuclei just keep the valence electrons from wandering away.

The solution to the Schrödinger equation for an infinite square-well is sinusodial with zero amplitude along the edges. An electron in the ground state is one big wave that fills the entire metal conductor— if the metal object is, say, a skyscraper, that’s an enormous electron! Or consider the electron that sits in the metallic hydrogen core of Jupiter. Fundamental particles really are waves— their sizes are not intrinsic.

The three-dimensional infinite square-well problem is solved in detail on the “Fermi Sea” Wikipedia page, in exactly our context: valence electrons in metal. The energy of a given state is

where we will ignore dimensionless constants of order one. If the electrons did not obey Pauli’s exclusion principle, the total energy of electrons would be

because they would all be in the ground state (barring excitation due to thermal noise). Since electrons are spinor-particles, the exclusion principle applies and the electrons fill one state each, from the ground state up to the state. (Thermal excitation only blurs the top few levels.) We can integrate for the total energy by representing as the first quadrant of a sphere— this is all worked out in detail on the above-mentioned Wikipedia page. The total energy is actually

The 5/3 power, applied to typical numbers of valence electrons (), makes a difference of a factor of in total energy. Metal would be much squishier (and denser) without it.

Now we can calculate the bulk modulus.

Armed with this prediction, I confronted the Periodic Table and was immediately overwhelmed by the qualitatively different kinds of metals. The transition metals, including some of the most familiar such as iron, gold, and tin, don’t fit the simple picture I presented at the beginning of this derivation because they have two unfilled shells, and the other shell can be fairly large (holds 18). I don’t know how many of these electrons to call “valence.” The non-transition metals are divided into semi-metalic “metaloids” and post-transition “poor metals.” I found the best agreement in the Boron and Carbon families, an indication that unaccounted-for systematic effects are lurking among the data, preferring certain electron configurations over others. Here are the data for the Boron and Carbon families, with an asterix marking the poor metals.

Element	Nuclei/V	Valence Electrons	Prediction	Measured	Ratio (meas/pred)
Boron	130	3	21000	32000	1.52
Silicon	48	4	6400	10000	1.56
Aluminum*	59	3	5600	7600	1.36
Thallium*	34	3	2200	4300	1.95
Tin*	36	4	4000	5800	1.45
Lead*	32	4	3300	4600	1.40

The fact that the ratio is not 1.0 is no surprise: we ignored constants of order unity. What is interesting is (a) the prediction and the measurement are the same order of magnitude, indicating that this mechanism really can explain most of the effect, and (b) there’s a correlation between the measurement and the prediction: the values vary by a factor of 7 from Thallium to Boron, but their ratios vary by less than a factor of 1.5.

So what happened to this being the effect which “becomes significant” in white dwarfs? It’s certainly significant in ordinary matter! It just isn’t a factor in normal stars, because stars are so hot that the kinetic energy of their electrons aren’t limited to the minimum-energy states. When stars cool into white dwarfs, they become more like metal.

Because I’m honest, here are the rest of the non-transition metals.

Element	Nuclei/V	Valence Electrons	Prediction	Measured	Ratio (meas/pred)
Arsenic	45	5	8300	2200	0.27
Antimony	32	5	4700	4200	0.89
Tellurium	28	6	5100	6500	1.27
Bismuth*	28	5	3800	3100	0.82

(No data for Gallium.) Including these, we can see variations of a factor of 7 in the ratio. Perhaps there isn’t anything special about the Boron and Carbon families; it could have more to do with metaloids versus poor metals. There are way too few elements to do a statistical analysis— to find out what’s really going on here, I would need to learn Chemistry.

Now it’s time to back up: have we answered our question? We have just explained (roughly, at least) how metal resists being crushed, assuming that we have the power to move the edges of the square-well potential. But why would pushing a metal face move the edge of the potential? That potential is set by the nuclei— couldn’t I push my finger ghost-like into the metal, leaving the nuclei, and therefore the square-well potential, fixed? (The metal’s nuclei and the nuclei in my finger are both very small; they won’t collide.) Here’s how I think about it: suppose we push a block of metal with a metal finger, all the same substance. The block plus the finger can be considered one object, and if they could interpenetrate, that would be the same as saying that the block-plus-finger object is shrinking, and the bulk modulus would be sure to prevent that! My finger is not made of metal, but it has the same effect. This still amazes me, because the outermost electrons in my finger are not free-roaming valence electrons; they form a wide variety of configurations, and yet it all still works— solids don’t interpenetrate.

Conclusion: which explaination was right?

As we have seen, Pauli’s exclusion principle has more to do with why things are hard than electromagnetism. But its role is not particularly simple, and it isn’t enough to say that “two particles can’t sit in the same place at the same time,” because they can and they do. The electrons (or really, electron-waves) in a metal all fill the whole structure, but at strictly different energy levels. And even this isn’t the direct cause of the contact force but an amplifier of it: the direct cause is the fact that energies of all the electron states scale with the size of the box they are forced to live in. One could point out that it is electromagnetism that keeps them in the box, but if that means that electromagnetism wins, it wins by an enormous technicality.

The disturbing thing about this picture is that it is not general. The exact reason that one material, like metal, is hard is not necessarily the reason that another material, like my finger, is somewhat hard. This might account for the vast diversity in pliability and texture in nature, but it makes me wonder if there’s a more general way of looking at it that I’m missing, or if such a complicated rule has an exception. Could some highly engineered substance be made intangible, like recent materials designed for invisibility?

That’s a tricky problem, because the same force that resists outside pressure is the force that keeps metal from collapsing. To make an intangible solid, one must find a way to balance the attractive force of the substance’s internal particles without being influenced by penetration from outside. We could do this if there were new fields in the Schrödinger equation which are strongly coupled to each other but weakly coupled to our fields. That would be a ghost universe, with planets we could orbit but never land on— in fact, we could orbit inside them!

This line of reasoning is unfair, because you can’t make new technology by rewriting the laws of physics. There is some value to thinking about it, though, because dark matter is a field with very little coupling to our own matter. This has been firmly established with gravitational observations (we could definitely orbit it), but not directly, with physical detectors (emphasizing the point that it couples very weakly). We don’t know if there are new strong interactions felt only by dark matter, which would be necessary to make dark planets, and all indications so far suggest that the vast majority of dark matter is softer than gas. But suppose that there are different kinds of stable dark matter particles, and only a small fraction of them interact strongly with themselves. This could be enough to make a planet here and there…

We’ll find out when the dark matter people living in the center of the earth send up a satellite to discover, “What’s beyond the Mantle?”