Transformation of Normals

It is commonly stated in the computer graphics literature that normals are transformed using the transpose of the inverse, often without explanation.

\[\V{n}'=(\M{M}^{-1})^T\cdot\V{n}\]

This bears some discussion. Consider a plane equation.

\[a\,x + b\,y + c\,z + d = 0\]

This equation defines the set of all points $(x, y, z)$ that fall on the plane $(a, b, c, d)$. This plane faces along the vector $(a, b, c)$ at a distance $d$ to the origin.

For example, if the top of your desk is 75 cm off the floor then $y=0.75$ m for all points on the surface of your desk. The normal vector of the top of your desk faces up, $(0, 1, 0)$, so

\[0\,x + 1\,y + 0\,z - 0.75 = 0\]

While we often think of the plane equation as the dot product of two vectors, it's enlightening to phrase it as a matrix product.

\[\begin{bmatrix} a & b & c & d\end{bmatrix}\cdot \begin{bmatrix} x \\ y \\ z \\ 1\end{bmatrix} = 0\]

Here, a position is represented as a column vector, while a plane is represented as a row vector. Take this to be the case universally.

Now, assume the plane equation holds for a vector $\V{v}=[x, y, z, 1]^T$ and a plane $\V{p}=[a, b, c, d]$. Position $\V{v}$ falls within plane $\V{p}$.

\[\V{p}\cdot\V{v}=0\]

Let's transform $\V{v}$.

\[\V{v}'=\M{M}\cdot\V{v}\]

The equation no longer holds for that same $\V{p}$.

\[\V{p}\cdot\V{v}'=\V{p}\cdot\M{M}\cdot\V{v}\neq 0\]

But we can fix it by introducing the transform inverse, which cancels out the transformation of the position.

\[\V{p}\cdot\M{M}^{-1}\cdot\M{M}\cdot\V{v}=0\]

From this we can see that if we transform of plane $\V{p}$ like so,

\[\V{p}'=\V{p}\cdot\M{M}^{-1}\]

then the plane equation holds for the transformed position and transformed plane.

\[\V{p}'\cdot\V{v}'=0\]

This, therefore, is how planes should transform: multiplied on the left of the inverse matrix.

But the incumbent mechanisms of computer graphics don't do transformation of row vectors on the left, they do transformation of column vectors on the right. We accommodate this and change a plane transformation into a position transformation simply by taking the transpose of both sides of the equation.

\[(\V{p}')^T=(\V{p}\cdot\M{M}^{-1})^T\] \[(\V{p}')^T=(\M{M}^{-1})^T\cdot(\V{p})^T\]

In this way, a plane can be treated like a position.

A normal vector, being an orientation, behaves as a plane rather than a position, with the fourth component $d=0$. As such, it's transformed by the transpose of the inverse.

\[\V{n}'=(\M{M}^{-1})^T\cdot\V{n}\]