25th July 2021, 1 min read

Diagonal of Squared Jacobian

Original post is here eklausmeier.goip.de/blog/2021/07-25-diagonal-of-squared-jacobian.

Assume $f$ is a $m$ -vector valued function in $n$ -variables, i.e., $f : U \subseteq R^{n} \to R^{m}$ . The Jacobian is given by

J = (\begin{matrix} f_{x_{1}}^{(1)} & \dots & f_{x_{n}}^{(1)} \\ ⋮ & ⋱ & ⋮ \\ f_{x_{1}}^{(m)} & \dots & f_{x_{n}}^{(m)} \end{matrix})

We use

f_{x_{i}}^{(j)} = \frac{\partial f^{(j)}}{\partial x_{i}} .

The diagonal elements $D_{i i}$ of $D := J^{T} J \in R^{m \times m}$ are

D_{i i} = {(f_{x_{i}}^{(1)})}^{2} + {(f_{x_{i}}^{(2)})}^{2} + \dots + {(f_{x_{i}}^{(m)})}^{2} .

One diagonal element can be computed in $O (m)$ multiplications and $O (m)$ additions. All $m$ diagonals can therefore be computed in $O (m^{2})$ multiplications and $O (m^{2})$ additions. That's the same effort as one matrix-vector multiplication.

Open questions:

How to fetch the diagonal elements in case of a matrix-free setting?
In case of projections with $(1, 0, \dots, 0)$ , $(0, 1, \dots, 0)$ , etc., do we incur too much effort?