On Wikipedia, the Hessian matrix is defined as,
$ {\displaystyle \mathbf {H} ={\begin{bmatrix}{\dfrac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\dfrac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\dfrac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\[2.2ex]{\dfrac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\dfrac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\dfrac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\[2.2ex]\vdots &\vdots &\ddots &\vdots \\[2.2ex]{\dfrac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\dfrac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\dfrac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}.}$
However, in the textbook "Optimization" by Chong and Zak, it is written,
This is different! Or is it?
Is there any problem if I exchanged the order of the partial derivative for all the off-diagonal terms?
Which definition is correct?
$\endgroup$02 Answers
$\begingroup$If $f$ is twice continuously differentiable, no, it makes no difference.
$\endgroup$7$\begingroup$As argued above, it does not matter all that much in practice. However, the definition that I remember, and given in Wikipedia as well, is $\mathbf{H}(f(x)) = \mathbf{J(\nabla} f(x))$. If you use this definition, the Wikipedia ordering is wrong (but it's a moot point).
$\endgroup$
