Note that the curve of intersection of the plane and cone is a closed set.
Since the distance from the origin is a continuous function, we know there is a point on the curve that gives a max and min.
We want the extrema of \(\displaystyle \L\\D(x,y,z)=x^{2}+y^{2}+z^{2}\) subject to the constraints:
\(\displaystyle \L\\4x^{2}+4y^{2}-z^{2}=0, \;\ and \;\ 2y+4z=5\)
D is the square of the function that gives the distance of the point from the origin. It's easier to work with than radicals.
The extrema occur at points (x,y,z) that are solutions of:
\(\displaystyle \L\\\begin{Bmatrix}{\nabla{D}\cdot({\nabla}(4x^{2}+4y^{2}-z^{2})X{\nabla}(2y+4z))=0\\4x^{2}+4y^{2}-z^{2}=0\\2y+4z=5\end{Bmatrix}\)
We have:
\(\displaystyle \L\\{\nabla}D=2xi+2yj+2zk\)
\(\displaystyle \L\\{\nabla}(4x^{2}+4y^{2}-z^{2})=8xi+8xj-2zk\)
\(\displaystyle \L\\{\nabla}(2y+4z)=2j+4k\)
\(\displaystyle \L\\{\nabla}(4x^{2}+4y^{2}-z^{2})X{\nabla}(0x+2y+4z)=\begin{vmatrix}i&j&k\\8x&8y&-2z\\0&2&4\end{Bmatrix}=\begin{vmatrix}8y&-2z\\2&4\end{vmatrix}i-\begin{vmatrix}8x&-2z\\0&4\end{vmatrix}j+\begin{vmatrix}8x&8y\\2&4\end{vmatrix}k\)
=\(\displaystyle \L\\(32y+4z)i-(32x)j+(16x)k\)
Then we have:
\(\displaystyle \L\\(2x)(32y+4z)-(2y)(32x)+(2z)(16x)=0\)
\(\displaystyle \L\\40xz=0\)
This means x or z equals 0.
If z=0, this implies that x=y=0, but (x,y,z) does not satisfy \(\displaystyle 2y+4z=5\). So, there's no solution with z=0.
If x=0, this implies z=2y or z=-2y.
Now, can you finish?. If not, write back.
Are you sure there isn't an x term in 2y+4z=5?.
I may have a mistake in my calculations. Check it out. Easy to do
