I assume this is what the figure looks like. You should've included the figure in your post. Anyway, here's my attempt. I hope it's not an effort in futility.
Look at the triangles. They're all over the place. Let's use
\(\displaystyle \L\\x^{2}+z^{2}=y^{2}\)
\(\displaystyle \L\\a^{2}+(20-x)^{2}=x^{2}\rightarrow\ x^{2}-40x+a^{2}+400=x^{2}\)
\(\displaystyle \L\\a^{2}=40x-400=40(x-10)\rightarrow\ a=\sqrt{40(x-10)}\)
Look at the triangle below the fold:
\(\displaystyle \L\\z^{2}=b^{2}+20^{2}\rightarrow\ z=\sqrt{b^{2}+400}\)[1]
\(\displaystyle \L\\z=a+b\rightarrow\ b=z-a=z-\sqrt{40(x-10)}\)
Plug this into [1] and we get:
\(\displaystyle \L\\z=\sqrt{(z-\sqrt{40(x-10)})^{2}+400}\)
Square both sides:
\(\displaystyle \L\\z^{2}=(z-\sqrt{40(x-10)})^{2}+400\)
Here's a good place to practice your algebra
Skipping anymore typing, when this is expanded out and solved for z, we get:
\(\displaystyle \L\\z=\frac{\sqrt{10}x}{\sqrt{x-10}}\)
Sub this into the original equation \(\displaystyle x^{2}+z^{2}=y^{2}\)
I will leave the differentiation and what-not up to you.
After you sub, it'll be entirely in terms of x. You should have
y=something. Differentiate,
set to 0 and solve for x.