I have solved using \(\displaystyle \bf{A.M \geq G.M}\) , Then \(\displaystyle \displaystyle x+y \geq \sqrt{xy}\leftrightarrow xy \leq \frac{1}{4}\)
Now Let \(\displaystyle z = x^4y+xy^4 = xy.\left(x+y \right).\left(x^2+y^2 -xy \right)\)
\(\displaystyle \displaystyle z = \left(xy\right).\left(1-3xy\right) = -3\left\{\left(xy\right)^2-\frac{1}{3}\left(xy\right)+\frac{1}{6^2}-\frac{1}{6^2}\right\}\)
So \(\displaystyle \displaystyle z = \frac{1}{12} - 3\left(xy-\frac{1}{6}\right)^2\)
So \(\displaystyle \displaystyle z_{Max} = \frac{1}{12}-\frac{1}{48} = \frac{3}{48} = \frac{1}{16}\) at \(\displaystyle \displaystyle xy = \frac{1}{4}\)