Hello,
The problem is
sin(xy) = x^2 - y
Your implicit differentiation
cos(xy)(xy'+y) = 2x - y'
is correct.
Let's see where we can go from here.
Multiplying out the left side yields
xy'cos(xy)+ycos(xy)=2x-y'
Now, let's move ycos(xy) to the right side and y' to the left side, so that we can have all terms with y' one one side, and everything else on the other.
xy'cos(xy)+y'=2x-ycos(xy)
Now, we can factor out y' on the right side, which yields
(xcos(xy)+1)y'=2x-ycos(xy)
Now, we can divide:
y'=(2x-ycos(xy))/(xcos(xy)+1)
there's the first derivative, simplified.
y'' might get a bit messy here, but let's try it.....
Using the quotient rule for the right side, we get:
y''=[(xcos(xy)+1)(2-(-ysin(xy)(xy'+y)+y'cos(xy))-(2x-ycos(xy))(-xsin(xy)(xy'+y)+cos(xy))]/(xcos(xy)+1)^2
Yep, pretty messy. There's got to be an easier way. Let me think for a bit, and see if you can work with this in the meantime.