We could give better help if you told us why you are stuck. We don't know if it is the formulas, if you know the formulas but can't put them together or if you have done that but can't do the differental.
I'll get you started.
Draw a cross section of the log, a circle with an inscribed rectangle. The base of the rectangle is w, the height is h. Connect two corners of the rectangle with a line.
11^2 = w^2 + h^2
You are told that
S = w^2*h
Solve the first equation for w^2
Substitute that for w in the second.
Find dS/dh
Set it = 0
Solve for h
Substitute that in the first equation and solve for w
Hint h is a little more than 6 and w a little less than 9.
If you need more show your work as far as you got.
