Your error comes when you set the polynomial to zero. All that tells you is that the travel agent makes zero money when x = 24 (i.e. when there's 40 people going). That's not very helpful at all. You want to know when that polynomial has its maximum value. Being that this is posted in "Intermediate/Advanced Algebra," I'll assume you don't know about the standard Calculus methods of setting the derivative to zero, etc.
There are a couple of different ways you might go about it. One is just simply to make a table. This works well in this case because x can only take on discreet integer values, but it may not for other problems where x can be any real number. If x = 0, 16 people go on the trip, so the trip costs (2400 - 0)(16 + 0) = 38400. If x = 1, 17 people go on the trip, so the trip costs (2400 - 100)(16 + 1) = 2300 * 17 = 39100. And so on. When does this produce the maximum value, and why?
Another option is to graph the polynomial. This also works well here, because the function you're trying to maximize is a parabola, but it may not work well for other more complex functions. What have you learned previously about the graphs of parabolas? When do they have a maximum, and when do they have a minimum? Where are these maxima/minima located on the graph? What, then, does that make the maximum here? From that, what is the value of x that maximizes the function? Does that answer agree with your tabular method? Do these agree with the given answer from your instructor?