Because they wanted to know what the function f is for any argument, so they gave a name to the argument they know, x^2 + 1, to see what would happen. (A lot of mathematical thinking involves trying something without being sure it will work -- just taking one step and hoping you've see better from the new location.)
I wouldn't describe it that way. They substituted y-2 for y in the new definition of f, because the goal was to find f(x^2 - 1), and x^2 - 1 is (x^2 + 1) - 2.
They could have done the same thing just by replacing y with x^2 - 1 literally, saying f(x^2 - 1) = (y - 1)(y + 3) where y = x^1 - 1, namely ((x^2 - 1) - 1)((x^2 - 1) + 3) = (x^2 - 2)(x^2 + 2).
Again, you are describing this in a way that may be confusing yourself. The expression f(y-2) just says, what comes out of the "machine" when you put y-2 into it.
Olympiad type questions are probably all about turning things inside out to challenge you with non-routine thinking!