Here is a method known as RREF (reduced row echelon form). The idea is elimination.
Start with your system only write it without the variables:
\(\displaystyle \begin{bmatrix}-5&3&2&1\\1&1&1&7\\2&-1&1&7\end{bmatrix}\)
The idea is to get it into the form \(\displaystyle \begin{bmatrix}1&0&0&a\\0&1&0&b\\0&0&1&c\end{bmatrix}\)
Where a, b, and c are the solutions to your system>
Now, let's multiply row 2 by -2 and add it to row 3, then we get:
\(\displaystyle \begin{bmatrix}-5&3&2&1\\1&1&1&7\\0&-3&-1&-7\end{bmatrix}\)
Now, multiply row 1 by 1/5 and add to row 2, then we get:
\(\displaystyle \begin{bmatrix}-5&3&2&1\\0&8/5&7/5&36/5\\0&-3&-1&-7\end{bmatrix}\)
Now, multiply row 2 by 5/8, then we get:
\(\displaystyle \begin{bmatrix}-5&3&2&1\\0&1&7/8&9/2\\0&-3&-1&-7\end{bmatrix}\)
Now, multiply row 2 by -3 and add to row 3, then we get :
\(\displaystyle \begin{bmatrix}-5&3&2&1\\0&1&7/8&9/2\\0&0&13/8&13/2\end{bmatrix}\)
Now, multiply row 3 by 8/13:
\(\displaystyle \begin{bmatrix}-5&3&2&1\\0&1&7/8&9/2\\0&0&1&4\end{bmatrix}\)....
See there?. we have z=4 in the last row.
Now, multiply row 3 by -7/8 and add to row 2, then we get:
\(\displaystyle \begin{bmatrix}-5&3&2&1\\0&1&0&1\\0&0&1&4\end{bmatrix}\)
Multiply row 2 by -3 and add to row 1, then we get:
\(\displaystyle \begin{bmatrix}-5&0&2&-2\\0&1&0&1\\0&0&1&4\end{bmatrix}\)
Multiply row 1 by -1/5, then we get:
\(\displaystyle \begin{bmatrix}1&0&-2/5&2/5\\0&1&0&1\\0&0&1&4\end{bmatrix}\)
Now, the last one finally. multiply row 3 by 2/5 and add to row 1, then we get:
\(\displaystyle \begin{bmatrix}1&0&0&2\\0&1&0&1\\0&0&1&4\end{bmatrix}\)
So, the solutions are \(\displaystyle \fbox{x=2, \;\ y=1, \;\ z=4}\)
There is no set way to do this. It is just a matter of whittling away at it and trying not to replace something you already done.
I know it seems tedious and long. That's because it is.
This method takes practice , like everything else.