I'm sorry, but I don't understand what you mean by this last line...?
Instead, try using the Quadratic Formula and the provided information. Plugging into the Formula, we get:
. . . . .\(\displaystyle x\, =\, \dfrac{-(-5)\, \pm\, \sqrt{\strut \,(-5)^2\, -\, 4(k\, -\,1)(3k\, +\, 4)\,}}{2(k\, -\, 1)}\)
. . . . .\(\displaystyle x\, =\, \dfrac{5\, \pm\, \sqrt{\strut \,25\, -\, 4(3k^2\, +\, k\, -\, 4)\,}}{2k\, -\, 2}\)
. . . . .\(\displaystyle x\, =\, \dfrac{5\, \pm\, \sqrt{\strut \, 41\, -\, 4k\, -\, 12k^2\,}}{2k\, -\, 2}\)
We are given that these two roots are reciprocals, so:
. . . . .\(\displaystyle \left(\dfrac{5\, -\, \sqrt{\strut \, 41\, -\, 4k\, -\, 12k^2\,}}{2k\, -\, 2}\right)\, \left(\dfrac{2k\, -\,2}{5\, +\, \sqrt{\strut \, 41\, -\, 4k\, -\, 12k^2\,}}\right)\, =\, 1\)
. . . . .\(\displaystyle 5\, -\, \sqrt{\strut \, 41\, -\, 4k\, -\, 12k^2\,}\, =\, 5\, +\, \sqrt{\strut \, 41\, -\, 4k\, -\, 12k^2\,}\)
For these to be equal, obviously the values of the square roots must be zero. For this to be true, the innards of the radicals must be zero. This gives us:
. . . . .\(\displaystyle 12k^2\, +\, 4k\, -\, 41\, =\, 0\)
. . . . .\(\displaystyle k\, =\, \dfrac{-(4)\, \pm\, \sqrt{\strut \, (4)^2\, -\, 4(12)(-41)\,}}{2(12)}\)
. . . . .\(\displaystyle k\, =\, \dfrac{-4\, \pm\, \sqrt{\strut \, 16\, +\, 1968\,}}{24}\)
. . . . .\(\displaystyle k\, =\, \dfrac{-4\, \pm\, \sqrt{\strut \, 1984\,}}{24}\, =\, \dfrac{-4\, \pm\, 8\, \sqrt{\strut \, 31\,}}{24}\, =\, \dfrac{-1\, \pm\, 2\, \sqrt{\strut\, 31\,}}{6}\)
Check my work, and check that these values for k actually do solve the equation.