Hi,
I'm trying to find the least squares transformation A that minimizes (AX-Y)'Q'Q(AX-Y), where X,Y,and Q are given, and Q is positive definite.
The normal least squares problem is to find A that minimizes ||AX-Y||^2 = (AX-Y)'(AX-Y), where X is a dxN matrix, Y is a dx1 matrix, N is the number of training examples, and d is the dimension of the data. The closed form solution is A = Y*pinv(X).
To solve my problem, these are the steps I have taken so far:
1) Multiply in Q
(X'A'Q'-Y'Q')(QAX-QY) = ||QAX-QY||^2
2) It now looks like a simple least squares problem, however, A is multiplied to the left by Q and right by X. If I let M = QA, and solve ||MX-QY||^2, then M = QY*pinv(X) = QA. Then A is just Y*pinv(X), which is just the least squares solution. This however, does not seem correct to me.
The reason why I feel this is incorrect is because, if X is right multiplied instead of left multiplied by A, i.e. (XA-Y)'Q'Q(XA-Y), then we are minimizing ||QXA-QY||^2. The classical solution for this problem is A = pinv(QX)*QY.
Please let me know what I am missing.
Thanks.
I'm trying to find the least squares transformation A that minimizes (AX-Y)'Q'Q(AX-Y), where X,Y,and Q are given, and Q is positive definite.
The normal least squares problem is to find A that minimizes ||AX-Y||^2 = (AX-Y)'(AX-Y), where X is a dxN matrix, Y is a dx1 matrix, N is the number of training examples, and d is the dimension of the data. The closed form solution is A = Y*pinv(X).
To solve my problem, these are the steps I have taken so far:
1) Multiply in Q
(X'A'Q'-Y'Q')(QAX-QY) = ||QAX-QY||^2
2) It now looks like a simple least squares problem, however, A is multiplied to the left by Q and right by X. If I let M = QA, and solve ||MX-QY||^2, then M = QY*pinv(X) = QA. Then A is just Y*pinv(X), which is just the least squares solution. This however, does not seem correct to me.
The reason why I feel this is incorrect is because, if X is right multiplied instead of left multiplied by A, i.e. (XA-Y)'Q'Q(XA-Y), then we are minimizing ||QXA-QY||^2. The classical solution for this problem is A = pinv(QX)*QY.
Please let me know what I am missing.
Thanks.