Reducing hyperplane from affine to linear via an additional dimension?

[Metronome]

I keep coming across this idea in the context of linear classifiers that an affine hyperplane can be made linear by incorporating an additional dimension.

1) This feels like overkill to me. Why not just assume an appropriate translation of the hyperplane and all data points such that the hyperplane passes through the origin within its original dimensions?

2) Does anyone have an animation to illustrate the geometry of moving the hyperplane and data points from 2 to 3 dimensions in this way? I can't quite visualize it from the 2D drawing on the chalkboard. The closest thing I found was this video, but I think it's something significantly different.

 

[blamocur]

1) This feels like overkill to me. Why not just assume an appropriate translation of the hyperplane and all data points such that the hyperplane passes through the origin within its original dimensions?

Not sure what your suggestion means, but one of the advantages of representing affine transforms as special case of linear transforms`is reducing products of affine transforms to multiplications of the corresponding matrices.

2) Does anyone have an animation

I don't, but I find it illustrative to consider linear transforms, say in [imath]\mathbb R^3[/imath] which keep [imath]z[/imath] unchanged. Applying them to vectors [imath](x,y,1)[/imath] is the same as applying an affine transform to [imath](x,y)[/imath].

 

[Metronome]

Not sure what your suggestion means, but one of the advantages of representing affine transforms as special case of linear transforms`is reducing products of affine transforms to multiplications of the corresponding matrices.

If the actual data points and (non-unique) hyperplane have the black form, can't we look for a solution of the red form, effectively just reassigning the origin of the coordinate system to some point on the candidate hyperplane? At the end, this would be undone to attain the black hyperplane solution. This just seems conceptually cleaner than introducing a dummy dimension.

 

[blamocur]

If the actual data points and (non-unique) hyperplane have the black form, can't we look for a solution of the red form, effectively just reassigning the origin of the coordinate system to some point on the candidate hyperplane? At the end, this would be undone to attain the black hyperplane solution. This just seems conceptually cleaner than introducing a dummy dimension.

I don't get it. Which solutions? What is "hyperplane solution" ?

You can use whatever approach works for you. I simply tried to explain why one of the approaches is popular. For example, having one familiar object (i.e. a matrix with a "dummy dimension") makes writing computer graphics software somewhat simpler and less error-prone.

 

[Metronome]

I don't get it. Which solutions? What is "hyperplane solution" ?

As I understand, the solution to the binary classification problem is any hyperplane such that all points of one type lie on one side and all points of the other type lie on the other side.

 

[blamocur]

As I understand, the solution to the binary classification problem is any hyperplane such that all points of one type lie on one side and all points of the other type lie on the other side.

What does it have to do with your original post?

 

[Metronome]

What does it have to do with your original post?

I'm not quite sure what else needs to be clarified. The problem is finding a hyperplane (which is naturally an affine object) such that all data points of one type lie on one side and all data points of the other type lie on the other side. The solution technique involves treating the space of possible solutions as linear instead of affine (that is, hyperplanes which pass through the origin). I appreciate why linearity may be nicer than affinity. What I do not appreciate is why increasing the dimension is the best way to force linearity, when it seems like the possible solutions and data points could simply be shifted onto the origin without increasing the dimension.

 

[blamocur]

I'm not quite sure what else needs to be clarified. The problem is finding a hyperplane (which is naturally an affine object) such that all data points of one type lie on one side and all data points of the other type lie on the other side. The solution technique involves treating the space of possible solutions as linear instead of affine (that is, hyperplanes which pass through the origin). I appreciate why linearity may be nicer than affinity. What I do not appreciate is why increasing the dimension is the best way to force linearity, when it seems like the possible solutions and data points could simply be shifted onto the origin without increasing the dimension.

You are right, my fault. I concentrated on transforms and forgot about the first sentence -- sorry.

I see this whole issue as a matter of style/taste. You can express your class of [imath]\mathbf x[/imath]'s as [imath]\mathbf A \cdot\mathbf x + \mathbf b < 0[/imath], or you can add one dimension to combine [imath]\mathbf A[/imath] and [imath]\mathbf b[/imath] into, say, [imath]\mathbf C[/imath] and express the same class of [imath]\mathbf x[/imath]'s as [imath]\mathbf C\cdot (\mathbf x, 1) < 0[/imath] , where[imath](\mathbf x,1)[/imath] is a vector containing [imath]\mathbf x[/imath] and one extra dimension set to 1. The resulting set of [imath]\mathbf x[/imath]'s should be the same.