Supporting vectors was first proposed by vladimir vapnik of bell laboratories and his colleagues in 1992. However, many do not know that the basic knowledge supporting vectors was developed in his doctoral thesis at moscow university in the 1960s. For decades, svm has been favoured by many because it uses fewer computing resources while allowing data scientists to obtain significant accuracy. Not to mention that it addresses both classification and return。
Basic concepts
Supporting vectors solve linear and non-linear problems and work well on many practical operational issues. The principle of supporting vectors is straightforward. The learning model draws a line that divides data points into several categories. In a dual question, the decision-making boundary uses the broadest street approach, maximizing the distance from each category to the nearest data point。
In a vector calculus, point accumulation measures the " quantity " of one vector in another vector and tells you the size of the force in the direction of the shift or in the direction of another vector。
For example, we have an unknown vector u and a legal vector that is vertically at the decision-making boundary. W w. U's point volume is the size of the force you pass in the vectorw direction. In this regard, if the unknown vector u is on the right side of the boundary, the constant b can be used as described below。
A sample located above the boundary where a positive sample is classified (+1) or below the boundary where a negative sample is classified (-1) may be indicated accordingly。
Decision-making rules
The determination of the decision boundary shall be done by mapping the positive and negative boundary in such a way as to maximize the width of the closest samples in each group and placing them on each group's boundary。
This rule will be the constraint to find the maximum boundary width. Assuming y-to-positive samples of +1 and negative samples of 1 x, both equations above can be expressed on either side of the equation by multiplying y by y. They are also called support vectors。
Decision-making rules - maximum width
Assuming we have a vector x+ on the positive boundary line and a vector x- on the negative boundary line. X + negativex - indicates the force of direction from negative to positive x +. If we do this in this direction with a unit vector that is vertically at the decision-making boundary, it will become the width between negative and positive borders. Note that w is the law-line vector, w-w-w-w-w-w-w-w-size。
We basically maximize this width to distinguish negative and positive data points. The following could be simplified. For mathematical convenience, the last form divides the size of w squared by two。
Find maximum bound width
The lagrand japanese equation can be used to solve the problem of restraint optimization. The maximum value of the target function is reduced if the unit is bound to change. In the case of binding, this equation is usually used to find the maximum or minimum value of the target function。
As we mentioned earlier, svm uses the widest street method to find the maximum width between the positive and negative borders. The question can be described using the target function and the lagrand japanese equation as defined below。
In summary, given that the sample is a support vector on the dividing line, the lagrand day minimizes the target function (which ultimately maximizes the width between the positive and negative boundary)。
The following can be simplified when the guidance on w and b is found from the above formula. Since y i and y j are labels or respond to variables, the equation can simply be minimized by maximizing the points of x i and x j. In other words, the maximization of width depends entirely on the sum of points of support vectors when drawing boundary lines。
In addition, it is determined whether the unknown vector u is on the right side of the decision-making boundary based on the accumulation of supporting vector x and u。
Kernel techniques
On-line issues, svm can easily draw decision boundaries to divide samples into multiple categories. However, if it is not possible to separate the data points by linear slices, the data points can be converted prior to the mapping of the decision boundary, known as “kernel techniques”。
Non-linear svm has been transformed into linear svm problems using kernel techniques. By using a special function called the kernel, which essentially maps the problem from input space to the new high space (known as the feature space) (x). A linear model is then used to separate the data points in the characteristic space. Linear models in feature space correspond to non-linear models in input space。
The basic svm rules can be expressed in the feature space as follows. The following equation replaces the size of w with the linear sum of w, y and x. The advantage of using the kernel is that the original equation will not change, because kernel conversion is abstract in phi。
This is an example of kernel function. It is generally possible to start with the simplest version of the conversion and then gradually model with increasingly advanced kernel functions in order to avoid overcompatibility。
