The 300-year-old newton act
Three princeton mathematicians found faster and stronger solutions, one of them chinese。
What's the newton method? I don't think it's a stranger to overexecuted students, but it's a constant search for the best solution for complex function f(x) near zero。
This is a very simple "approximate solvency" algorithm, which, because of its rapid pace of accumulation, is still being widely used today in areas such as computer visualization, logistics, finance and even pure mathematics, such as the development of auto-driving vehicles that distinguish traffic lights from parking signs。
But even so powerful, there is a disadvantage in the newton act, which is not applicable to all functions。
As a result, many mathematicians in the past few centuries have come to follow in an attempt to optimize on that basis. Now these three mathematicians have succeeded in expanding the range of functions that can be applied。
Like this complex binary function。
The new approach shows greater coherence and coverage than the traditional newton law。
According to the co-author, the newton method has 1,000 different applications for optimization, and their algorithms may replace it。
Let's see what happens。
Three mathematicians rewrite the classic newton act
Newton's method was born in the 17th century and was first introduced by a famous british mathematician, newton。
The core idea is to find the optimal solution to the function by constantly approaching the root or very small point of the function。
It's kind of like looking for the lowest point in a strange environment. In walking, the only information we need is two points: 1) whether or not we are on the upper or lower slope, i. E. The slope (a first-stage guide to the function); 2) and whether the slope is increased or decreased, i. E. The rate of change in the slope itself (a second-stage guide to the function)。
Using the above information, we can get a relatively quick approximation。
If the process is expressed mathematically, it is as follows:
Newton proved that, as long as the process was repeated, it would eventually approach the smallest value of the original complex function。
Moreover, the newton method has a significant efficiency advantage compared to an iterative approach (e. G., a decline in gradients), although the costing of each of these is higher than a decrease in gradients。
In simple terms, the newton method is more rapid than the gradient reduction, i. E. The lowest value is found within a smaller number of iterative periods and therefore also applies in a variety of situations。
But newton also warned:
While this method works in most cases, it is likely to go further and further if it starts at a point too far away from the real minimum。
And even more troublesome, the newton act has a significant disadvantage — not applicable to all functions。
Its core policy is to transform a complex function into a simpler function, and once the function is too complex, it is no longer the same。
Mathematicians then worked towards expanding the use of algorithms without sacrificing efficiency。
Until last summer, three researchers published the latest improvements to the newton act。
Extend the newton method to the widest functional category to date
In particular, they found that the newton method did not work well in dealing with certain complex functions (e. G., high-intensity functions) because it relied on the function's taylor to expand (a means of using guidance and multiple-formatting functions), which did not always provide a good description of the original function, especially when the function had many "valves " (local minimum values)。
So they suggested that if a function fulfils two conditions, it is easier to find the lowest value:
The former means that if you start looking from any location, you will not be caught up in the problem of a local minimum value, because there is only one minimum value and it will slide to this only minimum point, regardless of direction。
The latter means that the minimum values of the functions can be easily identified and calculated, since the functions of squares and forms are particularly easy to process, their squares are always non-negative and their minimum values are zero。
Next, in order to meet the above conditions, they used a technique called semi-planning to adjust taylor's roll-out, as follows:
1, fine-tuning taylor. Instead of directly using a functional taylor, it is fine-tuned so that it can be both convexed and squared。
2. Add adjustment factors. Add an adjustment factor to the taylor roll-out that will help them control the shape of the roll-out and bring it closer to the original function, while meeting the contours of peace and conditions。
3. Multi-benching. Their method allows for the use of any number of primers to carry out taylor's operation, which means that they can find the minimum value of the function more quickly. The use of more wizards allows algorithms to compress to a minimum at a higher speed (e. G., cubic speed)。
Eventually, they created this stronger version of the newton method, which can find the smallest values with fewer iterative numbers。
Their algorithms are as follows:
In the following function, the improved version of the newton iii (the newton iii method) provides theoretically for faster contraction and may be more effective in practice than the classic newton method, especially when the initial point is far from the lowest value point。
A chinese participant
This was done in collaboration with three mathematicians during princeton university。
Of these, jeffrey zhang, currently a post-doctoral researcher in biomedicine informatics and data science at yale university, works in large language models, data science and statistics, computational complexity, multi-formulation optimization, game theory and institutional design。
Prior to obtaining a ph. D. In preparatory science and financial engineering from princeton university, the mentor was professor amir ali ahmadi, also author of the paper。
Earlier, he obtained a bachelor's degree in computer science and economics and mathematics from yale university in 2014。
Another author, abraham chaudhry, is also a student of professor amir ali ahmadi and is currently a postdoctoral researcher at the georgia institute of technology. He studied undergraduate at brown university before he went to princeton。
In fact, before these three mathematicians appeared, many mathematicians tried。
As early as the 19th century, paffnut chebyshev, known as the "father of russian mathematics", proposed a newton method, which uses three equations (index 3) to approximate functions。
But when the original function involves multiple variables, his algorithm does not work。
More recently, in 2021, russian mathematician yuri nesterov demonstrated the function of using three equations to effectively approach any number of variables。
However, his approach could not be extended to the use of approximation functions such as four and five equations, which would otherwise be less efficient。
Now, three mathematicians took nestorov's results one step further。
As in the original version of the newton method, each of these new algorithms is still more costly to calculate than, for example, by decreasing the gradient。
Therefore, the current new exercise will not change the way auto-driving cars, machine learning algorithms or air traffic control systems operate. In these cases, the best option remains the decline in the gradient。
According to jason altschuler of the university of pennsylvania, many ideas for excellence take years to be fully put into practice. But it seems to be an entirely new perspective。
The algorithm developed by ahmadi, chaudhry and zhang could eventually go beyond the gradient in various applications, including machine learning, if, over time, the bottom computing technology required to run the newton method became more efficient, making the calculation cost of each of them lower。
The co-authors indicated that, theoretically, their current algorithms were indeed faster。
Papers:
Https://arxiv. Org/PDF/2311. 06374
♪ over ♪
Quantum position, qbitai
