|
|
| Line 1: |
Line 1: |
| In [[computer vision]], the '''Lucas–Kanade method''' is a widely used differential method for [[optical flow]] estimation developed by [[Bruce D. Lucas]] and [[Takeo Kanade]]. It assumes that the flow is essentially constant in a local neighbourhood of the [[pixel]] under consideration, and solves the basic optical flow equations for all the pixels in that neighbourhood, by the [[least squares method|least squares criterion]].<ref name=LKproc>
| | Hi there. My title is Sophia Meagher although it is not the name on my birth certificate. Alaska is where he's usually been residing. To perform lacross is something I truly enjoy performing. I am an invoicing officer and I'll be promoted quickly.<br><br>Here is my weblog - tarot readings ([http://www.edmposts.com/build-a-beautiful-organic-garden-using-these-ideas/ Our Web Page]) |
| B. D. Lucas and T. Kanade (1981), ''[http://www-cse.ucsd.edu/classes/sp02/cse252/lucaskanade81.pdf An iterative image registration technique with an application to stereo vision.]'' Proceedings of Imaging Understanding Workshop, pages 121--130
| |
| </ref><ref name="Lthesis">Bruce D. Lucas (1984) ''[http://www.ri.cmu.edu/pub_files/pub4/lucas_bruce_d_1984_1/lucas_bruce_d_1984_1.pdf Generalized Image Matching by the Method of Differences]'' (doctoral dissertation)
| |
| </ref>
| |
| | |
| By combining information from several nearby pixels, the Lucas–Kanade method can often resolve the inherent ambiguity of the optical flow equation. It is also less sensitive to image noise than point-wise methods. On the other hand, since it is a purely local method, it cannot provide flow information in the interior of uniform regions of the image.
| |
| | |
| == Concept ==
| |
| The Lucas–Kanade method assumes that the displacement of the image contents between two nearby instants (frames) is small and approximately constant within a neighborhood of the point ''p'' under consideration. Thus the [[optical flow|optical flow equation]] can be assumed to hold for all pixels within a window centered at ''p''. Namely, the local image flow (velocity) vector <math>(V_x,V_y)</math> must satisfy
| |
| | |
| :<math>I_x(q_1) V_x + I_y (q_1) V_y = -I_t(q_1)</math>
| |
| | |
| :<math>I_x(q_2) V_x + I_y (q_2) V_y = -I_t(q_2)</math>
| |
| | |
| :<math>\vdots</math>
| |
| | |
| :<math>I_x(q_n) V_x + I_y (q_n) V_y = -I_t(q_n)</math>
| |
| | |
| where <math>q_1,q_2,\dots,q_n</math> are the pixels inside the window, and <math>I_x(q_i),I_y(q_i),I_t(q_i)</math> are the partial derivatives of the image <math>I</math> with respect to position ''x'', ''y'' and time ''t'', evaluated at the point <math>q_i</math> and at the current time. | |
| | |
| These equations can be written in [[matrix (mathematics)|matrix]] form <math>A v = b</math>, where
| |
| :<math>A = \begin{bmatrix}
| |
| I_x(q_1) & I_y(q_1) \\[10pt]
| |
| I_x(q_2) & I_y(q_2) \\[10pt]
| |
| \vdots & \vdots \\[10pt]
| |
| I_x(q_n) & I_y(q_n)
| |
| \end{bmatrix},
| |
| \quad\quad
| |
| v =
| |
| \begin{bmatrix}
| |
| V_x\\[10pt]
| |
| V_y
| |
| \end{bmatrix},
| |
| \quad \mbox{and}\quad
| |
| b =
| |
| \begin{bmatrix}
| |
| -I_t(q_1) \\[10pt]
| |
| -I_t(q_2) \\[10pt]
| |
| \vdots \\[10pt]
| |
| -I_t(q_n)
| |
| \end{bmatrix}
| |
| </math>
| |
| | |
| This system has more equations than unknowns and thus it is usually over-determined. The Lucas–Kanade method obtains a compromise solution
| |
| by the [[linear least squares (mathematics)|least squares]] principle. Namely, it solves the 2×2 system
| |
| :<math>A^T A v=A^T b</math> or
| |
| :<math> \mathrm{v}=(A^T A)^{-1}A^T b</math>
| |
| where <math>A^T</math> is the [[transposed matrix|transpose]] of matrix <math>A</math>. That is, it computes
| |
| :<math>\begin{bmatrix}
| |
| V_x\\[10pt]
| |
| V_y
| |
| \end{bmatrix}
| |
| =
| |
| \begin{bmatrix}
| |
| \sum_i I_x(q_i)^2 & \sum_i I_x(q_i)I_y(q_i) \\[10pt]
| |
| \sum_i I_y(q_i)I_x(q_i) & \sum_i I_y(q_i)^2
| |
| \end{bmatrix}^{-1}
| |
| \begin{bmatrix}
| |
| -\sum_i I_x(q_i)I_t(q_i) \\[10pt]
| |
| -\sum_i I_y(q_i)I_t(q_i)
| |
| \end{bmatrix}
| |
| </math>
| |
| with the sums running from ''i''=1 to ''n''.
| |
| | |
| The matrix <math>A^T A</math> is often called the [[structure tensor]] of the image at the point ''p''.
| |
| | |
| == Weighted window ==
| |
| The plain least squares solution above gives the same importance to all ''n'' pixels <math>q_i</math> in the window. In practice it is usually better to give more weight to the pixels that are closer to the central pixel ''p''. For that, one uses the weighted version of the least squares equation,
| |
| :<math>A^T W A v=A^T W b</math>
| |
| or
| |
| :<math> \mathrm{v}=(A^T W A)^{-1}A^T W b</math>
| |
| where <math>W</math> is an ''n''×''n'' [[diagonal matrix]] containing the weights <math>W_{i i} = w_i</math> to be assigned to the equation of pixel <math>q_i</math>. That is, it computes
| |
| :<math>\begin{bmatrix}
| |
| V_x\\[10pt]
| |
| V_y
| |
| \end{bmatrix}
| |
| =
| |
| \begin{bmatrix}
| |
| \sum_i w_i I_x(q_i)^2 & \sum_i w_i I_x(q_i)I_y(q_i) \\[10pt]
| |
| \sum_i w_i I_x(q_i)I_y(q_i) & \sum_i w_i I_y(q_i)^2
| |
| \end{bmatrix}^{-1}
| |
| \begin{bmatrix}
| |
| -\sum_i w_i I_x(q_i)I_t(q_i) \\[10pt]
| |
| -\sum_i w_i I_y(q_i)I_t(q_i)
| |
| \end{bmatrix}
| |
| </math>
| |
| | |
| The weight <math>w_i</math> is usually set to a [[Gaussian distribution|Gaussian function]] of the distance between <math>q_i</math> and ''p''.
| |
| | |
| == Improvements and extensions ==
| |
| The least-squares approach implicitly assumes that the errors in the image data have a Gaussian distribution with zero mean. If one expects the window to contain a certain percentage of "[[outlier]]s" (grossly wrong data values, that do not follow the "ordinary" Gaussian error distribution), one may use statistical analysis to detect them, and reduce their weight accordingly.
| |
| | |
| The Lucas–Kanade method per se can be used only when the image flow vector <math>V_x,V_y</math> between the two frames is small enough for the differential equation of the optical flow to hold, which is often less than the pixel spacing. When the flow vector may exceed this limit, such as in stereo matching or warped document registration, the Lucas–Kanade method may still be used to refine some coarse estimate of the same, obtained by other means; for example, by [[extrapolation|extrapolating]] the flow vectors computed for previous frames, or by running the Lucas-Kanade algorithm on reduced-scale versions of the images. Indeed, the latter method is the basis of the popular [[Kanade–Lucas–Tomasi Feature Tracker|Kanade-Lucas-Tomasi (KLT)]] feature matching algorithm.
| |
| | |
| A similar technique can be used to compute differential [[affine mapping|affine]] deformations of the image contents.
| |
| | |
| == See also ==
| |
| * [[Optical flow]]
| |
| * [[Horn–Schunck method]]
| |
| * [[Corner_detection#The_Shi_and_Tomasi_corner_detection_algorithm|The Shi and Tomasi corner detection algorithm]]
| |
| | |
| ==References==
| |
| {{Reflist}}
| |
| | |
| ==External links==
| |
| * [http://www.cs.cmu.edu/~kangli/code/Image_Stabilizer.html The image stabilizer plugin for ImageJ] based on the Lucas–Kanade method
| |
| * [http://www.mathworks.com/matlabcentral/fileexchange/24677 Mathworks Lucas-Kanade] Matlab implementation of inverse and normal [[affine]] Lucas-Kanade
| |
| * [http://www.onera.fr/dtim-en/gpu-for-image/folkigpu.php FolkiGPU :] GPU implementation of an iterative Lucas-Kanade based optical flow
| |
| * [http://www.ces.clemson.edu/~stb/klt/ KLT]: An Implementation of the Kanade–Lucas–Tomasi Feature Tracker
| |
| * [http://www.ri.cmu.edu/people/kanade_takeo.html Takeo Kanade]
| |
| | |
| {{DEFAULTSORT:Lucas-Kanade method}}
| |
| [[Category:Motion in computer vision]]
| |
Hi there. My title is Sophia Meagher although it is not the name on my birth certificate. Alaska is where he's usually been residing. To perform lacross is something I truly enjoy performing. I am an invoicing officer and I'll be promoted quickly.
Here is my weblog - tarot readings (Our Web Page)