|
|
| (335 intermediate revisions by more than 100 users not shown) |
| Line 1: |
Line 1: |
| A '''Viterbi decoder''' uses the [[Viterbi algorithm]] for decoding a bitstream that has been
| | This is a preview for the new '''MathML rendering mode''' (with SVG fallback), which is availble in production for registered users. |
| encoded using a [[convolutional code]].
| |
|
| |
|
| There are other algorithms for decoding a convolutionally encoded stream (for example, the [[Robert Fano|Fano algorithm]]). The Viterbi algorithm is the most resource-consuming, but it does the [[maximum likelihood]] decoding. It is most often used for decoding convolutional codes with constraint lengths k<=10, but values up to k=15 are used in practice.
| | If you would like use the '''MathML''' rendering mode, you need a wikipedia user account that can be registered here [[https://en.wikipedia.org/wiki/Special:UserLogin/signup]] |
| | * Only registered users will be able to execute this rendering mode. |
| | * Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere. |
|
| |
|
| Viterbi decoding was developed by [[Andrew J. Viterbi]] and published in the paper "Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm", [[IEEE Transactions on Information Theory]], Volume IT-13, pages 260-269, in April, 1967.
| | Registered users will be able to choose between the following three rendering modes: |
|
| |
|
| There are both hardware (in modems) and software implementations of a Viterbi decoder.
| | '''MathML''' |
| | :<math forcemathmode="mathml">E=mc^2</math> |
|
| |
|
| == Hardware implementation == | | <!--'''PNG''' (currently default in production) |
| | :<math forcemathmode="png">E=mc^2</math> |
|
| |
|
| [[Image:Viterbi decoder hardware implementation.png|right|thumb|350px|A common way to implement a hardware viterbi decoder]]
| | '''source''' |
| | :<math forcemathmode="source">E=mc^2</math> --> |
|
| |
|
| A hardware Viterbi decoder for basic (not punctured) code usually consists of the following major blocks:
| | <span style="color: red">Follow this [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering link] to change your Math rendering settings.</span> You can also add a [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering-skin Custom CSS] to force the MathML/SVG rendering or select different font families. See [https://www.mediawiki.org/wiki/Extension:Math#CSS_for_the_MathML_with_SVG_fallback_mode these examples]. |
|
| |
|
| *Branch metric unit (BMU)
| | ==Demos== |
| *Path metric unit (PMU)
| |
| *Traceback unit (TBU)
| |
|
| |
|
| === Branch metric unit (BMU) === | | Here are some [https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Frederic.wang demos]: |
|
| |
|
| [[Image:Viterbi decoder hardware implementation BMU.png|right|thumb|350px|A sample implementation of a branch metric unit]]
| |
|
| |
|
| A branch metric unit's function is to calculate ''branch metrics'', which are normed distances between every possible symbol in the code alphabet, and the received symbol.
| | * accessibility: |
| | ** Safari + VoiceOver: [https://commons.wikimedia.org/wiki/File:VoiceOver-Mac-Safari.ogv video only], [[File:Voiceover-mathml-example-1.wav|thumb|Voiceover-mathml-example-1]], [[File:Voiceover-mathml-example-2.wav|thumb|Voiceover-mathml-example-2]], [[File:Voiceover-mathml-example-3.wav|thumb|Voiceover-mathml-example-3]], [[File:Voiceover-mathml-example-4.wav|thumb|Voiceover-mathml-example-4]], [[File:Voiceover-mathml-example-5.wav|thumb|Voiceover-mathml-example-5]], [[File:Voiceover-mathml-example-6.wav|thumb|Voiceover-mathml-example-6]], [[File:Voiceover-mathml-example-7.wav|thumb|Voiceover-mathml-example-7]] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Audio-Windows7-InternetExplorer.ogg Internet Explorer + MathPlayer (audio)] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-SynchronizedHighlighting-WIndows7-InternetExplorer.png Internet Explorer + MathPlayer (synchronized highlighting)] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Braille-Windows7-InternetExplorer.png Internet Explorer + MathPlayer (braille)] |
| | ** NVDA+MathPlayer: [[File:Nvda-mathml-example-1.wav|thumb|Nvda-mathml-example-1]], [[File:Nvda-mathml-example-2.wav|thumb|Nvda-mathml-example-2]], [[File:Nvda-mathml-example-3.wav|thumb|Nvda-mathml-example-3]], [[File:Nvda-mathml-example-4.wav|thumb|Nvda-mathml-example-4]], [[File:Nvda-mathml-example-5.wav|thumb|Nvda-mathml-example-5]], [[File:Nvda-mathml-example-6.wav|thumb|Nvda-mathml-example-6]], [[File:Nvda-mathml-example-7.wav|thumb|Nvda-mathml-example-7]]. |
| | ** Orca: There is ongoing work, but no support at all at the moment [[File:Orca-mathml-example-1.wav|thumb|Orca-mathml-example-1]], [[File:Orca-mathml-example-2.wav|thumb|Orca-mathml-example-2]], [[File:Orca-mathml-example-3.wav|thumb|Orca-mathml-example-3]], [[File:Orca-mathml-example-4.wav|thumb|Orca-mathml-example-4]], [[File:Orca-mathml-example-5.wav|thumb|Orca-mathml-example-5]], [[File:Orca-mathml-example-6.wav|thumb|Orca-mathml-example-6]], [[File:Orca-mathml-example-7.wav|thumb|Orca-mathml-example-7]]. |
| | ** From our testing, ChromeVox and JAWS are not able to read the formulas generated by the MathML mode. |
|
| |
|
| There are hard decision and soft decision Viterbi decoders. A hard decision Viterbi decoder receives a simple bitstream on its input, and a [[Hamming distance]] is used as a metric. A soft decision Viterbi decoder receives a bitstream containing information about the ''reliability'' of each received symbol. For instance, in a 3-bit encoding, this ''reliability'' information is encoded as follows:
| | ==Test pages == |
|
| |
|
| {| border
| | To test the '''MathML''', '''PNG''', and '''source''' rendering modes, please go to one of the following test pages: |
| | value
| | *[[Displaystyle]] |
| | meaning
| | *[[MathAxisAlignment]] |
| |-
| | *[[Styling]] |
| | ''000''
| | *[[Linebreaking]] |
| | strongest '''0'''
| | *[[Unique Ids]] |
| |-
| | *[[Help:Formula]] |
| | ''001''
| |
| | relatively strong '''0'''
| |
| |-
| |
| | ''010''
| |
| | relatively weak '''0'''
| |
| |-
| |
| | ''011''
| |
| | weakest '''0'''
| |
| |-
| |
| | ''100''
| |
| | weakest '''1'''
| |
| |-
| |
| | ''101''
| |
| | relatively weak '''1'''
| |
| |-
| |
| | ''110''
| |
| | relatively strong '''1'''
| |
| |-
| |
| | ''111''
| |
| | strongest '''1'''
| |
| |}
| |
|
| |
|
| Of course, it is not the only way to encode reliability data.
| | *[[Inputtypes|Inputtypes (private Wikis only)]] |
| | | *[[Url2Image|Url2Image (private Wikis only)]] |
| The ''squared'' [[Euclidean distance]] is used as a metric for soft decision decoders.
| | ==Bug reporting== |
| | | If you find any bugs, please report them at [https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=Math&version=master&short_desc=Math-preview%20rendering%20problem Bugzilla], or write an email to math_bugs (at) ckurs (dot) de . |
| === Path metric unit (PMU) ===
| |
| | |
| [[Image:Viterbi decoder hardware implementation PMU.png|right|thumb|350px|A sample implementation of a path metric unit for a specific K=4 decoder]]
| |
| | |
| A path metric unit summarizes branch metrics to get metrics for <math>2^{K-1}</math> paths, where K is the constraint length of the code, one of which can eventually be chosen as ''optimal''. Every clock it makes <math>2^{K-1}</math> decisions, throwing off wittingly nonoptimal paths. The results of these decisions are written to the memory of a traceback unit.
| |
| | |
| The core elements of a PMU are ''ACS (Add-Compare-Select)'' units. The way in which they are connected between themselves is defined by a specific code's [[trellis diagram]].
| |
| | |
| Since branch metrics are always <math>\ge 0</math>, there must be an additional circuit preventing metric counters from overflow (it isn't shown on the image). An alternate method that eliminates the need to monitor the path metric growth is to allow the path metrics to "roll over", to use this method it is necessary to make sure the path metric accumulators contain enough bits to prevent the "best" and "worst" values from coming within 2<sup>(n-1)</sup> of each other. The compare circuit is essentially unchanged.
| |
| | |
| [[Image:Viterbi decoder hardware implementation ACS.png|right|thumb|350px|A sample implementation of an ACS unit]] | |
| | |
| It is possible to monitor the noise level on the incoming bit stream by monitoring the rate of growth of the "best" path metric. A simpler way to do this is to monitor a single location or "state" and watch it pass "upward" through say four discrete levels within the range of the accumulator. As it passes upward through each of these thresholds, a counter is incremented that reflects the "noise" present on the incoming signal.
| |
| | |
| === Traceback unit (TBU) ===
| |
| | |
| [[Image:Viterbi decoder hardware implementation TBU.png|right|thumb|350px|A sample implementation of a traceback unit]]
| |
| | |
| Back-trace unit restores an (almost) maximum-likelihood path from the decisions made by PMU. Since it does it in inverse direction, a viterbi decoder comprises a FILO (first-in-last-out) buffer to reconstruct a correct order.
| |
| | |
| Note that the implementation shown on the image requires double frequency. There are some tricks that eliminate this requirement.
| |
| | |
| == Implementation issues == | |
| === Quantization for soft decision decoding ===
| |
| In order to fully exploit benefits of soft decision decoding, one needs to quantize the input signal properly. The optimal quantization zone width is defined by the following formula:
| |
| | |
| <math>\,\! T = \sqrt{N_0/2^k},</math>
| |
| | |
| where <math>N_0</math> is a noise power spectral density, and ''k'' is a number of bits for soft decision.
| |
| | |
| === Euclidean metric computation ===
| |
| | |
| The squared [[norm (mathematics)|norm]] (''<math>\ell</math><sub>2</sub>'') distance between the received and the actual symbols in the code alphabet may be further simplified into a linear sum/difference form, which makes it less computationally intensive.
| |
| | |
| Consider a 1/2 [[convolutional code]]r, which generates 2 bits (''00'', ''01'', ''10'' or ''11'') for every input bit (''1'' or ''0''). These ''Return-to-Zero'' signals are translated into a ''Non-Return-to-Zero'' form shown alongside.
| |
| | |
| {| border
| |
| | code alphabet
| |
| | vector mapping
| |
| |-
| |
| | ''00''
| |
| | ''1, 1''
| |
| |-
| |
| | ''01''
| |
| | ''1, -1''
| |
| |-
| |
| | ''10''
| |
| | ''-1, 1''
| |
| |-
| |
| | ''11''
| |
| | ''-1, -1''
| |
| |}
| |
| | |
| Each received symbol may be represented in vector form as '''v<sub>r</sub>''' = {r<sub>0</sub>, r<sub>1</sub>}, where r<sub>0</sub> and r<sub>1</sub> are soft decision values, whose magnitudes signify the ''joint reliability'' of the received vector, '''v<sub>r</sub>'''.
| |
| | |
| Every symbol in the code alphabet may, likewise, be represented by the vector '''v<sub>i</sub>''' = {±1, ±1}.
| |
| | |
| The actual computation of the Euclidean distance metric is:
| |
| | |
| <math>\,\!D = (\overrightarrow{v_r} - \overrightarrow{v_i})^2 = \overrightarrow{v_r}^2 - 2 \overrightarrow{v_r} \overrightarrow{v_i} + \overrightarrow{v_i}^2</math>
| |
| | |
| Each square term is a normed distance, depicting the ''energy'' of the symbol. For ex., the ''energy'' of the symbol '''v<sub>i</sub>''' = {±1, ±1} may be computed as
| |
| | |
| <math>\,\!\overrightarrow{v_i}^2 = (\pm 1)^2 + (\pm 1)^2 = 2</math>
| |
| | |
| Thus, the energy term of all symbols in the code alphabet is constant (at (''normalized'') value 2).
| |
| | |
| The ''Add-Compare-Select'' (''ACS'') operation compares the metric distance between the received symbol '''||v<sub>r</sub>||''' and any 2 symbols in the code alphabet whose paths merge at a node in the corresponding trellis, '''||v<sub>i</sub><sup>(0)</sup>||''' and '''||v<sub>i</sub><sup>(1)</sup>||'''. This is equivalent to comparing
| |
| | |
| <math>\,\!D_0 = \overrightarrow{v_r}^2 - 2 \overrightarrow{v_r} \overrightarrow{v_i^0} + \overrightarrow{v_i^0}^2</math>
| |
| | |
| and
| |
| | |
| <math>\,\!D_1 = \overrightarrow{v_r}^2 - 2 \overrightarrow{v_r} \overrightarrow{v_i^1} + \overrightarrow{v_i^1}^2</math>
| |
| | |
| But, from above we know that the ''energy'' of '''v<sub>i</sub>''' is constant (equal to (normalized) value of 2), and the ''energy'' of '''v<sub>r</sub>''' is the same in both cases. This reduces the comparison to a minima function between the 2 (middle) ''[[dot product]]'' terms,
| |
| | |
| <math>\,\!min(-2 \overrightarrow{v_r} \overrightarrow{v_i^0},-2 \overrightarrow{v_r} \overrightarrow{v_i^1}) = max(\overrightarrow{v_r} \overrightarrow{v_i^0}, \overrightarrow{v_r} \overrightarrow{v_i^1})</math>
| |
| | |
| since a ''min'' operation on negative numbers may be interpreted as an equivalent ''max'' operation on positive quantities.
| |
| | |
| Each ''[[dot product]]'' term may be expanded as
| |
| | |
| <math>\,\! max(\pm r_0 \pm r_1, \pm r_0 \pm r_1)</math>
| |
| | |
| where, the signs of each term depend on symbols, '''v<sub>i</sub><sup>(0)</sup>''' and '''v<sub>i</sub><sup>(1)</sup>''', being compared. Thus, the ''squared'' Euclidean metric distance calculation to compute the ''branch metric'' may be performed with a simple add/subtract operation.
| |
| | |
| === Traceback ===
| |
| | |
| The general approach to traceback is to accumulate path metrics for up to five times the constraint length (''5 * (K - 1)''), find the node with the largest accumulated cost, and begin traceback from this node.
| |
| | |
| However, computing the node which has accumulated the largest cost (either the largest or smallest integral path metric) involves finding the ''maxima'' or ''minima'' of several (usually 2<sup>''K-1''</sup>) numbers, which may be time consuming when implemented on embedded hardware systems.
| |
| | |
| Most communication systems employ Viterbi decoding involving data packets of fixed sizes, with a fixed [[bit]]/[[byte]] pattern either at the beginning or/and at the end of the data packet. By using the known [[bit]]/[[byte]] pattern as reference, the start node may be set to a fixed value, thereby obtaining a perfect Maximum Likelihood Path during traceback.
| |
| | |
| == Limitations ==
| |
| | |
| A physical implementation of a viterbi decoder will not yield an ''exact'' maximum-likelihood stream due to [[Quantization (signal processing)|quantization]] of the input signal, branch and path metrics, and finite ''traceback length''. Practical implementations do approach within 1dB of the ideal.
| |
| | |
| == Punctured codes ==
| |
| | |
| A hardware viterbi decoder of ''[[Puncturing|punctured]] codes'' is commonly implemented in such a way:
| |
| | |
| * A depuncturer, which transforms the input stream into the stream which looks like an original (non punctured) stream with ERASE marks at the places where bits were erased.
| |
| * A basic viterbi decoder understanding these ERASE marks (that is, not using them for branch metric calculation).
| |
| | |
| == Software implementation ==
| |
| | |
| See [[Viterbi algorithm]].
| |
| | |
| One of the most time-consuming operations is an ACS butterfly, which is usually implemented using an [[assembly language]] and appropriate instruction set extensions (such as [[SSE2]]) to speed up the decoding time.
| |
| | |
| == Applications ==
| |
| | |
| The Viterbi decoding algorithm is widely used in the following areas:
| |
| *Radio communication: digital TV ([[ATSC]], [[QAM (television)|QAM]], [[DVB-T]], etc.), [[Radio relay link|radio relay]], [[satellite communications]], [[PSK31]] digital mode for [[amateur radio]].
| |
| *Decoding [[trellis-coded modulation]] (TCM), the technique used in telephone-line modems to squeeze high [[spectral efficiency]] out of 3 kHz-bandwidth analog telephone lines.
| |
| *Computer storage devices such as [[hard disk drive]]s.
| |
| *[[Automatic speech recognition]]
| |
| | |
| ==External links==
| |
| *[http://arxiv.org/abs/cs/0504020v2 David Forney's take on the history of the Viterbi algorithm]
| |
| *[http://home.netcom.com/~chip.f/viterbi/tutorial.html Details on Viterbi decoding, as well as a bibliography].
| |
| *[http://www.1-core.com/library/comm/viterbi/ Viterbi algorithm explanation with the focus on hardware implementation issues].
| |
| *[http://quest.nasa.gov/saturn/qa/cassini/Error_correction.txt r=1/6 k=15 coding for the Cassini mission to Saturn].
| |
| *[http://www.spiral.net/software/viterbi.html Online Generator of optimized software Viterbi decoders (GPL)].
| |
| *[http://www.ka9q.net/code/fec/ GPL Viterbi decoder software for four standard codes].
| |
| *[http://opencores.org/project,viterbi_decoder_axi4s Generic Viterbi decoder hardware (GPL)].
| |
| | |
| [[Category:Data transmission]]
| |
| [[Category:Error detection and correction]]
| |
| | |
| [[ru:Алгоритм свёрточного декодирования Витерби]]
| |
