A perceptual model of motion quality for rendering with adaptive refresh-rate and resolution

GYORGY DENES, University of Cambridge

AKSHAY JINDAL, University of Cambridge

ALIAKSEI MIKHAILIUK, University of Cambridge

RAFAŁ K. MANTIUK, University of Cambridge

5 A PERCEPTUAL MODEL FOR MOTION QUALITY

5.3 From ${\proselabel{}{{σ}}}$ to quality

Blur introduced by eye motion, hold-type blur, and spatial resolution will result in the loss of sharpness. To quantify this in terms of loss of perceived quality, we map the physical amount of blur to the perceived quality difference in JND units. Our blur quality function is inspired by the energy models of blur detection [Watson and Ahumada 2011]. Such mapping is applied to the orthogonal (${\proselabel{perceptual}{{σ}}}_O$) and parallel (${\proselabel{perceptual}{{σ}}}_P$) components of the anisotropic blur separately, resulting in two independent quality values ($Q_O$ and $Q_P$).

As we are interested in content-independent predictions, we assume the worst-case scenario: an infinitely thin line (Dirac delta function $δ(x)$), which contains uniform energy across all spatial frequencies. When convolved with a Gaussian blur kernel ${\prosedeflabel{perceptual}{{σ}}}$ in the spatial domain , the resulting image is a Gaussian function with standard deviation ${\proselabel{perceptual}{{σ}}}$. The Fourier transform of this signal is also a Gaussian, given by:

where ${\prosedeflabel{perceptual}{{ω}}}$ is in cpd . To account for the spatial contrast sensitivity of visual system, we modulate the Fourier coefficients with the CSF

$$\DeclareMathOperator*{\argmax}{arg\,max} \DeclareMathOperator*{\argmin}{arg\,min} \begin{align*} \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'm̃', 'perceptual', 'def', false, 'm')", "id":"perceptual-m̃", "sym":"m̃", "func":"perceptual", "localFunc":"m", "type":"def", "case":"equation"} }{ {\textit{m̃}} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'ω', 'perceptual', 'use', true, 'm̃')", "id":"perceptual-ω", "sym":"ω", "func":"perceptual", "localFunc":"m̃", "type":"use", "case":"equation"} }{ {\mathit{ω}} };\idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'σ', 'perceptual', 'use', true, 'm̃')", "id":"perceptual-σ", "sym":"σ", "func":"perceptual", "localFunc":"m̃", "type":"use", "case":"equation"} }{ {\mathit{σ}} } \right) & = \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'CSF', 'perceptual', 'use', false, 'm̃')", "id":"perceptual-CSF", "sym":"CSF", "func":"perceptual", "localFunc":"m̃", "type":"use", "case":"equation"} }{ {\mathit{CSF}} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'ω', 'perceptual', 'use', true, 'm̃')", "id":"perceptual-ω", "sym":"ω", "func":"perceptual", "localFunc":"m̃", "type":"use", "case":"equation"} }{ {\mathit{ω}} } \right)\idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'm', 'perceptual', 'use', false, 'm̃')", "id":"perceptual-m", "sym":"m", "func":"perceptual", "localFunc":"m̃", "type":"use", "case":"equation"} }{ {\mathit{m}} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'ω', 'perceptual', 'use', true, 'm̃')", "id":"perceptual-ω", "sym":"ω", "func":"perceptual", "localFunc":"m̃", "type":"use", "case":"equation"} }{ {\mathit{ω}} };\idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'σ', 'perceptual', 'use', true, 'm̃')", "id":"perceptual-σ", "sym":"σ", "func":"perceptual", "localFunc":"m̃", "type":"use", "case":"equation"} }{ {\mathit{σ}} } \right)\\\eqlabel{ {"onclick":"event.stopPropagation(); onClickEq(this, 'perceptual', ['CSF', 'm', 'σ', 'ω', 'm̃'], true, 'm̃', ['ω', 'σ'], 'bcyDKM-JO8-DKSA9IENTRijPiSkgbSjPiTvPgykgd2hlcmUgz4kg4oiIIOKEnSwgz4PiiIgg4oSd');"} }{} \end{align*} \tag{13}\label{13}$$

where CSF is Barten’s CSF model with the recommended standard observer parameters and the background luminance of 100 $cd/{\proselabel{perceptual}{{m}}}^2$ [Barten 2003].

To compute the overall energy in a distorted signal, we sample a range of frequencies ${\proselabel{perceptual}{{ω}}}_i={1,2,...,64}$[cpd],andcomputethe blur energy as:

$$\DeclareMathOperator*{\argmax}{arg\,max} \DeclareMathOperator*{\argmin}{arg\,min} \begin{align*} \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$E_b$', 'perceptual', 'def', false, 'm̃')", "id":"perceptual-$E_b$", "sym":"$E_b$", "func":"perceptual", "localFunc":"m̃", "type":"def", "case":"equation"} }{ {E_b} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'σ', 'perceptual', 'use', true, '$E_b$')", "id":"perceptual-σ", "sym":"σ", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {\mathit{σ}} } \right) & = \sum_{\mathit{i}} {\left( \frac{\idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'm̃', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-m̃", "sym":"m̃", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {\textit{m̃}} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'ω', 'perceptual', 'use', true, '$E_b$')", "id":"perceptual-ω", "sym":"ω", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {\mathit{ω}} }_{ \mathit{i} };\idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, 'σ', 'perceptual', 'use', true, '$E_b$')", "id":"perceptual-σ", "sym":"σ", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {\mathit{σ}} } \right)}{\idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$m̃_{t,b}$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$m̃_{t,b}$", "sym":"$m̃_{t,b}$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {m̃_{t,b}} }} \right)}^{\idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$β_b$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$β_b$", "sym":"$β_b$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {β_b} }}\\\eqlabel{ {"onclick":"event.stopPropagation(); onClickEq(this, 'perceptual', ['$β_b$', 'σ', '$m̃_{t,b}$', 'm̃', 'ω', '$E_b$'], true, '$E_b$', ['σ'], 'YCRFX2IkYCjPgykgPSBzdW1faSAoIChtzIMoz4lfaSA7IM-DKSkvYCRtzINfe3QsYn0kYCleYCTOsl9iJGAgIHdoZXJlIM-DIOKIiCDihJ0=');"} }{} \end{align*} \tag{14}\label{14}$$

where ${\prosedeflabel{perceptual}{{m̃_{t,b}}}}$ is the threshold parameter and ${\prosedeflabel{perceptual}{{β_b}}}$ is the power parameter of the model . Both of these are fitted to psychophysical data in Section 6.3.

Energy differences can be interpreted as quality differences, yielding:

$$\DeclareMathOperator*{\argmax}{arg\,max} \DeclareMathOperator*{\argmin}{arg\,min} \begin{align*} \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$∆Q_P$', 'perceptual', 'def', false, '$E_b$')", "id":"perceptual-$∆Q_P$", "sym":"$∆Q_P$", "func":"perceptual", "localFunc":"$E_b$", "type":"def", "case":"equation"} }{ {∆Q_P} } & = \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$E_b$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$E_b$", "sym":"$E_b$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {E_b} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$σ_P^A$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$σ_P^A$", "sym":"$σ_P^A$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {σ_P^A} } \right) - \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$E_b$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$E_b$", "sym":"$E_b$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {E_b} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$σ_P^B$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$σ_P^B$", "sym":"$σ_P^B$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {σ_P^B} } \right)\\\eqlabel{ {"onclick":"event.stopPropagation(); onClickEq(this, 'perceptual', ['$E_b$', '$σ_P^A$', '$σ_P^B$', '$∆Q_P$'], false, [], [], 'YCTiiIZRX1AkYCA9IGAkRV9iJGAoYCTPg19QXkEkYCkgLSBgJEVfYiRgKGAkz4NfUF5CJGAp');"} }{} \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$∆Q_O$', 'perceptual', 'def', false, '$E_b$')", "id":"perceptual-$∆Q_O$", "sym":"$∆Q_O$", "func":"perceptual", "localFunc":"$E_b$", "type":"def", "case":"equation"} }{ {∆Q_O} } & = \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$E_b$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$E_b$", "sym":"$E_b$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {E_b} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$σ_O^A$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$σ_O^A$", "sym":"$σ_O^A$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {σ_O^A} } \right) - \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$E_b$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$E_b$", "sym":"$E_b$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {E_b} }\left( \idlabel{ {"onclick":"event.stopPropagation(); onClickSymbol(this, '$σ_O^B$', 'perceptual', 'use', false, '$E_b$')", "id":"perceptual-$σ_O^B$", "sym":"$σ_O^B$", "func":"perceptual", "localFunc":"$E_b$", "type":"use", "case":"equation"} }{ {σ_O^B} } \right)\\\eqlabel{ {"onclick":"event.stopPropagation(); onClickEq(this, 'perceptual', ['$E_b$', '$σ_O^B$', '$σ_O^A$', '$∆Q_O$'], false, [], [], 'YCTiiIZRX08kYCA9IGAkRV9iJGAoYCTPg19PXkEkYCkgLSBgJEVfYiRgKGAkz4NfT15CJGAp');"} }{} \end{align*} \tag{15}\label{15}$$

substituting in the standard deviations of the blur components for $A$ and $B$, in the directions parallel ($P$) and orthogonal ($O$) to SPEM.

We further explain why an energy model is suitable to predict JND differences in the supplemental material (Section S.1).