spyrit.core.meas.DynamicLinear.build_dynamic_forward

DynamicLinear.build_dynamic_forward(motion: DeformationField, mode: str = 'bilinear', warping: str = 'image', verbose: bool = False) → None

Builds the dynamic forward operator \(H_{\rm{dyn}}\).

\[\text{diag}(H x_{t=1, ..., M}) = H_{\rm{dyn}} x,\]

where \(x_{t=1, ..., M} \in \mathbb{R}^{N \times M}\) is the temporal signal of interest, \(H \in \mathbb{R}^{M \times N}\) is the static acquisition matrix, \(x \in \mathbb{R}^L\) is the reference frame defined over an extended field-of-view, and \(H_{\rm{dyn}} \in \mathbb{R}^{M \times L}\) is the dynamic forward operator that compensates the motion.

The dynamic measurement matrix \(H_{\rm{dyn}}\) is obtained by motion-compensation to a reference time, leveraging known deformation field.

The output is stored in the attribute self.H_dyn.

Important

There are two ways of building the dynamic matrix, namely warping='pattern' or warping='image'. When warping='pattern', the input deformation field motion needs to be respectively the inverse deformation field that compensates the motion. When warping='image', the input deformation field motion needs to be the direct deformation field that induces the motion.

Reminder: When looking at the images vectors as continuous functions from \(\mathbb{R}^2\) to \(\mathbb{R}\), we define the direct deformation as the function \(u \colon \mathbb{Z}^3 \mapsto \mathbb{R}^2\) such that, for \(k \in \{1, ..., M\}\) and \((i, j) \in \mathbb{Z}^2\),

\[x_{t=k}(i, j) = x_{t=1}(u(t=k, i, j))\]

The inverse deformation field is defined as \(v=u^{-1}\).

Note

Warping sharp patterns introduces a bias in the model due to interpolation artifacts. We recommend to exploit the image regularity by setting warping='image'.

Args:

motion (DeformationField): Deformation field representing the scene motion. Need to pass the direct deformation field when warping is set to ‘image’, and the inverse deformation field when warping is set to ‘pattern’.

mode (str): Interpolation mode for constructing the dynamic matrix. Defaults to ‘bilinear’.

warping (str): Choose between ‘image’ or ‘pattern’. This parameter decides whether to warp the patterns or the (unknown) image to recover when building the dynamic measurement matrix. Defaults to ‘image’.

Returns:

None. The dynamic measurement matrix is stored in the attribute self.H_dyn.

Example:

>>> import torch
>>> from spyrit.core.noise import Poisson
>>> from spyrit.core.warp import DeformationField
>>> from spyrit.core.meas import DynamicLinear
>>>
>>> def_field = DeformationField(torch.rand([400, 50, 50, 2]) * 2 - 1)  # dummy deformation field with 400 frames
>>> x = torch.rand([1, 3, 50, 50])  # dummy RGB reference image of size 50x50
>>> x_motion = def_field(x)  # dummy video obtained by warping x with def_field
>>> H = torch.rand([400, 40*40])  # dummy static measurement matrix
>>>
>>> alpha = 5  # noise level
>>> noise_op = Poisson(alpha=alpha, g=1/alpha)
>>> meas_op = DynamicLinear(H, time_dim=1, meas_shape=(40, 40), img_shape=(50, 50), noise_model=noise_op)
>>> print(meas_op)
DynamicLinear(
  (noise_model): Poisson()
)
>>>
>>> meas_op.build_dynamic_forward(def_field)
>>> print(meas_op.H_dyn.shape)
torch.Size([400, 2500])

References:

[Maitre2024_2] Maitre, T., Bretin, E., Phan, R., Ducros, N., & Sdika, M. (2024, October). Dynamic single-pixel imaging on an extended field of view without warping the patterns. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 275-284). Cham: Springer Nature Switzerland. DOI: 10.1007/978-3-031-72104-5_27

[Maitre2026] (Submitted to TIP) Maitre, T., Bretin, E., Mahieu-Williame, L., Phan, R., Sdika, M., & Ducros, N. (2025). Dual-arm motion-compensated single-pixel imaging. HAL Id: hal-05068181