spyrit.core.meas.DynamicLinearSplit.build_dynamic_forward

DynamicLinearSplit.build_dynamic_forward(motion: DeformationField, mode: str = 'bilinear', warping: str = 'image', verbose: bool = False) → None

Builds the dynamic forward operator \(A_{\rm{dyn}}\).

\[\text{diag}(A x_{t=1, ..., 2M}) = A_{\rm{dyn}} x,\]

where \(x_{t=1, ..., 2M} \in \mathbb{R}^{N \times 2M}\) is the temporal signal of interest, \(A \in \mathbb{R}^{2M \times N}\) is the splitted static acquisition matrix, \(x \in \mathbb{R}^L\) is the reference frame defined over an extended field-of-view, and \(A_{\rm{dyn}} \in \mathbb{R}^{M \times L}\) is the splitted dynamic forward operator that compensates the motion.

The dynamic measurement matrix \(A_{\rm{dyn}}\) is obtained by motion-compensation to a reference time, leveraging known deformation field.

The output is stored in the attribute self.A_dyn.

Important

There are two ways of building the dynamic matrix, namely warping='pattern' or warping='image'. When warping='pattern', the input deformation field motion needs to be respectively the inverse deformation field that compensates the motion. When warping='image', the input deformation field motion needs to be the direct deformation field that induces the motion.

Reminder: When looking at the images vectors as continuous functions from \(\mathbb{R}^2\) to \(\mathbb{R}\), we define the direct deformation as the function \(u \colon \mathbb{Z}^3 \mapsto \mathbb{R}^2\) such that, for \(k \in \{1, ..., 2M\}\) and \((i, j) \in \mathbb{Z}^2\),

\[x_{t=k}(i, j) = x_{t=1}(u(t=k, i, j))\]

The inverse deformation field is defined as \(v=u^{-1}\).

Note

Warping sharp patterns introduces a bias in the model due to interpolation artifacts. We recommend to exploit the image regularity by setting warping='image'.

Note

When working with splitted measurements, it is common practice to exploit the problem’s linearity by using a differential measurement strategy. This allows to eliminate ambient light and dark current offsets. The attribute H_dyn applies the differential strategy after motion compensation to avoid an additional error term [ref journal].

Args:

motion (DeformationField): Deformation field representing the scene motion. Need to pass the deformation field when warping is set to ‘image’, and the inverse deformation field when warping is set to ‘pattern’.

mode (str): Interpolation mode for constructing the dynamic matrix. Defaults to ‘bilinear’.

warping (str): Choose between ‘image’ or ‘pattern’. This parameter decides whether to warp the patterns or the (unknown) image to recover when building the dynamic measurement matrix. Defaults to ‘image’.

Returns:

None. The dynamic measurement matrix is stored in the attribute self.A_dyn.

Example:

>>> import torch
>>> from spyrit.core.noise import Poisson
>>> from spyrit.core.warp import DeformationField
>>> from spyrit.core.meas import DynamicLinearSplit
>>>
>>> def_field = DeformationField(torch.rand([800, 50, 50, 2]) * 2 - 1)  # dummy deformation field with 400 frames
>>> x = torch.rand([1, 3, 50, 50])  # dummy RGB reference image of size 50x50
>>> x_motion = def_field(x)  # dummy video obtained by warping x with def_field
>>> H = torch.rand([400, 40*40])  # dummy static measurement matrix
>>>
>>> alpha = 5  # noise level
>>> noise_op = Poisson(alpha=alpha, g=1/alpha)
>>> meas_op = DynamicLinearSplit(H, time_dim=1, meas_shape=(40, 40), img_shape=(50, 50), noise_model=noise_op)
>>> print(meas_op)
DynamicLinearSplit(
  (noise_model): Poisson()
)
>>>
>>> meas_op.build_dynamic_forward(def_field)
>>> print(meas_op.A_dyn.shape)
torch.Size([800, 2500])
>>> print(meas_op.H_dyn.shape)
torch.Size([400, 2500])

References:

[Maitre2024_2] Maitre, T., Bretin, E., Phan, R., Ducros, N., & Sdika, M. (2024, October). Dynamic single-pixel imaging on an extended field of view without warping the patterns. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 275-284). Cham: Springer Nature Switzerland. DOI: 10.1007/978-3-031-72104-5_27

[Maitre2026] (Submitted to TIP) Maitre, T., Bretin, E., Mahieu-Williame, L., Phan, R., Sdika, M., & Ducros, N. (2025). Dual-arm motion-compensated single-pixel imaging. HAL Id: hal-05068181