qfeval_functions.functions.nancorrel

nancorrel(x, y, dim=(), keepdim=False)[source]

Compute Pearson correlation coefficient between two tensors, ignoring NaN values.

This function calculates the Pearson correlation coefficient between tensors x and y along the specified dimension(s), excluding any pairs where either value is NaN. The correlation coefficient measures the linear relationship between two variables and ranges from -1 (perfect negative correlation) to 1 (perfect positive correlation), with 0 indicating no linear correlation.

Unlike correl(), this function handles missing data by ignoring NaN values in the computation. If either x or y has a NaN at a given position, that pair is excluded from the correlation calculation.

The NaN-aware Pearson correlation is computed as:

\[r = \frac{\text{E}[(X - \mu_X)(Y - \mu_Y)]}{ \sqrt{\text{E}[(X - \mu_X)^2]}\sqrt{\text{E}[(Y - \mu_Y)^2]}}\]

where the expectations are computed only over valid (non-NaN) pairs.

Parameters:

x (Tensor) – The first input tensor.
y (Tensor) – The second input tensor. Must be the same shape as x.
dim (Union[int, Tuple[int, ...]]) – The dimension(s) along which to compute the correlation. If not specified (default is empty tuple), computes element-wise correlation and sums the result.
keepdim (bool) – Whether the output tensor has dim retained or not. Default is False.

Returns:

The Pearson correlation coefficient(s), computed only over valid (non-NaN) pairs. The shape depends on the input dimensions and the keepdim parameter.

Return type:

Tensor

Example

>>> # Perfect positive correlation with some NaNs
>>> x = torch.tensor([1.0, 2.0, nan, 4.0, 5.0])
>>> y = torch.tensor([2.0, 4.0, 6.0, 8.0, nan])
>>> QF.nancorrel(x, y, dim=0)
tensor(1.)

>>> # 2D tensors with NaN values
>>> x = torch.tensor([[1.0, nan, 3.0],
...                   [4.0, 5.0, nan]])
>>> y = torch.tensor([[2.0, 4.0, 5.0],
...                   [nan, 10.0, 12.0]])
>>> QF.nancorrel(x, y, dim=1)
tensor([1., nan])

>>> # Comparison with regular correl (which would give NaN)
>>> x_with_nan = torch.tensor([1.0, 2.0, nan, 4.0])
>>> y_with_nan = torch.tensor([2.0, 4.0, 6.0, 8.0])
>>> QF.nancorrel(x_with_nan, y_with_nan, dim=0)  # Ignores NaN
tensor(1.)

>>> # With keepdim
>>> x = torch.tensor([[1.0, nan, 3.0],
...                   [4.0, 5.0, 6.0]])
>>> y = torch.tensor([[2.0, 4.0, 6.0],
...                   [8.0, 10.0, 12.0]])
>>> QF.nancorrel(x, y, dim=1, keepdim=True)
tensor([[1.],
        [1.]])