pyPDAF.PDAF.assimilate_lknetf

pyPDAF.PDAF.assimilate_lknetf()

It is recommended to use pyPDAF.PDAF.localomi_assimilate() or pyPDAF.PDAF.localomi_assimilate_lknetf_nondiagR().

PDAF-OMI modules require fewer user-supplied functions and improved efficiency.

A hybridised LETKF and LNETF [1] for a single DA step. The LNETF computes the distribution up to the second moment similar to Kalman filters but using a nonlinear weighting similar to particle filters. This leads to an equal weights assumption for the prior ensemble. The hybridisation with LETKF is expected to lead to improved performance for quasi-Gaussian problems. The function should be called at each model step.

The function is a combination of pyPDAF.PDAF.put_state_lknetf() and pyPDAF.PDAF.get_state().

User-supplied functions are executed in the following sequence:

  1. py__collect_state_pdaf

  2. py__prepoststep_state_pdaf

  3. py__init_n_domains_p_pdaf

  4. py__init_dim_obs_pdaf

  5. py__obs_op_pdaf (for each ensemble member)

  6. py__init_obs_pdaf (if global adaptive forgetting factor type_forget=1 is used in pyPDAF.PDAF.init())

  7. py__init_obsvar_pdaf (if global adaptive forgetting factor is used)

  8. loop over each local domain:

    1. py__init_dim_l_pdaf

    2. py__init_dim_obs_l_pdaf

    3. py__g2l_state_pdaf

    4. py__g2l_obs_pdaf (localise each ensemble member in observation space)

    5. py__init_obs_l_pdaf

    6. py__init_obsvar_l_pdaf (only called if local adaptive forgetting factor type_forget=2 is used)

    7. py__prodRinvA_pdaf

    8. py__likelihood_l_pdaf

    9. core DA algorithm

    10. py__l2g_state_pdaf

  9. py__obs_op_pdaf (only called with HKN and HNK options called for each ensemble member)

  10. py__likelihood_hyb_l_pda

  11. py__init_obsvar_l_pdaf (only called if local adaptive forgetting factor type_forget=2 is used)

  12. py__prodRinvA_hyb_l_pdaf

  13. py__prepoststep_state_pdaf

  14. py__distribute_state_pdaf

  15. py__next_observation_pdaf

Deprecated since version 1.0.0: This function is replaced by pyPDAF.PDAF.localomi_assimilate() and pyPDAF.PDAF.localomi_assimilate_lknetf_nondiagR()

References

Parameters:

  • py__collect_state_pdaf (Callable[dim_p:int, state_p : ndarray[tuple[dim_p], np.float64]]) –

    Collect state vector from model/any arrays to pdaf arrays

    Callback Parameters
    • dim_pint

      • pe-local state dimension

    • state_pndarray[tuple[dim_p], np.float64]

      • local state vector

    Callback Returns
    • state_pndarray[tuple[dim_p], np.float64]

      • local state vector

  • py__distribute_state_pdaf (Callable[dim_p:int, state_p : ndarray[tuple[dim_p], np.float64]]) –

    distribute a state vector from pdaf to the model/any arrays

    Callback Parameters
    • dim_pint

      • PE-local state dimension

    • state_pndarray[tuple[dim_p], np.float64]

      • PE-local state vector

    Callback Returns
    • state_pndarray[tuple[dim_p], np.float64]

      • PE-local state vector

  • py__init_dim_obs_pdaf (Callable[step:int, dim_obs_p:int]) –

    The primary purpose of this function is to obtain the dimension of the observation vector. In OMI, in this function, one also sets the properties of obs_f, read the observation vector from files, setting the observation error variance when diagonal observation error covariance matrix is used. The pyPDAF.PDAF.omi_gather_obs function is also called here.

    Callback Parameters
    • stepint

      • current time step

    • dim_obs_pint

      • dimension of observation vector

    Callback Returns
    • dim_obs_pint

      • dimension of observation vector

  • py__obs_op_pdaf (Callable[step:int, dim_p:int, dim_obs_p:int, state_p : ndarray[tuple[dim_p], np.float64], m_state_p : ndarray[tuple[dim_obs_p], np.float64]]) –

    Observation operator

    Callback Parameters
    • stepint

      • Current time step

    • dim_pint

      • Size of state vector (local part in case of parallel decomposed state)

    • dim_obs_pint

      • Size of PE-local observation vector

    • state_pndarray[tuple[dim_p], np.float64]

      • Model state vector

    • m_state_pndarray[tuple[dim_obs_p], np.float64]

      • Observed state vector (i.e. the result after applying the observation operator to state_p)

    Callback Returns
    • m_state_pndarray[tuple[dim_obs_p], np.float64]

      • Observed state vector (i.e. the result after applying the observation operator to state_p)

  • py__init_obs_pdaf (Callable[step:int, dim_obs_p:int, observation_p : ndarray[tuple[dim_obs_p], np.float64]]) –

    Initialize PE-local observation vector

    Callback Parameters
    • stepint

      • Current time step

    • dim_obs_pint

      • Size of the observation vector

    • observation_pndarray[tuple[dim_obs_p], np.float64]

      • Vector of observations

    Callback Returns
    • observation_pndarray[tuple[dim_obs_p], np.float64]

      • Vector of observations

  • py__init_obs_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, observation_l : ndarray[tuple[dim_obs_l], np.float64]]) –

    Init. observation vector on local analysis domain

    Callback Parameters
    • domain_pint

      • Index of current local analysis domain

    • stepint

      • Current time step

    • dim_obs_lint

      • Local size of the observation vector

    • observation_lndarray[tuple[dim_obs_l], np.float64]

      • Local vector of observations

    Callback Returns
    • observation_lndarray[tuple[dim_obs_l], np.float64]

      • Local vector of observations

  • py__prepoststep_pdaf (Callable[step:int, dim_p:int, dim_ens:int, dim_ens_l:int, dim_obs_p:int, state_p : ndarray[tuple[dim_p], np.float64], uinv : ndarray[tuple[dim_ens-1, dim_ens-1], np.float64], ens_p : ndarray[tuple[dim_p, dim_ens], np.float64], flag:int]) –

    Preprocesse the ensemble before analysis and postprocess the ensemble before distributing to the model for next forecast

    Callback Parameters
    • stepint

      • current time step (negative for call before analysis/preprocessing)

    • dim_pint

      • PE-local state vector dimension

    • dim_ensint

      • number of ensemble members

    • dim_ens_lint

      • number of ensemble members run serially on each model task

    • dim_obs_pint

      • PE-local dimension of observation vector

    • state_pndarray[tuple[dim_p], np.float64]

      • pe-local forecast/analysis state (the array ‘state_p’ is generally not initialised in the case of ESTKF/ETKF/EnKF/SEIK, so it can be used freely here.)

    • uinvndarray[tuple[dim_ens-1, dim_ens-1], np.float64]

      • Inverse of the transformation matrix in ETKF and ESKTF; inverse of matrix formed by right singular vectors of error covariance matrix of ensemble perturbations in SEIK/SEEK. not used in EnKF.

    • ens_pndarray[tuple[dim_p, dim_ens], np.float64]

      • PE-local ensemble

    • flagint

      • pdaf status flag

    Callback Returns
    • state_pndarray[tuple[dim_p], np.float64]

      • pe-local forecast/analysis state (the array ‘state_p’ is generally not initialised in the case of ESTKF/ETKF/EnKF/SEIK, so it can be used freely here.)

    • uinvndarray[tuple[dim_ens-1, dim_ens-1], np.float64]

      • Inverse of the transformation matrix in ETKF and ESKTF; inverse of matrix formed by right singular vectors of error covariance matrix of ensemble perturbations in SEIK/SEEK. not used in EnKF.

    • ens_pndarray[tuple[dim_p, dim_ens], np.float64]

      • PE-local ensemble

  • py__prodRinvA_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, rank:int, obs_l : ndarray[tuple[dim_obs_l], np.float64], A_l : ndarray[tuple[dim_obs_l, rank], np.float64], C_l : ndarray[tuple[dim_obs_l, rank], np.float64]]) –

    Provide product R^-1 A on local analysis domain

    Callback Parameters
    • domain_pint

      • Index of current local analysis domain

    • stepint

      • Current time step

    • dim_obs_lint

      • Number of local observations at current time step (i.e. the size of the local observation vector)

    • rankint

      • Number of the columns in the matrix processes here. This is usually the ensemble size minus one (or the rank of the initial covariance matrix)

    • obs_lndarray[tuple[dim_obs_l], np.float64]

      • Local vector of observations

    • A_lndarray[tuple[dim_obs_l, rank], np.float64]

      • Input matrix provided by PDAF

    • C_lndarray[tuple[dim_obs_l, rank], np.float64]

      • Output matrix

    Callback Returns
    • C_lndarray[tuple[dim_obs_l, rank], np.float64]

      • Output matrix

  • py__prodRinvA_hyb_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, dim_ens:int, obs_l : ndarray[tuple[dim_obs_l], np.float64], gamma:float, A_l : ndarray[tuple[dim_obs_l, dim_ens], np.float64], C_l : ndarray[tuple[dim_obs_l, dim_ens], np.float64]]) –

    Provide product R^-1 A on local analysis domain with hybrid weight

    Callback Parameters
    • domain_pint

      • Index of current local analysis domain

    • stepint

      • Current time step

    • dim_obs_lint

      • Number of local observations at current time step (i.e. the size of the local observation vector)

    • dim_ensint

      • Number of the columns in the matrix processes here. This is usually the ensemble size minus one (or the rank of the initial covariance matrix)

    • obs_lndarray[tuple[dim_obs_l], np.float64]

      • Local vector of observations

    • gammafloat

      • Hybrid weight provided by PDAF

    • A_lndarray[tuple[dim_obs_l, dim_ens], np.float64]

      • Input matrix provided by PDAF

    • C_lndarray[tuple[dim_obs_l, dim_ens], np.float64]

      • Output matrix

    Callback Returns
    • C_lndarray[tuple[dim_obs_l, dim_ens], np.float64]

      • Output matrix

  • py__init_n_domains_p_pdaf (Callable[step:int, n_domains_p:int]) –

    Provide number of local analysis domains

    Callback Parameters
    • stepint

      • current time step

    • n_domains_pint

      • pe-local number of analysis domains

    Callback Returns
    • n_domains_pint

      • pe-local number of analysis domains

  • py__init_dim_l_pdaf (Callable[step:int, domain_p:int, dim_l:int]) –

    Init state dimension for local ana. domain

    Callback Parameters
    • stepint

      • current time step

    • domain_pint

      • current local analysis domain

    • dim_lint

      • local state dimension

    Callback Returns
    • dim_lint

      • local state dimension

  • py__init_dim_obs_l_pdaf (Callable[domain_p:int, step:int, dim_obs_f:int, dim_obs_l:int]) –

    Initialize dim. of obs. vector for local ana. domain

    Callback Parameters
    • domain_pint

      • index of current local analysis domain

    • stepint

      • current time step

    • dim_obs_fint

      • full dimension of observation vector

    • dim_obs_lint

      • local dimension of observation vector

    Callback Returns
    • dim_obs_lint

      • local dimension of observation vector

  • py__g2l_state_pdaf (Callable[step:int, domain_p:int, dim_p:int, state_p : ndarray[tuple[dim_p], np.float64], dim_l:int, state_l : ndarray[tuple[dim_l], np.float64]]) –

    Get state on local ana. domain from full state

    Callback Parameters
    • stepint

      • current time step

    • domain_pint

      • current local analysis domain

    • dim_pint

      • pe-local full state dimension

    • state_pndarray[tuple[dim_p], np.float64]

      • pe-local full state vector

    • dim_lint

      • local state dimension

    • state_lndarray[tuple[dim_l], np.float64]

      • state vector on local analysis domain

    Callback Returns
    • state_lndarray[tuple[dim_l], np.float64]

      • state vector on local analysis domain

  • py__l2g_state_pdaf (Callable[step:int, domain_p:int, dim_l:int, state_l : ndarray[tuple[dim_l], np.float64], dim_p:int, state_p : ndarray[tuple[dim_p], np.float64]]) –

    Init full state from state on local analysis domain

    Callback Parameters
    • stepint

      • current time step

    • domain_pint

      • current local analysis domain

    • dim_lint

      • local state dimension

    • state_lndarray[tuple[dim_l], np.float64]

      • state vector on local analysis domain

    • dim_pint

      • pe-local full state dimension

    • state_pndarray[tuple[dim_p], np.float64]

      • pe-local full state vector

    Callback Returns
    • state_pndarray[tuple[dim_p], np.float64]

      • pe-local full state vector

  • py__g2l_obs_pdaf (Callable[domain_p:int, step:int, dim_obs_f:int, dim_obs_l:int, mstate_f : ndarray[tuple[dim_p], np.intc], dim_p:int, mstate_l : ndarray[tuple[dim_l], np.intc], dim_l:int]) –

    Restrict full obs. vector to local analysis domain

    Callback Parameters
    • domain_pint

      • Index of current local analysis domain

    • stepint

      • Current time step

    • dim_obs_fint

      • Size of full observation vector for model sub-domain

    • dim_obs_lint

      • Size of observation vector for local analysis domain

    • mstate_fndarray[tuple[dim_p], np.intc]

      • Full observation vector for model sub-domain

    • dim_pint

      • Size of full observation vector for model sub-domain

    • mstate_lndarray[tuple[dim_l], np.intc]

      • Observation vector for local analysis domain

    • dim_lint

      • Size of observation vector for local analysis domain

    Callback Returns
    • mstate_lndarray[tuple[dim_l], np.intc]

      • Observation vector for local analysis domain

  • py__init_obsvar_pdaf (Callable[step:int, dim_obs_p:int, obs_p : ndarray[tuple[dim_obs_p], np.float64], meanvar:float]) –

    Initialize mean observation error variance

    Callback Parameters
    • stepint

      • Current time step

    • dim_obs_pint

      • Size of observation vector

    • obs_pndarray[tuple[dim_obs_p], np.float64]

      • Vector of observations

    • meanvarfloat

      • Mean observation error variance

    Callback Returns
    • meanvarfloat

      • Mean observation error variance

  • py__init_obsvar_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, obs_l : ndarray[tuple[dim_obs_p], np.float64], dim_obs_p:int, meanvar_l:float]) –

    Initialize local mean observation error variance

    Callback Parameters
    • domain_pint

      • Index of current local analysis domain

    • stepint

      • Current time step

    • dim_obs_lint

      • Local dimension of observation vector

    • obs_lndarray[tuple[dim_obs_p], np.float64]

      • Local observation vector

    • dim_obs_pint

      • Dimension of local observation vector

    • meanvar_lfloat

      • Mean local observation error variance

    Callback Returns
    • meanvar_lfloat

      • Mean local observation error variance

  • py__likelihood_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, obs_l : ndarray[tuple[dim_obs_l], np.float64], resid_l : ndarray[tuple[dim_obs_l], np.float64], likely_l:float]) –

    Compute likelihood

    Callback Parameters
    • domain_pint

      • Index of current local analysis domain

    • stepint

      • Current time step

    • dim_obs_lint

      • Number of local observations at current time step (i.e. the size of the local observation vector)

    • obs_lndarray[tuple[dim_obs_l], np.float64]

      • Local vector of observations

    • resid_lndarray[tuple[dim_obs_l], np.float64]

      • nput vector holding the local residual

    • likely_lfloat

      • Output value of the local likelihood

    Callback Returns
    • likely_lfloat

      • Output value of the local likelihood

  • py__likelihood_hyb_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, obs_l : ndarray[tuple[dim_obs_l], np.float64], resid_l : ndarray[tuple[dim_obs_l], np.float64], gamma:float, likely_l:float]) –

    Compute likelihood with hybrid weight

    Callback Parameters
    • domain_pint

      • Index of current local analysis domain

    • stepint

      • Current time step

    • dim_obs_lint

      • Number of local observations at current time step (i.e. the size of the local observation vector)

    • obs_lndarray[tuple[dim_obs_l], np.float64]

      • Local vector of observations

    • resid_lndarray[tuple[dim_obs_l], np.float64]

      • Input vector holding the local residual

    • gammafloat

      • Hybrid weight provided by PDAF

    • likely_lfloat

      • Output value of the local likelihood

    Callback Returns
    • likely_lfloat

      • Output value of the local likelihood

  • py__next_observation_pdaf (Callable[stepnow:int, nsteps:int, doexit:int, time:float]) –

    Routine to provide number of forecast time steps until next assimilations, model physical time and end of assimilation cycles

    Callback Parameters
    • stepnowint

      • the current time step given by PDAF

    • nstepsint

      • number of forecast time steps until next assimilation; this can also be interpreted as number of assimilation function calls to perform a new assimilation

    • doexitint

      • whether to exit forecasting (1 for exit)

    • timefloat

      • current model (physical) time

    Callback Returns
    • nstepsint

      • number of forecast time steps until next assimilation; this can also be interpreted as number of assimilation function calls to perform a new assimilation

    • doexitint

      • whether to exit forecasting (1 for exit)

    • timefloat

      • current model (physical) time

Returns:

flag – Status flag

Return type:

int