pyPDAF.PDAF.assimilate_hyb3dvar_lestkf

pyPDAF.PDAF.assimilate_hyb3dvar_lestkf()

It is recommended to use pyPDAF.PDAF.localomi_assimilate_hyb3dvar_lestkf() or pyPDAF.PDAF.localomi_assimilate_hyb3dvar_lestkf_nondiagR().

PDAF-OMI modules require fewer user-supplied functions and improved efficiency.

Hybrid 3DEnVar for a single DA step where the background error covariance is hybridised by a static background error covariance, and a flow-dependent background error covariance estimated from ensemble. The 3DVar generates an ensemble mean and the ensemble perturbation is generated by LESTKF in this implementation. This function should be called at each model time step.

The function is a combination of pyPDAF.PDAF.put_state_hyb3dvar_lestkf() and pyPDAF.PDAF.get_state().

User-supplied functions are executed in the following sequence:

1. py__collect_state_pdaf

2. py__prepoststep_state_pdaf

3. py__init_dim_obs_pdaf

4. py__obs_op_pdaf

5. py__init_obs_pdaf

6. The iterative optimisation:

   1. py__cvt_pdaf

   2. py__cvt_ens_pdaf

   3. py__obs_op_lin_pdaf

   4. py__prodRinvA_pdaf

   5. py__obs_op_adj_pdaf

   6. py__cvt_adj_pdaf

   7. py__cvt_adj_ens_pdaf

   8. core DA algorithm

7. py__cvt_pdaf

8. py__cvt_ens_pdaf

9. Perform LESTKF:

   1. py__init_n_domains_p_pdaf

   2. py__init_dim_obs_pdaf

   3. py__obs_op_pdaf (for each ensemble member)

   4. py__init_obs_pdaf (if global adaptive forgetting factor type_forget=1 in pyPDAF.PDAF.init())

   5. py__init_obsvar_pdaf (if global adaptive forgetting factor is used)

   6. loop over each local domain:

      1. py__init_dim_l_pdaf

      2. py__init_dim_obs_l_pdaf

      3. py__g2l_state_pdaf

      4. py__g2l_obs_pdaf (localise mean ensemble in observation space)

      5. py__init_obs_l_pdaf

      6. py__g2l_obs_pdaf (localise each ensemble member in observation space)

      7. py__init_obsvar_l_pdaf (only called if local adaptive forgetting factor type_forget=2 is used)

      8. py__prodRinvA_l_pdaf

      9. core DA algorithm

      10. py__l2g_state_pdaf

10. py__prepoststep_state_pdaf

11. py__distribute_state_pdaf

12. py__next_observation_pdaf

Deprecated since version 1.0.0: This function is replaced by pyPDAF.PDAF.localomi_assimilate_hyb3dvar_lestkf() and pyPDAF.PDAF.localomi_assimilate_hyb3dvar_lestkf_nondiagR()

Parameters:

• py__collect_state_pdaf (Callable[dim_p:int, state_p : ndarray[tuple[dim_p], np.float64]]) –

  Collect state vector from model/any arrays to pdaf arrays

  Callback Parameters

  • dim_pint

    • pe-local state dimension

  • state_pndarray[tuple[dim_p], np.float64]

    • local state vector

  Callback Returns

  • state_pndarray[tuple[dim_p], np.float64]

    • local state vector

• py__distribute_state_pdaf (Callable[dim_p:int, state_p : ndarray[tuple[dim_p], np.float64]]) –

  distribute a state vector from pdaf to the model/any arrays

  Callback Parameters

  • dim_pint

    • PE-local state dimension

  • state_pndarray[tuple[dim_p], np.float64]

    • PE-local state vector

  Callback Returns

  • state_pndarray[tuple[dim_p], np.float64]

    • PE-local state vector

• py__init_dim_obs_pdaf (Callable[step:int, dim_obs_p:int]) –

  The primary purpose of this function is to obtain the dimension of the observation vector. In OMI, in this function, one also sets the properties of obs_f, read the observation vector from files, setting the observation error variance when diagonal observation error covariance matrix is used. The pyPDAF.PDAF.omi_gather_obs function is also called here.

  Callback Parameters

  • stepint

    • current time step

  • dim_obs_pint

    • dimension of observation vector

  Callback Returns

  • dim_obs_pint

    • dimension of observation vector

• py__obs_op_pdaf (Callable[step:int, dim_p:int, dim_obs_p:int, state_p : ndarray[tuple[dim_p], np.float64], m_state_p : ndarray[tuple[dim_obs_p], np.float64]]) –

  Observation operator

  Callback Parameters

  • stepint

    • Current time step

  • dim_pint

    • Size of state vector (local part in case of parallel decomposed state)

  • dim_obs_pint

    • Size of PE-local observation vector

  • state_pndarray[tuple[dim_p], np.float64]

    • Model state vector

  • m_state_pndarray[tuple[dim_obs_p], np.float64]

    • Observed state vector (i.e. the result after applying the observation operator to state_p)

  Callback Returns

  • m_state_pndarray[tuple[dim_obs_p], np.float64]

    • Observed state vector (i.e. the result after applying the observation operator to state_p)

• py__init_obs_pdaf (Callable[step:int, dim_obs_p:int, observation_p : ndarray[tuple[dim_obs_p], np.float64]]) –

  Initialize observation vector

  Callback Parameters

  • stepint

    • Current time step

  • dim_obs_pint

    • Size of the observation vector

  • observation_pndarray[tuple[dim_obs_p], np.float64]

    • Vector of observations

  Callback Returns

  • observation_pndarray[tuple[dim_obs_p], np.float64]

    • Vector of observations

• py__prodRinvA_pdaf (Callable[step:int, dim_obs_p:int, rank:int, obs_p : ndarray[tuple[dim_obs_p], np.float64], A_p : ndarray[tuple[dim_obs_p, rank], np.float64], C_p : ndarray[tuple[dim_obs_p, rank], np.float64]]) –

  Provide product R^-1 A

  Callback Parameters

  • stepint

    • Current time step

  • dim_obs_pint

    • Number of observations at current time step (i.e. the size of the observation vector)

  • rankint

    • Number of the columns in the matrix processes here. This is usually the ensemble size minus one (or the rank of the initial covariance matrix)

  • obs_pndarray[tuple[dim_obs_p], np.float64]

    • Vector of observations

  • A_pndarray[tuple[dim_obs_p, rank], np.float64]

    • Input matrix provided by PDAF

  • C_pndarray[tuple[dim_obs_p, rank], np.float64]

    • Output matrix

  Callback Returns

  • C_pndarray[tuple[dim_obs_p, rank], np.float64]

    • Output matrix

• py__cvt_ens_pdaf (Callable[iter:int, dim_p:int, dim_ens:int, dim_cvec_ens:int, ens_p : ndarray[tuple[dim_p, dim_ens], np.float64], v_p : ndarray[tuple[dim_cvec_ens], np.float64], Vv_p : ndarray[tuple[dim_p], np.float64]]) –

  Apply control vector transform matrix (ensemble)

  Callback Parameters

  • iterint

    • Iteration of optimization

  • dim_pint

    • PE-local dimension of state

  • dim_ensint

    • Ensemble size

  • dim_cvec_ensint

    • Dimension of control vector

  • ens_pndarray[tuple[dim_p, dim_ens], np.float64]

    • PE-local ensemble

  • v_pndarray[tuple[dim_cvec_ens], np.float64]

    • PE-local control vector

  • Vv_pndarray[tuple[dim_p], np.float64]

    • PE-local state increment

  Callback Returns

  • Vv_pndarray[tuple[dim_p], np.float64]

    • PE-local state increment

• py__cvt_adj_ens_pdaf (Callable[iter:int, dim_p:int, dim_ens:int, dim_cv_ens_p:int, ens_p : ndarray[tuple[dim_p, dim_ens], np.float64], Vcv_p : ndarray[tuple[dim_p], np.float64], cv_p : ndarray[tuple[dim_cv_ens_p], np.float64]]) –

  Apply adjoint control vector transform matrix (ensemble var)

  Callback Parameters

  • iterint

    • Iteration of optimization

  • dim_pint

    • PE-local observation dimension

  • dim_ensint

    • Ensemble size

  • dim_cv_ens_pint

    • PE-local dimension of control vector

  • ens_pndarray[tuple[dim_p, dim_ens], np.float64]

    • PE-local ensemble

  • Vcv_pndarray[tuple[dim_p], np.float64]

    • PE-local input vector

  • cv_pndarray[tuple[dim_cv_ens_p], np.float64]

    • PE-local result vector

  Callback Returns

  • cv_pndarray[tuple[dim_cv_ens_p], np.float64]

    • PE-local result vector

• py__cvt_pdaf (Callable[iter:int, dim_p:int, dim_cvec:int, cv_p : ndarray[tuple[dim_cvec], np.float64], Vv_p : ndarray[tuple[dim_p], np.float64]]) –

  Apply control vector transform matrix to control vector

  Callback Parameters

  • iterint

    • Iteration of optimization

  • dim_pint

    • PE-local observation dimension

  • dim_cvecint

    • Dimension of control vector

  • cv_pndarray[tuple[dim_cvec], np.float64]

    • PE-local control vector

  • Vv_pndarray[tuple[dim_p], np.float64]

    • PE-local result vector (state vector increment)

  Callback Returns

  • Vv_pndarray[tuple[dim_p], np.float64]

    • PE-local result vector (state vector increment)

• py__cvt_adj_pdaf (Callable[iter:int, dim_p:int, dim_cvec:int, Vcv_p : ndarray[tuple[dim_p], np.float64], cv_p : ndarray[tuple[dim_cvec], np.float64]]) –

  Apply adjoint control vector transform matrix

  Callback Parameters

  • iterint

    • Iteration of optimization

  • dim_pint

    • PE-local observation dimension

  • dim_cvecint

    • Dimension of control vector

  • Vcv_pndarray[tuple[dim_p], np.float64]

    • PE-local result vector (state vector increment)

  • cv_pndarray[tuple[dim_cvec], np.float64]

    • PE-local control vector

  Callback Returns

  • cv_pndarray[tuple[dim_cvec], np.float64]

    • PE-local control vector

• py__obs_op_lin_pdaf (Callable[step:int, dim_p:int, dim_obs_p:int, state_p : ndarray[tuple[dim_p], np.float64], m_state_p : ndarray[tuple[dim_obs_p], np.float64]]) –

  Linearized observation operator

  Callback Parameters

  • stepint

    • Current time step

  • dim_pint

    • PE-local dimension of state

  • dim_obs_pint

    • Dimension of observed state

  • state_pndarray[tuple[dim_p], np.float64]

    • PE-local model state

  • m_state_pndarray[tuple[dim_obs_p], np.float64]

    • PE-local observed state

  Callback Returns

  • m_state_pndarray[tuple[dim_obs_p], np.float64]

    • PE-local observed state

• py__obs_op_adj_pdaf (Callable[step:int, dim_p:int, dim_obs_p:int, state_p : ndarray[tuple[dim_p], np.float64], m_state_p : ndarray[tuple[dim_obs_p], np.float64]]) –

  Adjoint observation operator

  Callback Parameters

  • stepint

    • Current time step

  • dim_pint

    • PE-local dimension of state

  • dim_obs_pint

    • Dimension of observed state

  • state_pndarray[tuple[dim_p], np.float64]

    • PE-local model state

  • m_state_pndarray[tuple[dim_obs_p], np.float64]

    • PE-local observed state

  Callback Returns

  • state_pndarray[tuple[dim_p], np.float64]

    • PE-local model state

• py__init_dim_obs_f_pdaf (Callable[step:int, dim_obs_p:int]) –

  Initialize dimension of observation vector

  Callback Parameters

  • stepint

    • current time step

  • dim_obs_pint

    • dimension of observation vector

  Callback Returns

  • dim_obs_pint

    • dimension of observation vector

• py__obs_op_f_pdaf (Callable[step:int, dim_p:int, dim_obs_p:int, state_p : ndarray[tuple[dim_p], np.float64], m_state_p : ndarray[tuple[dim_obs_p], np.float64]]) –

  Observation operator

  Callback Parameters

  • stepint

    • Current time step

  • dim_pint

    • Size of state vector (local part in case of parallel decomposed state)

  • dim_obs_pint

    • Size of observation vector

  • state_pndarray[tuple[dim_p], np.float64]

    • Model state vector

  • m_state_pndarray[tuple[dim_obs_p], np.float64]

    • Observed state vector (i.e. the result after applying the observation operator to state_p)

  Callback Returns

  • m_state_pndarray[tuple[dim_obs_p], np.float64]

    • Observed state vector (i.e. the result after applying the observation operator to state_p)

• py__init_obs_f_pdaf (Callable[step:int, dim_obs_f:int, observation_f : ndarray[tuple[dim_obs_f], np.float64]]) –

  Initialize PE-local observation vector

  Callback Parameters

  • stepint

    • Current time step

  • dim_obs_fint

    • Size of the full observation vector

  • observation_fndarray[tuple[dim_obs_f], np.float64]

    • Full vector of observations

  Callback Returns

  • observation_fndarray[tuple[dim_obs_f], np.float64]

    • Full vector of observations

• py__init_obs_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, observation_l : ndarray[tuple[dim_obs_l], np.float64]]) –

  Init. observation vector on local analysis domain

  Callback Parameters

  • domain_pint

    • Index of current local analysis domain

  • stepint

    • Current time step

  • dim_obs_lint

    • Local size of the observation vector

  • observation_lndarray[tuple[dim_obs_l], np.float64]

    • Local vector of observations

  Callback Returns

  • observation_lndarray[tuple[dim_obs_l], np.float64]

    • Local vector of observations

• py__prodRinvA_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, rank:int, obs_l : ndarray[tuple[dim_obs_l], np.float64], A_l : ndarray[tuple[dim_obs_l, rank], np.float64], C_l : ndarray[tuple[dim_obs_l, rank], np.float64]]) –

  Provide product R^-1 A on local analysis domain

  Callback Parameters

  • domain_pint

    • Index of current local analysis domain

  • stepint

    • Current time step

  • dim_obs_lint

    • Number of local observations at current time step (i.e. the size of the local observation vector)

  • rankint

    • Number of the columns in the matrix processes here. This is usually the ensemble size minus one (or the rank of the initial covariance matrix)

  • obs_lndarray[tuple[dim_obs_l], np.float64]

    • Local vector of observations

  • A_lndarray[tuple[dim_obs_l, rank], np.float64]

    • Input matrix provided by PDAF

  • C_lndarray[tuple[dim_obs_l, rank], np.float64]

    • Output matrix

  Callback Returns

  • C_lndarray[tuple[dim_obs_l, rank], np.float64]

    • Output matrix

• py__init_n_domains_p_pdaf (Callable[step:int, n_domains_p:int]) –

  Provide number of local analysis domains

  Callback Parameters

  • stepint

    • current time step

  • n_domains_pint

    • pe-local number of analysis domains

  Callback Returns

  • n_domains_pint

    • pe-local number of analysis domains

• py__init_dim_l_pdaf (Callable[step:int, domain_p:int, dim_l:int]) –

  Init state dimension for local ana. domain

  Callback Parameters

  • stepint

    • current time step

  • domain_pint

    • current local analysis domain

  • dim_lint

    • local state dimension

  Callback Returns

  • dim_lint

    • local state dimension

• py__init_dim_obs_l_pdaf (Callable[domain_p:int, step:int, dim_obs_f:int, dim_obs_l:int]) –

  Initialize dim. of obs. vector for local ana. domain

  Callback Parameters

  • domain_pint

    • index of current local analysis domain

  • stepint

    • current time step

  • dim_obs_fint

    • full dimension of observation vector

  • dim_obs_lint

    • local dimension of observation vector

  Callback Returns

  • dim_obs_lint

    • local dimension of observation vector

• py__g2l_state_pdaf (Callable[step:int, domain_p:int, dim_p:int, state_p : ndarray[tuple[dim_p], np.float64], dim_l:int, state_l : ndarray[tuple[dim_l], np.float64]]) –

  Get state on local ana. domain from full state

  Callback Parameters

  • stepint

    • current time step

  • domain_pint

    • current local analysis domain

  • dim_pint

    • pe-local full state dimension

  • state_pndarray[tuple[dim_p], np.float64]

    • pe-local full state vector

  • dim_lint

    • local state dimension

  • state_lndarray[tuple[dim_l], np.float64]

    • state vector on local analysis domain

  Callback Returns

  • state_lndarray[tuple[dim_l], np.float64]

    • state vector on local analysis domain

• py__l2g_state_pdaf (Callable[step:int, domain_p:int, dim_l:int, state_l : ndarray[tuple[dim_l], np.float64], dim_p:int, state_p : ndarray[tuple[dim_p], np.float64]]) –

  Init full state from state on local analysis domain

  Callback Parameters

  • stepint

    • current time step

  • domain_pint

    • current local analysis domain

  • dim_lint

    • local state dimension

  • state_lndarray[tuple[dim_l], np.float64]

    • state vector on local analysis domain

  • dim_pint

    • pe-local full state dimension

  • state_pndarray[tuple[dim_p], np.float64]

    • pe-local full state vector

  Callback Returns

  • state_pndarray[tuple[dim_p], np.float64]

    • pe-local full state vector

• py__g2l_obs_pdaf (Callable[domain_p:int, step:int, dim_obs_f:int, dim_obs_l:int, mstate_f : ndarray[tuple[dim_p], np.intc], dim_p:int, mstate_l : ndarray[tuple[dim_l], np.intc], dim_l:int]) –

  Restrict full obs. vector to local analysis domain

  Callback Parameters

  • domain_pint

    • Index of current local analysis domain

  • stepint

    • Current time step

  • dim_obs_fint

    • Size of full observation vector for model sub-domain

  • dim_obs_lint

    • Size of observation vector for local analysis domain

  • mstate_fndarray[tuple[dim_p], np.intc]

    • Full observation vector for model sub-domain

  • dim_pint

    • Size of full observation vector for model sub-domain

  • mstate_lndarray[tuple[dim_l], np.intc]

    • Observation vector for local analysis domain

  • dim_lint

    • Size of observation vector for local analysis domain

  Callback Returns

  • mstate_lndarray[tuple[dim_l], np.intc]

    • Observation vector for local analysis domain

• py__init_obsvar_pdaf (Callable[step:int, dim_obs_p:int, obs_p : ndarray[tuple[dim_obs_p], np.float64], meanvar:float]) –

  Initialize mean observation error variance

  Callback Parameters

  • stepint

    • Current time step

  • dim_obs_pint

    • Size of observation vector

  • obs_pndarray[tuple[dim_obs_p], np.float64]

    • Vector of observations

  • meanvarfloat

    • Mean observation error variance

  Callback Returns

  • meanvarfloat

    • Mean observation error variance

• py__init_obsvar_l_pdaf (Callable[domain_p:int, step:int, dim_obs_l:int, obs_l : ndarray[tuple[dim_obs_p], np.float64], dim_obs_p:int, meanvar_l:float]) –

  Initialize local mean observation error variance

  Callback Parameters

  • domain_pint

    • Index of current local analysis domain

  • stepint

    • Current time step

  • dim_obs_lint

    • Local dimension of observation vector

  • obs_lndarray[tuple[dim_obs_p], np.float64]

    • Local observation vector

  • dim_obs_pint

    • Dimension of local observation vector

  • meanvar_lfloat

    • Mean local observation error variance

  Callback Returns

  • meanvar_lfloat

    • Mean local observation error variance

• py__prepoststep_pdaf (Callable[step:int, dim_p:int, dim_ens:int, dim_ens_l:int, dim_obs_p:int, state_p : ndarray[tuple[dim_p], np.float64], uinv : ndarray[tuple[dim_ens-1, dim_ens-1], np.float64], ens_p : ndarray[tuple[dim_p, dim_ens], np.float64], flag:int]) –

  Preprocesse the ensemble before analysis and postprocess the ensemble before distributing to the model for next forecast

  Callback Parameters

  • stepint

    • current time step (negative for call before analysis/preprocessing)

  • dim_pint

    • PE-local state vector dimension

  • dim_ensint

    • number of ensemble members

  • dim_ens_lint

    • number of ensemble members run serially on each model task

  • dim_obs_pint

    • PE-local dimension of observation vector

  • state_pndarray[tuple[dim_p], np.float64]

    • pe-local forecast/analysis state (the array ‘state_p’ is generally not initialised in the case of ESTKF/ETKF/EnKF/SEIK, so it can be used freely here.)

  • uinvndarray[tuple[dim_ens-1, dim_ens-1], np.float64]

    • Inverse of the transformation matrix in ETKF and ESKTF; inverse of matrix formed by right singular vectors of error covariance matrix of ensemble perturbations in SEIK/SEEK. not used in EnKF.

  • ens_pndarray[tuple[dim_p, dim_ens], np.float64]

    • PE-local ensemble

  • flagint

    • pdaf status flag

  Callback Returns

  • state_pndarray[tuple[dim_p], np.float64]

    • pe-local forecast/analysis state (the array ‘state_p’ is generally not initialised in the case of ESTKF/ETKF/EnKF/SEIK, so it can be used freely here.)

  • uinvndarray[tuple[dim_ens-1, dim_ens-1], np.float64]

    • Inverse of the transformation matrix in ETKF and ESKTF; inverse of matrix formed by right singular vectors of error covariance matrix of ensemble perturbations in SEIK/SEEK. not used in EnKF.

  • ens_pndarray[tuple[dim_p, dim_ens], np.float64]

    • PE-local ensemble

• py__next_observation_pdaf (Callable[stepnow:int, nsteps:int, doexit:int, time:float]) –

  Routine to provide number of forecast time steps until next assimilations, model physical time and end of assimilation cycles

  Callback Parameters

  • stepnowint

    • the current time step given by PDAF

  • nstepsint

    • number of forecast time steps until next assimilation; this can also be interpreted as number of assimilation function calls to perform a new assimilation

  • doexitint

    • whether to exit forecasting (1 for exit)

  • timefloat

    • current model (physical) time

  Callback Returns

  • nstepsint

    • number of forecast time steps until next assimilation; this can also be interpreted as number of assimilation function calls to perform a new assimilation

  • doexitint

    • whether to exit forecasting (1 for exit)

  • timefloat

    • current model (physical) time

Returns:

outflag – Status flag

Return type:

int