###################
Configuration files
###################

Format
======
pygeodyn configuration files are simply text files with ``.conf`` extension where each line describes an input parameter of the computation. The format of these lines is quite simple:

.. code-block:: none

    name_of_the_parameter type value

In words: first, the name of the parameter (see below for the list), then the type (*int* for an integer, *str* for a string...) and finally the value, all separated by a space.

If a parameter is not set in the configuration file, the default value will be used (warning: some parameters are mandatory).


List of possible parameters
===========================
- Lb
    Maximal spherical harmonic degree of the magnetic field.

    - **Type**: *int*
    - **Default value**: None (must be given)

- Lu
    Maximal spherical harmonic degree of the core flow.

    - **Type**: *int*
    - **Default value**: None (must be given)

- Lsv
    Maximal spherical harmonic degree of the secular variation.

    - **Type**: *int*
    - **Default value**: None (must be given)

- Ly
    Maximal spherical harmonic degree of the constraint on the shear. Only used if ``-shear`` is 1.

    - **Type**: *int*
    - **Default value**: None (must be given)

- Nth_legendre
    Number of angles to use for the evaluation of Legendre polynomials.

    Reducing this value can speed up computations but at the cost of numerical precision. It is therefore advised to leave it at its default value.

    - **Type**: *int*
    - **Default value**: 64

- t_start_analysis
    Starting analysis time for the algorithm.

    - **Type**: *float* (decimal year)
    - **Default value**: None (must be given)

- t_end_analysis
    Ending analysis time for the algorithm.

    - **Type**: *float* (decimal year)
    - **Default value**: None (must be given)

- N_dta_start_forecast
    Starting forecast N x dta before ``t_start_analysis``.
    N must be greater or equal to 1.

    - **Type**: *int*
    - **Default value**: 1

- N_dta_end_forecast
    Ending forecast N x dta after ``t_end_analysis``.

    - **Type**: *int*
    - **Default value**: 1 if AR3, 0 if AR1 or diag

- dt_f
    Timestep between two forecast steps. 

    - **Type**: *float* (decimal year)
    - **Default value**: None (must be given)

- dt_a_f_ratio
    Number of forecasts per analysis. 

    dt_a_f_ratio = dt_analysis / dt_forecast

    - **Type**: *int*
    - **Default value**: None (must be given)

- AR_type
    Type of Auto-Regressive (AR) process to use in forecasts. The possible values are *diag* (for diagonal AR-1), *AR1* (for dense AR-1) and *AR3* (for dense AR-3)

    - **Type**: *str*
    - **Default value**: *diag*

- combined_U_ER_forecast
    Controls if we consider U and ER independant (0) or not (1) in the AR process (forecast).

    We advise to keep U and ER independant (0) since combining U and ER for the AR process gives less stable results.

    - **Type**: *int*
    - **Default value**: *0*

- TauU
    Time constant for the core flow diagonal AR-1, given in number of months or years. Only used if AR_type was set to *diag*.

    - **Type**: *months* or *years*
    - **Default value**: *30 years*

- TauE
    Time constant for the subgrid errors diagonal AR-1, given in number of months or years. Only used if AR_type was set to *diag*.

    - **Type**: *months* or *years*
    - **Default value**: *10 years*

- TauG
    Galvanic time, given in number of months or years. Only used if ``-shear`` is 1.

    - **Type**: *months* or *years*
    - **Default value**: *20 years*

- dt_sampling
    Apply a subsampling to the prior according to dt_sampling. Used only for the AR1 process.

    - **Type**: float (decimal year)
    - **Default value**: 5

- dt_smoothing
    Duration of a blackman smoothing window applied to the prior. Used only for the AR3 process.

    - **Type**: float (decimal year)
    - **Default value**: 3.2

- N_pca_u
    The algorithm can work with a Principal Component Analysis (PCA) of the core flow rather than on the core flow itself. It reduces the number of coefficients to compute and therefore the computation time. This parameter sets the number of components to use.

    If in doubt, leave this untouched.

    - **Type**: *int*
    - **Default value**: 0 (no PCA performed)

- pca_norm
    Normalisation to use when performing the PCA. Only used if N_pca is different from 0. The supported values are *None* and *energy*. With *None*, the PCA is performed directly on the Gauss coefficients of the core flow. With *energy*, the PCA is performed so that it is normalised with respect to energy (details in Gillet et al. 2019). This has only a minimal impact on the final result.

    - **Type**: *str*
    - **Default value**: *None* (as a string)

- core_state_init
    Method to use to generate the initial core state (at the first time). The possible values are *normal*, and *from_file*.

    With *normal*, the inital core state values for B, U, E, SV are initialised from priors. The realisations are generated from a normal draw around the prior background values with *normal*.

    With *from_file*, the complete initial core state is the core state of a given file at a given date.

    - **Type**: *str*
    - **Default value**: *normal*

  out_analysis
    Add the analysed states to the output hdf5 file. The possible values are *0* (not output) and *1* (output).

    - **Type**: *int*
    - **Default value**: *1*

  out_forecast
    Add the forecast states to the output hdf5 file. The possible values are *0* (not output) and *1* (output).

    - **Type**: *int*
    - **Default value**: *0*

- out_computed
    Add the computed states (analysis and forecast) to the output hdf5 file. The possible values are *0* (not output) and *1* (output).

    - **Type**: *int*
    - **Default value**: *0*

  out_misfits
    Add the misfits to the output hdf5 file. The possible values are *0* (not output) and *1* (output).

    - **Type**: *int*
    - **Default value**: *0*

- out_format
    Controls the format of the output hdf5 file. The possible values are *float32* (single precision) and *float64* (double precision).

    - **Type**: *str*
    - **Default value**: *float32*

- init_file
    File to use for the core state initialisation. Only used if core_state_init is set to *from_file*.

    Must be a valid path pointing towards a valid pygeodyn file.

    - **Type**: *str*
    - **Default value**: None (must be given if core_state_init is set to *from_file*)

- init_date
    Date to use for the core state initialisation. Only used if core_state_init is set to *from_file*.

    Must be an analysis decimal year time contained in the pygeodyn file given in init_file.

    - **Type**: *float*
    - **Default value**: None (must be given if core_state_init is set to *from_file*)

- prior_dir
    Directory containing the prior data. 
    
    This directory should contain magnetic field, core flow and subgrid errors data that will be used to derive the correlation matrices used in the data assimiliation steps.

    The given path can be absolute or relative to the configuration file.

    - **Type**: *str*
    - **Default value**: ./data/priors/71path

- prior_dir_shear
    Directory containing the prior data for the shear computation.  Only used if ``-shear`` is 1.
    
    This directory should contain the core flow and the error of representativeness for the shear that will be used to derive the correlation matrices used in the shear computation.

    The given path can be absolute or relative to the configuration file.

    - **Type**: *str*
    - **Default value**: ``prior_dir``

- prior_type
    Type of the prior data contained in ``prior_dir``. Must correspond to a reading method implemented in pygeodyn/inout/priors.

    The supplied prior types are *0path* and *50path*, *71path* and *100path* that are results of free runs of the eponymous geodynamo simulations by J. Aubert ([AFF13]_ and [AGF17]_).
 
    - **Type**: *str*
    - **Default value**: *71path*

- prior_type_shear
    Type of the prior data contained in ``prior_dir_shear``. Only used if ``-shear`` is 1.
    
    Must correspond to a reading method implemented in pygeodyn/inout/priors.

    The supplied prior types are *0path* and *50path*, *71path* and *100path* that are results of free runs of the eponymous geodynamo simulations by J. Aubert ([AFF13]_ and [AGF17]_).
 
    - **Type**: *str*
    - **Default value**: ``prior_type``

- obs_dir
    Directory containing the observation data. This directory should contain magnetic and secular variation data (measurements and errors) that will be used in the estimations performed in analysis steps.

    The given path can be absolute or relative to the configuration file.

    - **Type**: *str*
    - **Default value**: ./data/observations/COVOBS-x2_maglat

- obs_type
    Type of the observation data contained in obs_dir. Must correspond to a reading method implemented in pygeodyn/inout/observations.

    The supplied observation types can be
    - *COVOBS*, a stochastic model from [GJF13]_ (extended in [GBF15]_),
    - *COVOBS_hdf5*, a more recent version of the stochastic model COVOBS, 
    - *chaos_hdf5*, an estimation of earth magnetic field using ground observatory and satellite data (SWARM and CHAMP missions)
    - *GO_VO*, measurements of Ground Observatories and preprocessed satellite data (CHAMP, Swarm, Oersted, Cryosat and composite) under the form of Virtual Observatories spread all around the globe ([BHF18]_, [Hammer18]_).

    - **Type**: *str*
    - **Default value**: *COVOBS_hdf5*

- SW_err
    Used if obs_type is *GO_VO*.
    Specify if we consider SWARM diagonal or dense error matrix R.
    ONLY APPLY IF SWARM 4MONTHS DATA!!!

    - **Type**: *str*
    - **Default value**: *diag*

- remove_spurious
	The graphical Lasso parameter. 
    
    Dictate how the covariance matrix in the Kalman gain will be approximated. Big values will approximate the covariance matrix by its diagonal or almost diagonal and small values will result in a covariance matrix still close to the empirical estimate of the covariance.
	In practise a parameter bigger than 1 will always return a diagonal matrix. 
    
    For the AR1, a good value can be estimated as :math:`\frac{35}{N}`, with N the number of realisations [IGF23]_.
    
    Default value will result in a diagonal covariance matrix.
    If it does not converge, try a larger value.

    - **Type**: *float*
    - **Default value**: *np.inf*

- remove_spurious_shear_u
	The graphical Lasso parameter. Only used if ``-shear`` is 1. 
    Dictate how the core flow covariance matrix will be approximated. Big values will approximate the covariance matrix by its diagonal or almost diagonal and small values will result in a covariance matrix still close to the empirical estimate of the covariance.
	In practise a parameter bigger than 1 will always return a diagonal matrix.
    Default value will result in a diagonal covariance matrix.
    If it does not converge, try a larger value.
    
    - **Type**: *float*
    - **Default value**: *np.inf*

- remove_spurious_shear_err
	The graphical Lasso parameter. Only used if ``-shear`` is 1.
    Dictate how the error of representativeness covariance matrix will be approximated. Big values will approximate the covariance matrix by its diagonal or almost diagonal and small values will result in a covariance matrix still close to the empirical estimate of the covariance.
	In practise a parameter bigger than 1 will always return a diagonal matrix.
    Default value will result in a diagonal covariance matrix.
    If it does not converge, try a larger value.

    - **Type**: *float*
    - **Default value**: *np.inf*

- obs_select
	Only if obs_type is equal to GO_VO. Dictionnary that defines the selection of ground and virtual observatories data used in the algorithm. For instance, to use the secular variation (*SV*) of the ground obsertories and both the magnetic field (*MF*) and the *SV* for the CHAMP satellite, define "obs_select dict {'GROUND':'SV', 'CHAMP':('MF', 'SV')}". Possible observatories are *GROUND*, *CHAMP*, *SWARM*, *OERSTED*, *CRYOSAT* and *COMPOSITE*, and corresponding files name should start with *GO*, *CH*, *SW*, *OR*, *CR* or *CO*.

    Default value will search for all available data in folder except *COMPOSITE*

    - **Type**: *dict*
    - **Default value**: *all*

- discard_high_lat
	All observations that are located at a geomagnetic latitude higher than the value of discard_high_lat will not be taken into account. Inclination of the dipole is computed using the first three Gauss coefficients from Chaos 7 data at the closest time from the observations. The default value is 0 which means no observations will be discarded. 

    - **Type**: *float*
    - **Default value**: *0*

- obs_mod_errors_dir
    .. warning::

        This has not been throughly tested ! Use only if you really need to and you know what you are doing !

    Directory containing modelisation errors for the magnetic field (*MF.err*) and the secular variation (*SV.err*). The files must exist and have store errors for *Nb* coefficients for MF and *Nsv* for SV.

    This allows to add constant errors to observations of type *COVOBS_hdf5*.

    - **Type**: *str*
    - **Default value**: *None*


References
==========

.. [AFF13] Aubert, J., Finlay, C. C. & Fournier, A. Bottom-up control of geomagnetic secular variation by the Earth’s inner core, *Nature*, 502, 219–223, 2013. doi:10.1038/nature12574
.. [AGF17] Aubert, J., Gastine, T. & Fournier, A. Spherical convective dynamos in the rapidly rotating asymptotic regime, *Journal of Fluid Mechanics*, 813, 558–593, 2017. doi:10.1017/jfm.2016.789
.. [BGA17] Barrois, O., Gillet, N., & Aubert, J. Contributions to the geomagnetic secular variation from a reanalysis of core surface dynamics, *Geophysical Journal International*, 211(1), 50-68, 2017. doi:10.1093/gji/ggx280
.. [BHF18] Barrois, O., Hammer, M. D., Finlay, C. C., Martin, Y. & Gillet, N. Assimilation of ground and satellite magnetic measurements: inference of core surface magnetic and velocity field changes, *Geophysical Journal International*, 2018. doi:10.1093/gji/ggy297
.. [GBF15] Gillet, N., Barrois, O. & Finlay, C. C. Stochastic forecasting of the geomagnetic field from the COV-OBS.x1 geomagnetic field model, and candidate models for IGRF-12, *Earth, Planets and Space*, 67, 2015. doi:10.1186/s40623-015-0225-z
.. [GHA19] Gillet, N., Huder, L., & Aubert, J. A reduced stochastic model of core surface dynamics based on geodynamo simulations, *Geophysical Journal International*, 219(1), 522-539, 2019. doi:10.1093/gji/ggz313
.. [GJF13] Gillet, N., Jault, D., Finlay, C. C. & Olsen, N. Stochastic modeling of the Earth’s magnetic field: Inversion for covariances over the observatory era, *Geochemistry, Geophysics, Geosystems*, 14, 766–786, 2013. doi:10.1002/ggge.20041
.. [Hammer18] Hammer, M. D. Local Estimation of the Earth’s Core Magnetic Field. (Technical University of Denmark (DTU), 2018).
.. [IGF23] Istas, M., Gillet, N., Finlay, C. C., Hammer, M. D., & Huder, L. Transient core surface dynamics from ground and satellite geomagnetic data, *Geophysical Journal International*, 233, 1890–1915, 2023. doi:10.1093/gji/ggad039