egse.hk

cgse-common

This code is part of the cgse-common package.

egse.hk ¶

Classes:

Name	Description
`HKError`	An HK-specific error.
`TmDictionaryColumns`	Enumeration of the relevant columns in the TM dictionary spreadsheet.

Functions:

Name	Description
`convert_hk_names`	Converts the names of the HK parameters in the given dictionary.
`get_hk_info`	Read the HK spreadsheet and extract information for the given HK parameter.
`get_housekeeping`	Returns the timestamp(s) and housekeeping value(s) for the housekeeping parameter with the given name.
`read_conversion_dict`	Read the HK spreadsheet and compose conversion dictionary for HK names.

HKError ¶

Bases: Exception

An HK-specific error.

TmDictionaryColumns ¶

Bases: str, Enum

Enumeration of the relevant columns in the TM dictionary spreadsheet.

The relevant columns are:

Name	Description
STORAGE_MNEMONIC	Column with the storage mnemonic of the process that generated the HK
CORRECT_HK_NAMES	Column with the correct HK names (that can be used in `get_housekeeping`)
ORIGINAL_EGSE_HK_NAMES	Column with the names that were originally used in `get_housekeeping` the device protocol
SYNOPTICS_ORIGIN	Column with the origin of the synoptics at the current site
TIMESTAMP_NAMES	Column with the name of the timestamps
DASHBOARD	Column with the name of the dashboard that holds the HK metric

convert_hk_names ¶

convert_hk_names(original_hk, conversion_dict)

Converts the names of the HK parameters in the given dictionary.

The names/keys in the given dictionary of HK parameters (original_hk) are replaced by the names from the given conversion dictionary. The original dictionary is left unchanged, a new dictionary is returned.

Parameters:

Name	Type	Description	Default
`original_hk`	`dict`	Original dictionary of HK parameters.	required
`conversion_dict`	`dict`	Dictionary with the original HK names as keys and the new HK names as values.	required

Returns:

Type	Description
`dict`	A new dictionary of HK parameters with the corrected HK names.

get_hk_info ¶

get_hk_info(hk_name, setup=None)

Read the HK spreadsheet and extract information for the given HK parameter.

The spreadsheet contains the following information

storage mnemonic of the component that generates the HK
original HK name
HK name with the correct prefix
name of the column (in the HK file) with the corresponding timestamp

Parameters:

Name	Type	Description	Default
`hk_name`	`str`	Name of the HK parameter.	required
`setup`	`Optional[Setup]`	the Setup to use, if None, the setup will be loaded from the configuration manager.	`None`

Returns:

Type	Description
`tuple`	A tuple where (1) the first field contains the storage mnemonic of the component that generates the given HK parameter, and (2) the second field contains the name of the column in the HK file with the corresponding timestamp.

Raises:

Type	Description
`HKError`	when `hk_name` is not known.

get_housekeeping ¶

get_housekeeping(
    hk_name,
    obsid=None,
    od=None,
    time_window=None,
    data_dir=None,
    setup=None,
)

Returns the timestamp(s) and housekeeping value(s) for the housekeeping parameter with the given name.

It is possible to indicate for which obsid or which OD the housekeeping is to be returned. If neither of them is specified, the latest daily files are used.

When the time window has not been specified, the last timestamp and housekeeping value will be returned for the given OD. It is possible that a component stopped writing HK for some reason, and that the last housekeeping value is older than you would want. It is therefore important to inspect the corresponding timestamp.

When the time window has been specified, the relevant housekeeping will be read:

determine the sampling rate (compare the timestamps for the last 2 lines in the housekeeping file);
determine how many samples we need to read (starting at the back);
read the required number of line, starting at the back;
for each of the read lines, append the timestamp and HK value to the arrays that will be returned

Parameters:

Name	Type	Description	Default
`hk_name`	`str`	Name of the housekeeping parameter.	required
`obsid`	`Union[ObservationIdentifier, str, int]`	Observation identifier. This can be an ObservationIdentifier object, a string in format TEST_LAB or TEST_LAB_SETUP, or an integer representing the test ID; optional.	`None`
`od`	`str`	Identifier for the OD (YYYYMMDD); optional.	`None`
`time_window`	`int`	Length of the time window over which to retrieve the housekeeping [s]. The time window ends at the moment this method is called. If not given, the latest housekeeping value is returned.	`None`
`data_dir`	`str`	Folder (with sub-folders /daily and /obs) in which the HK files are stored. If this argument is not provided, the data_dir will be determined from the environment variable `${PROJECT}_DATA_STORAGE_LOCATION`.	`None`
`setup`	`Optional[Setup]`	the Setup to use, if None, the setup will be loaded from the configuration manager	`None`

Raises:

Type	Description
`HKError`	when one of the following problems occur * no obsid nor an od argument was provided * no HK measures were found for the given parameter and obsid/od

Returns:

Type	Description
`tuple \| ndarray`	A tuple or an array based on the time window: - If the time window has not been specified: the most recent timestamp and housekeeping value. - If the time window has been specified: an array of timestamps and an array of housekeeping values, belonging to the specified time window.

get_housekeeping_names ¶

get_housekeeping_names(
    name_filter=None, device_filter=None, setup=None
)

Return HK names, storage mnemonic, and description.

The TM dictionary is read into a Pandas DataFrame. If a device filter is given, only the rows pertaining to the given storage mnemonic are kept. If a name filter is given, only keep the rows for which the HK parameter name contains the given name filter.

The result is as a Pandas DataFrame with the following columns:

"CAM EGSE mnemonic": Name of the HK parameter;
"Storage mnemonic": Storage mnemonic of the device producing the HK;
"Description": Description of the HK parameter.

Synopsis:

get_housekeeping_names(name_filter="RAW", device_filter="N-FEE-HK")
get_housekeeping_names(name_filter="RAW", device_filter="N-FEE-HK", setup=setup)

Parameters:

Name	Type	Description	Default
`name_filter`	`str`	Filter the HK dataframe, based on (a part of) the name of the HK parameter(s)	`None`
`device_filter`	`str`	Filter the HK dataframe, based on the given storage mnemonic	`None`
`setup`	`Setup`	the Setup to be used, if None, the setup will be loaded from the configuration manager.	`None`

Returns:

Type	Description
`DataFrame`	Pandas DataFrame with the HK name, storage mnemonic, and description of the HK parameters that pass the given filter.

get_indices ¶

get_indices(filename, hk_name, timestamp_name)

Return the column number of the timestamp and given HK parameter in the given HK file.

Parameters:

Name	Type	Description	Default
`filename`	`str`	HK file in which to look for the given HK parameter.	required
`hk_name`	`str`	Name of the HK parameter.	required
`timestamp_name`	`str`	Name of the corresponding timestamp.	required

Returns:

Type	Description
`tuple`	A tuple with: Index of the column with the timestamps. Index of the column with the HK parameter with the given name.

get_last_non_empty ¶

get_last_non_empty(filename, timestamp_index, hk_index)

Return the timestamp and HK value for last real value.

Parameters:

Name	Type	Description	Default
`filename`	`str`	HK file in which to look for the given HK parameter.	required
`timestamp_index`	`int`	Index of the column with the timestamps.	required
`hk_index`	`int`	Index of the column with the HK parameter with the given name.	required

Returns:

Type	Description
`tuple`	The timestamp and HK value with the last real value.

get_sampling_rate ¶

get_sampling_rate(filename, timestamp_name)

Return the sampling rate for the HK file with the given name [s].

The sampling rate is determined as the difference between the timestamps of the last two lines of the HK file.

Parameters:

Name	Type	Description	Default
`filename`	`str`	Name of the HK file. We do not check explicitly whether this file exists.	required
`timestamp_name`	`str`	the name of the column containing the timestamp	required

Returns:

Type	Description
`float`	Sampling rate for the HK file with the given name [s].

get_storage_mnemonics ¶

get_storage_mnemonics(setup=None)

Return the list of the storage mnemonics from the TM dictionary.

Parameters:

Name	Type	Description	Default
`setup`	`Setup`	the Setup to be used, if None, the setup will be loaded from the configuration manager.	`None`

Returns:

Type	Description
`list`	List of the storage mnemonics from the TM dictionary.

read_conversion_dict ¶

read_conversion_dict(
    storage_mnemonic, use_site=False, setup=None
)

Read the HK spreadsheet and compose conversion dictionary for HK names.

The spreadsheet contains the following information:

- storage mnemonic of the component that generates the HK
- original HK name (as is comes from the device itself)
- HK name with the correct prefix
- name of the column (in the HK file) with the corresponding timestamp

Parameters:

Name	Type	Description	Default
`storage_mnemonic`	`str`	Storage mnemonic of the component for which to compose the conversion dictionary	required
`use_site`	`bool`	Indicate whether the prefixes of the new HK names are TH-specific	`False`
`setup`	`Optional[Setup]`	the Setup to be used, if None, the setup will be loaded from the configuration manager.	`None`

Returns:

Type	Description
`dict`	Dictionary with the original HK names as keys and the converted HK names as values.