shrecc.treatment

Functions

add_missing_elements(df, existing_elements, all_elements)

Adds missing elements (rows or columns) to a dataframe.

calculate_Z_cons(filename, L_series, output, Z_indices)

Calculates the consumption matrix Z_cons.

calculate_results(year, n_c, n_p, t_index, Z_net, ...)

Calculates results by inverting matrices over time.

concatenate_results(results, Z)

Concatenates results into a single DataFrame.

create_block_diagonal_matrix(data_list)

Creates a block diagonal matrix from a list of data arrays.

data_processing(data_df, year[, path_to_data])

Processes data, adds missing countries, and correctly divides them between consumption and demand.

load_from_pickle(filename)

Loads an object from a pickle file.

process_matrix(df, operation[, axis])

Applies a specified operation to a dataframe or matrix.

process_results_light(results, filename, n_c)

Processes the results to a lighter format.

save_to_pickle(obj, filename)

Saves an object to a pickle file.

treating_data(year, n_c, n_p, t_index, Z_net, data_dir)

Function called from data_processing. This includes heavy operations, so it's advised to run this part on a server.

Module Contents

shrecc.treatment.add_missing_elements(df, existing_elements, all_elements, axis=0, fill_value=0)[source]

Adds missing elements (rows or columns) to a dataframe.

Parameters:

  • df (pd.DataFrame) – The dataframe to which elements will be added.

  • existing_elements (set) – The set of existing elements in the dataframe.

  • all_elements (set) – The set of all possible elements.

  • axis (int) – The axis along which to add missing elements (0 for index, 1 for columns).

  • fill_value – The value to fill for the missing elements.

Returns:

The dataframe with missing elements added.

Return type:

pd.DataFrame

shrecc.treatment.calculate_Z_cons(filename, L_series, output, Z_indices)[source]

Calculates the consumption matrix Z_cons.

Parameters:

  • year (int) – The selected year.

  • L_series (pd.DataFrame) – The L series DataFrame.

  • output (pd.Series) – The output series.

  • Z_indices (dict) – The Z indices dictionary.

Returns:

The consumption matrix Z_cons.

Return type:

pd.DataFrame

shrecc.treatment.calculate_results(year, n_c, n_p, t_index, Z_net, Z_load, data_dir)[source]

Calculates results by inverting matrices over time.

Parameters:

  • year (int) – The selected year.

  • n_c (int) – The number of countries.

  • n_p (int) – The number of production mix elements.

  • t_index (pd.Index) – The time index.

  • Z_net (pd.DataFrame) – The net consumption dataframe.

  • Z_load (pd.DataFrame) – The load dataframe.

  • data_dir (Path) – location of the data.

Returns:

The results dictionary.

Return type:

dict

shrecc.treatment.concatenate_results(results, Z)[source]

Concatenates results into a single DataFrame.

Parameters:

  • results_light (dict) – The light results dictionary.

  • Z (pd.DataFrame) – The network dataframe.

Returns:

The concatenated DataFrame.

Return type:

pd.DataFrame

shrecc.treatment.create_block_diagonal_matrix(data_list)[source]

Creates a block diagonal matrix from a list of data arrays.

Parameters:

data_list (list) – A list of data arrays to be combined into a block diagonal matrix.

Returns:

A block diagonal matrix containing the input data.

Return type:

pd.DataFrame

shrecc.treatment.data_processing(data_df, year, path_to_data=None)[source]

Processes data, adds missing countries, and correctly divides them between consumption and demand.

Parameters:

  • data_df (pd.DataFrame) – All of the downloaded data for the selected year (from get_data).

  • year (int) – The selected year, e.g., 2023.

  • path_to_data (str or Path) – location of the data.

Returns:

Doesn’t return anything, but saves files to the directory ‘data’.

Return type:

None

shrecc.treatment.load_from_pickle(filename)[source]

Loads an object from a pickle file.

Parameters:

filename (Path) – Path to the filename where the object will be saved.

Returns:

The object loaded from the pickle file.

Return type:

object

shrecc.treatment.process_matrix(df, operation, axis=1, **kwargs)[source]

Applies a specified operation to a dataframe or matrix.

Parameters:

  • df (pd.DataFrame) – The dataframe or matrix to process.

  • operation (str) – The operation to perform (e.g., ‘normalize’, ‘reorder_levels’).

  • axis (int) – The axis along which the operation should be applied, if applicable.

  • kwargs – Additional arguments for specific operations.

Returns:

The processed dataframe or matrix.

Return type:

pd.DataFrame

shrecc.treatment.process_results_light(results, filename, n_c)[source]

Processes the results to a lighter format.

Parameters:

  • results (dict) – The results dictionary.

  • filename (Path) – Path to the filename where the object will be saved.

  • n_c (int) – The number of countries.

Returns:

The light results dictionary.

Return type:

dict

shrecc.treatment.save_to_pickle(obj, filename)[source]

Saves an object to a pickle file.

Parameters:

  • obj – The object to be saved.

  • filename (Path) – Path to the filename where the object will be saved.

Returns:

None

shrecc.treatment.treating_data(year, n_c, n_p, t_index, Z_net, data_dir)[source]

Function called from data_processing. This includes heavy operations, so it’s advised to run this part on a server.

Parameters:

  • year (int) – The selected year, passed from data_processing.

  • n_c (int) – The number of all countries (including missing ones) in the dataframe, passed from data_processing.

  • n_p (int) – The number of production mix elements, passed from data_processing.

  • t_index (pd.Index) – The time index, passed from data_processing.

  • Z_net (pd.DataFrame) – The net consumption dataframe, passed from data_processing.

  • data_dir (Path) – location of the data.

Returns:

Doesn’t return anything; everything gets saved to the directory ‘data’.

Return type:

None