geonature.core.imports.checks.sql.utils¶

Functions¶

`get_duplicates_query`(imprt, dest_field[, whereclause])
`report_erroneous_rows`(imprt, entity, error_type, ...)	Report erroneous rows in a transient table and update validity of rows based on error level.
`print_transient_table`(imprt[, columns])	Print the content of the transient table for a given import.
`transient_table_to_dataframe`(→ pandas.DataFrame)	Get the content of the transient table for a given import as a pandas DataFrame.

Module Contents¶

geonature.core.imports.checks.sql.utils.get_duplicates_query(imprt, dest_field, whereclause=sa.true())[source]¶

geonature.core.imports.checks.sql.utils.report_erroneous_rows(imprt, entity, error_type, error_column, whereclause, error_comment=None, level_validity_mapping={'ERROR': False})[source]¶

Report erroneous rows in a transient table and update validity of rows based on error level.

This function reports errors found in imported data based on a WHERE clause, and updates the validity column of affected rows if the error level is specified in level_validity_mapping. By default, errors with the level « ERROR » mark the row as invalid.

Parameters¶

imprtTImport

The current import object.

entityEntity

The entity associated with the error. Must be defined if the error level is « ERROR ».

error_typestr

Type of error to report. Must correspond to a record in ImportUserErrorType.

error_columnstr

Name of the column where the error is detected. Can be mapped via imprt.fieldmapping.

whereclausesqlalchemy.sql.elements.ClauseElement

SQL clause defining the rows affected by the error.

error_commentstr, optional

Optional comment to include extra explanation to describe the error in the current import context.

level_validity_mappingdict, optional

Dictionary mapping error levels to validity values: - If the level is not in the dictionary, the row validity remains unchanged. - If the level is present, validity is set according to the associated value:

False: The row is marked as erroneous.

None: The row is marked as should not be imported.

By default, only the level « ERROR » is mapped to False.

Raises¶

AssertionError: If entity is not defined for an error of level « ERROR ».

Examples¶

>>> # Example usage for reporting an "ERROR" level error
>>> report_erroneous_rows(
...     imprt=my_import,
...     entity=my_entity,
...     error_type="MISSING_VALUE",
...     error_column="customer_name",
...     whereclause=(transient_table.c.customer_name == None),
...     error_comment="Customer name missing",
... )

geonature.core.imports.checks.sql.utils.print_transient_table(imprt: geonature.core.imports.models.TImports, columns=None)[source]¶

Print the content of the transient table for a given import.

Parameters¶

imprtTImports: The import to print.
columnslist, optional: The columns to print. If None, all columns are printed.

geonature.core.imports.checks.sql.utils.transient_table_to_dataframe(imprt: geonature.core.imports.models.TImports, columns=None) → pandas.DataFrame[source]¶

Get the content of the transient table for a given import as a pandas DataFrame.

Parameters¶

imprtTImports: The import to get.
columnslist, optional: The columns to include in the DataFrame. If None, all columns are included.

Returns¶

pd.DataFrame: The content of the transient table as a DataFrame.