EGMStoolkit.functions.egmsdatatools

module documentation

(source)

Module to add some functions post-process the data from EGMS, with `EGMStoolkit`

The module adds some functions, required by EGMStoolkit to post-process the EGMS data.

(From EGMStoolkit package)

Changelog:

0.3.0: Improve the cross-platform compatibility, Oct. 2025, Alexis Hrysiewicz
0.2.16: Bug fix in the datagridding (duplication of -of option), Jul. 2025, Alexis Hrysiewicz
0.2.14: SEcond fix regarding the location of GTiff coordinates (i.e., -co AREA_OR_POINT=Point), end-Jan 2025, Alexis Hrysiewicz
0.2.13: Fix regarding the location of GTiff coordinates (i.e., -co AREA_OR_POINT=Point), Jan 2025, Alexis Hrysiewicz
0.2.11: Fix regarding the input and output directory for data gridding, Aug. 2024, Alexis Hrysiewicz
0.2.9: A correction of typo., Apr. 2024, Alexis Hrysiewicz
0.2.6: Bug fixes regarding the windows system + GDAL version (some information), Feb. 2024, Alexis Hrysiewicz
0.2.5: Add the interpolation processing for the .vrt file + optional function arguments for "duplicate" point and vrt files, Feb. 2024, Alexis Hrysiewicz
0.2.4: Add the possibility to merge the L3 .csv file into a .vrt file and fix the problem with the L2 datasets, Feb. 2024, Alexis Hrysiewicz
0.2.3: Add the possibility to merge the L2 .csv file into a .vrt file (but can fail), Feb. 2024, Alexis Hrysiewicz
0.2.2: Optimisation of clipping based on ogr2ogr, Feb. 2024, Alexis Hrysiewicz
0.2.1: Remove the duplicate points for L2 datasets, Feb. 2024, Alexis Hrysiewicz
0.2.0: Script structuring, Jan. 2024, Alexis Hrysiewicz
0.1.0: Initial version, Nov. 2023

Function	`convertcsv`	Interpolation of the data into a .tif raster
Function	`dataclipping`	Clip the datasets
Function	`datagridding`	Interpolation of the data into a .tif raster
Function	`datamergingcsv`	Merge the datasets in csv format
Function	`datamergingtiff`	Merge the datasets in tiff format (only for L3 level)
Function	`filemergingcsv`	Sub-function to merge the data in .csv format
Function	`filemergingcsvvrt`	Sub-function to merge the data in .csv format by using the .vrt format (only for L2a/L2b datasets and constant headers)
Function	`filemergingtiff`	Sub-function to merge the geotiff data
Function	`listtodictmerged`	Sub-function to conver the list of a merged dictionary
Function	`removerawdata`	Delete the raw dataset(s) from EGMS
Variable	`keygdalprog`	Undocumented
Variable	`latlon_to_meter`	Pyproj class Lat/lon (epsg:4326) to meter (epsg:3035)
Variable	`meter_to_latlon`	Pyproj class meter (epsg:3035) to Lat/lon (epsg:4326)
Variable	`source_crs`	Source CRS.
Variable	`target_crs`	Target CRS.

def convertcsv(outputdir: Optional[str] = '.'+os.sep+'Output', inputdir: Optional[str] = '.'+os.sep+'Output', namefile: Optional[Union[bool, None]] = 'all', format: Optional[Union[bool, None]] = 'ESRI Shapefile', verbose: Optional[bool] = True, log: Optional[Union[str, None]] = None): (source) ¶

Interpolation of the data into a .tif raster

Parameters
outputdir:`str`, `Optional`	Output directory [Default: './Ouput']
inputdir:`str`, `Optional`	Input directory [Default: './Ouput']
namefile:`str`	File name for interpolation [Default: 'all']
format:`str`	Format of the output file [Default: 'ESRI Shapefile']
verbose:`bool`, `Optional`	Verbose [Default: `True`]
log:`str` or `None`, `Optional`	Log file [Default: `None`]

def dataclipping(outputdir: Optional[str] = '.'+os.sep+'Output', inputdir: Optional[str] = '.'+os.sep+'Output', namefile: Optional[str] = 'all', shapefile: Optional[str] = 'bbox.shp', __clipuseogr2ogr__: Optional[bool] = False, verbose: Optional[bool] = True, log: Optional[Union[str, None]] = None): (source) ¶

Clip the datasets

Parameters
outputdir:`str`, `Optional`	Input directory [Default: './Ouput']
inputdir:`str`, `Optional`	Input directory [Default: './Ouput']
namefile:`str`, `Optional`	Name of the selected file [Default: 'all']
shapefile:`str`, `Optional`	Shapefile of the ROI [Default: 'bbox.sph']
__clipuseogr2ogr__:`bool`, optional	Use ogr2ogr for clipping [Default: False]
verbose:`bool`, `Optional`	Verbose [Default: `True`]
log:`str` or `None`, `Optional`	Log file [Default: `None`]

def datagridding(paragrid, outputdir: Optional[str] = '.'+os.sep+'Output', inputdir: Optional[str] = '.'+os.sep+'Output', namefile: Optional[Union[bool, None]] = 'all', AREA_OR_POINT: Optional[str] = 'Point', verbose: Optional[bool] = True, log: Optional[Union[str, None]] = None): (source) ¶

Interpolation of the data into a .tif raster

Parameters
paragrid:`dict`	Dictionnary of the interpolation parameters
outputdir:`str`, `Optional`	Output directory [Default: './Ouput']
inputdir:`str`, `Optional`	Input directory [Default: './Ouput']
namefile:`str`	File name for interpolation [Default: 'all']
AREA_OR_POINT:`str`, `Optional`	Metadata for the GTiff creation regarding the coordinate locations. Can be POINT or AREA. [Default: 'POINT']
verbose:`bool`, `Optional`	Verbose [Default: `True`]
log:`str` or `None`, `Optional`	Log file [Default: `None`]

def datamergingcsv(outputdir: Optional[str] = '.'+os.sep+'Output', inputdir: Optional[str] = '.'+os.sep+'Output', paratosave: Optional[any] = 'all', mode: Optional[str] = 'onlist', infoEGMSdownloader: Optional[any] = None, __removeduplicate__: Optional[bool] = True, __length_threshold__: Optional[int] = 1000, __usevrtmerging__: Optional[bool] = False, verbose: Optional[bool] = True, log: Optional[Union[str, None]] = None): (source) ¶

Merge the datasets in csv format

Parameters
outputdir:`str`, `Optional`	Input directory [Default: './Ouput']
inputdir:`str`, `Optional`	Input directory [Default: './Ouput']
paratosave:`str`, `Optional`	Selected parameters [Default: 'all']
mode:`str`, `Optional`	Mode on selection. Can be 'onlist' or 'onfiles' [Default: 'onlist']
infoEGMSdownloader:`str`, `Optional`	Class of EGMSdownloader. Can be 'onlist' or 'onfiles' [Default: `None`]
__removeduplicate__:`bool`	Remove the duplicate points [Default: True]
__length_threshold__:`int`	Length for the concave hull [Default: 1000]
__usevrtmerging__:`bool`	Use the vrt for merging [Default: False]
verbose:`bool`, `Optional`	Verbose [Default: `True`]
log:`str` or `None`, `Optional`	Log file [Default: `None`]

def datamergingtiff(outputdir: Optional[str] = '.'+os.sep+'Output', inputdir: Optional[str] = '.'+os.sep+'Output', paratosave: Optional[any] = 'all', mode: Optional[str] = 'onlist', infoEGMSdownloader: Optional[any] = None, verbose: Optional[bool] = True, log: Optional[Union[str, None]] = None): (source) ¶

Merge the datasets in tiff format (only for L3 level)

Parameters
outputdir:`str`, `Optional`	Input directory [Default: './Ouput']
inputdir:`str`, `Optional`	Input directory [Default: './Ouput']
paratosave:`str`, `Optional`	Selected parameters [Default: 'all']
mode:`str`, `Optional`	Mode on selection. Can be 'onlist' or 'onfiles' [Default: 'onlist']
infoEGMSdownloader:`str`, `Optional`	Class of EGMSdownloader. Can be 'onlist' or 'onfiles' [Default: `None`]
verbose:`bool`, `Optional`	Verbose [Default: `True`]
log:`str` or `None`, `Optional`	Log file [Default: `None`]

def filemergingcsv(inputdir, outputdir, name, listfile, paratosave, mode_duplicate, length_duplicate, verbose, log): (source) ¶

Sub-function to merge the data in .csv format

Parameters
inputdir:`str`	Input directory
outputdir:`str`	Input directory
name:`str`	Name of the selected file
listfile:`list`	List of files
paratosave:`list`	Lists of saved parameters
mode_duplicate:`bool`	Mode to remove the duplicate points
length_duplicate:`int`	Length for concave hull
verbose:`bool`	Verbose
log:`str` or `None`	Log file

def filemergingcsvvrt(inputdir, outputdir, name, listfile, paratosave, mode_duplicate, length_duplicate, verbose, log): (source) ¶

Sub-function to merge the data in .csv format by using the .vrt format (only for L2a/L2b datasets and constant headers)

Warning: there is a bug with the raw .csv files if the duplicate mode is False and all parameters are saved.

Parameters
inputdir:`str`	Input directory
outputdir:`str`	Input directory
name:`str`	Name of the selected file
listfile:`list`	List of files
paratosave:`list`	Lists of saved parameters
mode_duplicate:`bool`	Mode to remove the duplicate points
length_duplicate:`int`	Length for concave hull
verbose:`bool`	Verbose
log:`str` or `None`	Log file

def filemergingtiff(inputdir, outputdir, name, listfile, verbose, log): (source) ¶

Sub-function to merge the geotiff data

Parameters
inputdir:`str`	Input directory
outputdir:`str`	Input directory
name:`str`	Name of the selected file
listfile:`list`	List of files
verbose:`bool`	Verbose
log:`str` or `None`	Log file

def listtodictmerged(list): (source) ¶

Sub-function to conver the list of a merged dictionary

Parameters
list:`list`	List of files
Returns
filedict (dict) release (str) level (str) track (str) L3compall (str)

def removerawdata(inputdir: Optional[str] = '.'+os.sep+'Output', forcemode: Optional[bool] = False, verbose: Optional[bool] = True, log: Optional[Union[str, None]] = None): (source) ¶

Delete the raw dataset(s) from EGMS

Parameters
inputdir:`str`, `Optional`	Input directory [Default: './Ouput']
forcemode:`bool`, `Optional`	Force mode [Default: `False`]
verbose:`bool`, `Optional`	Verbose [Default: `True`]
log:`str` or `None`, `Optional`	Log file [Default: `None`]

keygdalprog: str = (source) ¶

Undocumented

latlon_to_meter = (source) ¶

Pyproj class Lat/lon (epsg:4326) to meter (epsg:3035)

meter_to_latlon = (source) ¶

Pyproj class meter (epsg:3035) to Lat/lon (epsg:4326)

source_crs: str = (source) ¶

Source CRS.

target_crs: str = (source) ¶

Target CRS.