ICESat-2 ATL03 photons to river cross-sections processing scripts

<p dir="ltr">These scripts extract, process and filter ICESat-2 crossing data for cross-section definition in rivers.</p><p dir="ltr">Copyright © [2024] DHI A/S</p><p dir="ltr">Permission is hereby granted, free of charge, to any person obtaining a copy</p><p dir="ltr">of this software and associated documentation files (the "Software"), to deal</p><p dir="ltr">in the Software without restriction, including without limitation the rights</p><p dir="ltr">to use, copy, modify, merge, publish, distribute, sublicense, and/or sell</p><p dir="ltr">copies of the Software, and to permit persons to whom the Software is</p><p dir="ltr">furnished to do so, subject to the following conditions:</p><p><br></p><p dir="ltr">The above copyright notice and this permission notice shall be included in all</p><p dir="ltr">copies or substantial portions of the Software.</p><p><br></p><p dir="ltr">THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR</p><p dir="ltr">IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,</p><p dir="ltr">FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE</p><p dir="ltr">AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER</p><p dir="ltr">LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,</p><p dir="ltr">OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE</p><p dir="ltr">SOFTWARE.</p><p dir="ltr"><br></p><p dir="ltr">The methods were developed on a stretch of the Mekong River</p><p dir="ltr">- Input files and intermediate products to reproduce the resulting cross-sections are included.</p><p dir="ltr">ICESat-2 products are not included, but are available from NSIDC:</p><p dir="ltr">https://nsidc.org/data/atl03</p><p dir="ltr">https://nsidc.org/data/atl08</p><p dir="ltr">https://nsidc.org/data/atl13</p><p dir="ltr">Version 5 of the ICESat-2 products were used in the making of this paper. Newer versions are now available.</p><p dir="ltr">All example file-paths must be replaced with actual filepaths before scripts are run.</p><p><br></p><p><br></p><p><br></p><p dir="ltr">The python scripts require the following modules</p><p dir="ltr">shapely - © Copyright 2011-2024, Sean Gillies and Shapely contributors. https://creativecommons.org/licenses/by/3.0/us/</p><p dir="ltr">geopandas - © Copyright 2013–2024, GeoPandas developers. </p><p dir="ltr">rasterio - Copyright (c) 2016, MapBox All rights reserved.</p><p dir="ltr">scipy - Copyright © 2001, 2002 Enthought, Inc. All rights reserved. Copyright © 2003-2019 SciPy Developers. All rights reserved.</p><p dir="ltr">pyproj - Copyright (c) 2006-2018, Jeffrey Whitaker. Copyright (c) 2019-2023, Open source contributors.</p><p dir="ltr">h5py - Copyright (c) 2008 Andrew Collette and contributors</p><p dir="ltr">hampel - Copyright (c) 2018 The Python Packaging Authority</p><p dir="ltr"><br></p><p dir="ltr"><br></p><p dir="ltr">To ensure the required intermediate products are ready when needed, use the following script run order:</p><p dir="ltr">- ATL13 00Extract and 01EGM08_HarmSynth in any Python installation</p><p dir="ltr">- ATL13 02QGIS_Hub_Processing in the QGIS Python console</p><p dir="ltr">- ATL13 03 through 07</p><p dir="ltr">- ATL08 00 through 04 (02 in QGIS)</p><p dir="ltr">- ATL03 00 through 15 (02 in QGIS)</p><p dir="ltr">- ATL13 08 through 14</p><p><br></p><p dir="ltr">Some scripts cannot be run without in-situ data from the Mekong River Commission, but intermediate products are available to run subsequent scripts.</p><p><br></p><p dir="ltr">Inputs</p><p dir="ltr">- ATL03, ATL08 and ATL13 .h5 files in their respective folders</p><p dir="ltr">- MekongStrech_4k_Buffer_4326.shp</p><p dir="ltr">- 4km buffer on the river chainage line of the stretch. Made in UTM48N, but reprojected to epsg:4326 to subset ICESat-2 data points</p><p dir="ltr">- EGM2008 program</p><p dir="ltr">- Relevant files downloaded from https://earth-info.nga.mil/php/download.php?file=egm-08spherical should be placed in the folder 'EGM2008_SphericalHarmonics</p><p dir="ltr">- MekongStrechChainage1m.gpkg</p><p dir="ltr">- River chainage line converted to points at 1 m intervals using the QGIS pluging QChain</p><p dir="ltr">- Must be UTM CRS for distancing</p><p dir="ltr">- occurrence_100E_20Nv1_3_2020.tif</p><p dir="ltr">- Raster of surface water occurance used to filter out points over land. The raster with the relevant AOI is found at https://global-surface-water.appspot.com/download</p><p dir="ltr">- EGM08_HarmSynth_1k.tif</p><p dir="ltr">- Raster with resolution 1 km x 1 km of EGM08 geoid correction for the area of interest. UTM CRS</p><p dir="ltr">- URBS Catchment contributing discharge time series</p><p dir="ltr">- Available from MRC upon request</p><p dir="ltr">- Discharge Time Series</p><p dir="ltr">- Discharge time series from rating curves at 3 gauging stations. Available from MRC upon request</p><p dir="ltr">- IRIS_netcdf_v2.nc</p><p dir="ltr">- IRIS database of ATL13 slopes, available at https://zenodo.org/record/8202984</p><p dir="ltr">- SWORD_MekongStretch_v15.gpkg</p><p dir="ltr">- Geopackage of relevant reaches, extracted from the SWORD database found at https://www.swordexplorer.com/</p><p dir="ltr">- WSS_07092023_0913.txt</p><p dir="ltr">- ICE2WSS r-package export of ATL13 slope estimates from the same set of ATL13 files.</p><p><br></p><p dir="ltr">Outputs</p><p dir="ltr">- ICESat intermediate products are found in subsequent folders: GPKGs, UTM, HubDist, Referenced, Chained</p><p dir="ltr">- ATL13_Collected_GlobalSurface.gpkg</p><p dir="ltr">- A combined geopackage of all ATL13 tracks processed so far. Used in subsequent processing</p><p dir="ltr">- ATL13WSE.csv</p><p dir="ltr">- Table of Water Surface Elevation estimates. Columns: TimeTrack,weightedWSE,STD,Chainage,UTM48N_x,UTM48N_y,Time,Track,ctime</p><p dir="ltr">- SlopeFrame.csv</p><p dir="ltr">- Table of Water Surface Slope estimates. Columns: Time,ChainMin,ChainMax,Slope</p><p dir="ltr">- flowAcc_Chainage.csv</p><p dir="ltr">- Table of Upstream Drainage Area increase by river chainage. Columns: cngmeters,HubName,HubDist</p><p dir="ltr">- catchmentMeanRunoff.csv</p><p dir="ltr">- Table of Catchment mean discharge contribution, area and mean runoff generation. Columns: Catchment,Mean Discharge,Median,Chain_Start,Chain_End,UDA_Start,UDA_End,UDA_tot,Runoff</p><p dir="ltr">- Runoff_accum.csv</p><p dir="ltr">- Table of Discharge accumulation along river chainage. Columns: cngmeters,Runoff_accum</p><p dir="ltr">- insituChainages.csv</p><p dir="ltr">- Table of river chainage position of each gauging station</p><p dir="ltr">- FirstLastPoint.csv</p><p dir="ltr">Table of first and last point position of each ATL03 crossing of the river</p><p dir="ltr">- ATL03CrossMap_Available.gpkg</p><p dir="ltr">- Geopackage showing ATL03 crossings available for cross-section definition</p><p dir="ltr">- ATLPairs.csv</p><p dir="ltr">- Table of matching ATL03 and ATL08 crossings</p><p dir="ltr">- ATL03_06CrossSectionSheet.csv</p><p dir="ltr">- Table of Cross-section parameters needed for definition. Columns: Crosssection,Tracks ATL13Time,Hampel_window,Hampel_n,Rolling_window,PlotLL,PlotUL,Use_ATL08,</p><p dir="ltr">ATL08_Window,Shape_Parameter,Bankpairs,xlim</p><p dir="ltr">- ATL03_06CrossSectionSheet_Rapids.csv</p><p dir="ltr">- Similar table as above, but with an additional cross-section</p><p dir="ltr">- ATL03_06CrossSecMakerSheet_DepthCalced_{}.csv</p><p dir="ltr">- Filled cross-section definition sheet with depths calculated from Manning's eq. {} indicates the assumed Mannings' n</p><p dir="ltr">- ATL03CrossMap_Used.gpkg</p><p dir="ltr">- Geopackage showing ATL03 crossings used for cross-section definition. The Parameter 'Used' indicates that the crossing is found in the list for definition of a cross-section</p><p dir="ltr">- XS_export_{}.txt</p><p dir="ltr">- Text export of the cross-sections. {} indicates the assumed Mannings' n</p><p dir="ltr">- ATL03_06CrossSecMakerSheet_DepthCalced_Split_{}_{}.csv</p><p dir="ltr">- Filled cross-section definition sheet with depths calculated from Manning's eq. {} indicates the assumed upstream Mannings' n and chainage position</p><p dir="ltr">- XS_export_Split_{}_{}.txt</p><p dir="ltr">- Text export of the cross-sections. {} indicates the assumed upstream Mannings' n and chainage position</p><p dir="ltr">- ATL03_06CrossSecMakerSheet_DepthCalced_Rapids_{}_{}.csv</p><p dir="ltr">- Filled cross-section definition sheet with depths calculated from Manning's eq with an additional cross-section. {} indicates the assumed upstream Mannings' n and chainage position</p><p dir="ltr">- XS_export_Rapids_{}_{}.txt</p><p dir="ltr">- Text export of the cross-sections with an additional cross-section. {} indicates the assumed upstream Mannings' n and chainage position</p><p dir="ltr">- ATL13WSE_and_Res_Hydro_Base_Short_ATL3XSs_{}.csv</p><p dir="ltr">- ATL13 WSE and corresponding model WSE. {} indicates model name</p><p dir="ltr">- model_WL_{}.csv</p><p dir="ltr">- WL time series at the river chainage of the gauging stations. {} indicates model name</p><p dir="ltr">- errorFrame_ICESat-2_WSE.csv</p><p dir="ltr">- Table of model WSE residuals compared to ATL13 WSE estimates</p><p dir="ltr">- Hydrostations_WL_Error.csv</p><p dir="ltr">- Table of model residuals and performance measures of modelled time series compared to gauging station series</p><p dir="ltr">- ErrorFrame_Merged.csv</p><p dir="ltr">- Combined table of above residuals</p><p dir="ltr">- ICE2WSS_Mekong.csv</p><p dir="ltr">- ICE2WSS slope estimates for relevant reaches</p><p dir="ltr">- SWORD_IRIS_WSS.csv</p><p dir="ltr">- IRIS slope estimates for relevant reaches</p><p><br></p><p><br></p>