Packages Good for Large Dataset Processing

Three criteria to evaluate if a package or library is suitable for processing very large datasets. Implement core functions in a language that is efficient for computation C, C++, FORTRAN, Rust are rather good on this. Python, R, and other high-level languages are not very good. Support multithreading and even distributed computing Relying on single core, single thread would not produce good performance. It is necessary to support multi-threading, either on the same computer or a cluster Enable efficient indexing [Read More]

Vector Tiles for Leaflet

Install GDAL Tools The GDAL tools are installed from https://www.gisinternals.com/. Install the core with Python applications, which contain some useful raster tools like gdal2tile.py. Alternatively, it can be installed using OSGeo4W. However, OSGeo4W can interfere with PostgreSQL/PostGIS on GDAL for the setting. Therefore, it is not recommended to use OSGeo4W for GDAL. As QGIS is part of OSGeo, it comes with GDAL pre-installed. So, it is also good to use the GDAL within QGIS. [Read More]