Optimizing GIS Data for Speed, Accuracy, and Scale | Quarticle Blog
Learn how to clean, optimize, and integrate GIS data for high-performance analytics. Discover techniques to improve speed, accuracy, and decision-making.
3.3 Cartographic Principles – Geographic Information Technologies
In addition to the effective use of colors and symbols, a map that is well designed will significantly enhance its ability to relate pertinent spatial…
Scale and Resolution – Making Effective Maps: Cartographic Visualization for GIS
The creation of this work was supported by Open CU Boulder 2023-2024, a grant funded by the Colorado Department of Higher Education with additional support from the CU Office of the President, CU Office of Academic Affairs, CU Boulder Office of the Provost, and CU Boulder University Libraries.
Source Cooperative is the data publishing utility for the web, allowing trusted organizations and individuals to publish data of any kind at any scale.
Google-Microsoft-OSM Open Buildings - combined by VIDA · VIDA · Source Cooperative
This dataset merges Google's V3 Open Buildings, Microsoft's GlobalMLFootprints, and OpenStreetMap building footprints. It contains 2,705,459,584 footprints and is divided into 200 partitions. Each footprint is labelled with its respective source, either Google, Microsoft, or OpenStreetMap. It can be accessed in cloud-native geospatial formats such as GeoParquet, FlatGeobuf and PMTiles.
A minimal columnar query engine with lazy execution on datasets larger than RAM. Provides dplyr-like verbs (filter(), select(), mutate(), group_by(), summarise(), joins, window functions) and common aggregations (n(), sum(), mean(), min(), max(), sd(), first(), last()) backed by a pure C11 pull-based execution engine and a custom on-disk format (.vtr).
Access and interrogate EMODnet (European Marine Observation and Data Network) Web Feature Service data https://emodnet.ec.europa.eu/en/emodnet-web-service-documentation#data-download-services. This includes listing existing data sources, and getting data from each of them.
Tools to run system processes in the background. It can check if a background process is running; wait on a background process to finish; get the exit status of finished processes; kill background processes. It can read the standard output and error of the processes, using non-blocking connections. processx can poll a process for standard output or error, with a timeout. It can also poll several processes at once.
Contract helpers built with S7 for expressing runtime protocols around ordinary S7 dispatch. Structural interfaces describe small sets of required S7 generics, while explicit traits record registered implementations with optional default methods and associated metadata. Optional runtime checks can validate argument and return specifications in contract-scoped evaluation.
Share R objects across processes on the same machine via a single copy in POSIX shared memory (Linux, macOS) or a Win32 file mapping (Windows). Every process reads from the same physical pages through the R Alternative Representation (ALTREP) framework, giving lazy, zero-copy access. Shared objects serialize compactly as their shared memory name rather than their full contents.
Provides tools to check variables contained in the user environment, and inspect the currently loaded package namespaces. The intended use is to allow user scripts to throw errors or warnings if unwanted variables exist or if unwanted packages are loaded.
A large C/C++-based package for advanced data transformation and statistical computing in R that is extremely fast, class-agnostic, robust, and programmer friendly. Core functionality includes a rich set of S3 generic grouped and weighted statistical functions for vectors, matrices and data frames, which provide efficient low-level vectorizations, OpenMP multithreading, and skip missing values by default. These are integrated with fast grouping and ordering algorithms (also callable from C), and efficient data manipulation functions. The package also provides a flexible and rigorous approach to time series and panel data in R, fast functions for data transformation and common statistical procedures, detailed (grouped, weighted) summary statistics, powerful tools to work with nested data, fast data object conversions, functions for memory efficient R programming, and helpers to effectively deal with variable labels, attributes, and missing data. It seamlessly supports base R objects/classes as well as units, integer64, xts/ zoo, tibble, grouped_df, data.table, sf, and pseries/pdata.frame.
Comparing R's {targets} and dbt for Data Engineering
I’m getting more and more into data engineering these days and having used R for a long time, I’m seeing a lot of problems that look nail-shaped to my R-shaped hammer. The available tools to solve those problems exist for (presumably) very good reasons, so I wanted to take some time to dig into how to use them and compare their workflows to what I would otherwise naively do in R.
Implements the Dagster Pipes protocol, enabling R scripts to communicate with the Dagster orchestrator. R scripts can receive execution context and report asset materializations, check results, and log messages back to Dagster.