No Clocks

GeoETL is a CLI tool for geospatial data conversion and processing, built with Rust and Apache DataFusion.

Share R objects across processes on the same machine via a single copy in POSIX shared memory (Linux, macOS) or a Win32 file mapping (Windows). Every process reads from the same physical pages through the R Alternative Representation (ALTREP) framework, giving lazy, zero-copy access. Shared objects serialize compactly as their shared memory name rather than their full contents.

Share data without duplication using zero-copy patterns. Reduce storage costs and enable collaboration across streaming and lakehouse systems.

Patterns and Techniques for Writing High-Performance Applications with Go

A blog about RStats by David Schoch

Introducing how to use Rcpp

MapLibre Tile Specification

Provides tools to check variables contained in the user environment, and inspect the currently loaded package namespaces. The intended use is to allow user scripts to throw errors or warnings if unwanted variables exist or if unwanted packages are loaded.

A large C/C++-based package for advanced data transformation and statistical computing in R that is extremely fast, class-agnostic, robust, and programmer friendly. Core functionality includes a rich set of S3 generic grouped and weighted statistical functions for vectors, matrices and data frames, which provide efficient low-level vectorizations, OpenMP multithreading, and skip missing values by default. These are integrated with fast grouping and ordering algorithms (also callable from C), and efficient data manipulation functions. The package also provides a flexible and rigorous approach to time series and panel data in R, fast functions for data transformation and common statistical procedures, detailed (grouped, weighted) summary statistics, powerful tools to work with nested data, fast data object conversions, functions for memory efficient R programming, and helpers to effectively deal with variable labels, attributes, and missing data. It seamlessly supports base R objects/classes as well as units, integer64, xts/ zoo, tibble, grouped_df, data.table, sf, and pseries/pdata.frame.

I’m getting more and more into data engineering these days and having used R for a long time, I’m seeing a lot of problems that look nail-shaped to my R-shaped hammer. The available tools to solve those problems exist for (presumably) very good reasons, so I wanted to take some time to dig into how to use them and compare their workflows to what I would otherwise naively do in R.

Implements the Dagster Pipes protocol, enabling R scripts to communicate with the Dagster orchestrator. R scripts can receive execution context and report asset materializations, check results, and log messages back to Dagster.

Rust geodesy

sx: Scalable Spatial Data Analysis

FileOutputStream is for writing to a file; BufferOutputStream writes to a buffer; You can create one and pass it to any of the table writers, for example.

See examples from @anthonynorth njtierney/demo-geotargets#1

The handler is the basic building block of the wk package. In particular, the wk_handle() generic allows operations written as handlers to "just work" with many different input types. The wk package provides the wk_void() handler, the wk_format() handler, the wk_debug() handler, the wk_problems() handler, and wk_writer()s for wkb(), wkt(), xy(), and sf::st_sfc()) vectors.

Proof of Concept 'GeoPackage' to Arrow Converter

When facing large geopackages, you can chunk it in geoparquet files and process each chunk seperately. - AURIN-OFFICE/geoparquet_data_manager

I have been too used to PostgreSQL in the time I have spent in my career so far. PostgreSQL and MySQL are what I had worked with in the beginning.