No Clocks

GDALVector is a C++ class exposed in R via Rcpp modules.

It encapsulates a single OGRLayer and the GDALDataset that contains it.

An object of class GDALVector persists an open connection to the dataset, and exposes methods to: retrieve layer information set attribute and spatial filters set ignored fields/selected fields traverse and read feature data by traditional row-based cursor fetch the full feature set and return as a data frame fetch progressively in batches of n features at a time read via column-oriented ArrowArrayStream write new features in a layer upsert edit/overwrite existing features delete perform operations within transactions

A GDALVector object is typically generated with a call to new() but is also returned from certain ogr_*() functions. The documentation for ?GDALVector gives a full list of available methods in the Usage section, along with their descriptions under Details. Methods of the class are accessed using the $ operator.

Several of the stand-alone ogr_*() functions are grouped under the documentation topics ?ogr_manage and ?ogr_define

The ogr_manage functions can be used to: create new vector datasets from scratch test for existence of datasets, layers and fields test dataset and layer capabilities create new layers in an existing dataset delete layers create new attribute and geometry fields on an existing layer rename and delete fields read and write field domains edit data with SQL statements

ogr_define provides documentation and helper functions for defining feature classes. An OGR feature class definition (a.k.a. layer definition) is modeled in R as a named list containing zero or more attribute field definitions, along with one or more geometry field definitions. Specifications of the the list structures for these definitions are given in ?ogr_define. The associated helper functions make it easy to create new layer definitions from scratch or modify an existing definition. A layer definition is convenient but not required for creating a new vector dataset, or a new layer within an existing dataset, using ogr_ds_create() / ogr_layer_create().

OGR facilities for vector geoprocessing are available in ogr_proc(). This function can perform the following GIS overlay operations: Intersection, Union, SymDifference, Identity, Update, Clip and Erase (https://en.wikipedia.org/wiki/Vector_overlay). ogr_proc() is basically an R port of the command-line utility ogr_layer_algebra included in the GDAL Python bindings. In both cases, these are interfaces to library functions in the OGR C++ API.

ogrinfo() and ogr2ogr(), provide R wrappers of the GDAL command-line utilities ogrinfo and ogr2ogr. These functions support all of the command-line arguments described in the GDAL documentation, providing a powerful set of capabilities for obtaining information about an OGR-supported data source, converting vector data between file formats, and potentially editing data with SQL statements. ogr_reproject() is a convenience wrapper around ogr2ogr() for reprojecting vector layers with a user friendly interface.

Virtual Systems Interface Bindings to the GDAL’s Virtual Systems Interface (VSI) implement standard file system operations abstracted for URLs, cloud storage services, Zip/GZip/7z/RAR, in-memory files, as well as “regular” local file systems. This provides a single interface for operating on file system objects, that works the same for any storage backend. The vsi_*() functions have general utility not limited to operating on spatial data sources.

Class VSIFile implements bindings to the GDAL VSIVirtualHandle API, providing analogs of Standard C file I/O functions that operate on VSI file systems (seek(), tell(), read(), write(), etc.)

Existing data management functions that operate on both raster and vector data sources include addFilesInZip() (supporting create/append to a potentially Seek-Optimized ZIP file), deleteDataset(), identifyDriver() and inspectDataset().

Creation of new vector datasets, and schema modification in existing datasets, are performed with the ogr_*() stand-alone functions. These are “one-off” operations that attempt to open the dataset with update access, perform modifications, and then close the dataset.

Objects of class GDALVector are used for obtaining layer information and reading/writing feature data. A GDALVector object provides a persistent connection to a layer and the vector dataset that contains it. That is, once instantiated, a GDALVector object represents a live connection to the dataset until its $close() method is called. The connection may be read-only (by default), or may be with update access allowing insert of new features, modifying exiting features and deleting features. Currently, class GDALVector does not provide methods for modifying the layer schem

Relational database management systems (RDBMSs, e.g., GPKG / SQLite, PostgreSQL / PostGIS) generally support multiple connections including concurrent reads (e.g., multiple instances of GDALVector reading from different layers). It is also possible to have one or more GDALVector objects instantiated on RDBMS-based layers for read access, while another GDALVector object performs write operations on a different layer in the same database. Database locking mechanisms for write operations are specific to the driver and underlying RDBMS (see, e.g., SQLite Configuration Options and Performance Hints).

A GDALVector object supports traditional cursor-based traversal over the features (rows) in a layer, and optionally, column-oriented retrieval via Apache Arrow C stream interface (with GDAL >= 3.6). Data retrieval may be performed against a vector layer in full, a layer with attribute and/or spatial filters applied, or a layer defined as the result set of an SQL statement executed on the underlying data source.

OGR methods that retrieve a single feature (i.e., $getFeature(), $getNextFeature()) return data in a named list of fields and their values. GDALVector also provides the $fetch() method to retrieve a batch of n features from the current cursor position and return them in a data frame. The $fetch() method is an analog of DBI::dbFetch() with essentially identical calling semantics. Specifying n = -1 or n = Inf will retrieve all features from the beginning (honoring any spatial or attribute filters that may be in effect).

Currently, gdalraster implements only minimal S3 class interfaces for R objects containing the returned feature data. A single feature as returned by $getFeature()/$getNextFeature() is a list object carrying the "OGRFeature" class attribute. A data frame returned by $fetch() carries the "OGRFeatureSet" class attribute. S3 methods are provided for the print() and plot() generics as a convenience for examining output. Otherwise, there are currently no S3 classes for geometries or geometry columns, no concept of “sticky” geometry as implemented in package sf, and no automatic propagation of coordinate reference systems. S3 interfaces may expand in future versions, but gdalraster leans toward minimalism and the use of simple, lightweight objects for holding raw data.

The $fetch() method of a GDALVector object returns geometries in a list column of WKB raw vectors by default. The Geometry API functions (g_*()) also operate by default on lists of WKB raw vectors. Representation of geometries as WKB is compact and performant, and seamlessly integrates with the parsing, conversion, manipulation and plotting functions in package wk (a gdalraster dependency for vector plotting).

The $getArrowStream() method of a GDALVector object allows retrieving data in a column-oriented memory format. It exposes an ArrowArrayStream on a layer as a nanoarrow_array_stream object. The nanoarrow package provides functionality to consume the array stream and import to R data structures (which are generally column oriented). nanoarrow provides helpers to facilitate zero-copy data transfer among R bindings to libraries implementing the Arrow C data interface. It is possible to pass nanoarrow objects to many functions in the arrow package. nanoarrow objects also integrate with the extension types implemented in the geoarrow package.

A data frame returned by the $fetch() method of a GDALVector object (i.e., an "OGRFeatureSet") can be converted to an sf data frame with sf::st_sf(). A value for the crs argument can be obtained from method $getSpatialRef() of the GDALVector object from which the data were read (assuming no subsequent transformation of geometries has been performed).

OGR field types are returned as the following R data types. R currently lacks a native 64-bit integer type. Support for 64-bit integer values in R is provided by the bit64 package (represented as numeric values carrying the "integer64" class attribute). OGR NULL values are returned as type-specific NA (i.e., NA, NA_integer_, NA_integer64_, NA_real_, NA_character_). When retrieving a batch of features as a data frame ("OGRFeatureSet"), some field types will be contained in a data frame list column as indicated: OFTInteger: integer value OFTInteger subtype OFSTBoolean: logical value OFTIntegerList: vector of integer (list column) OFTInteger64: numeric value carrying the "integer64" class attribute OFTInteger64 subtype OFSTBoolean: logical value OFTInteger64List: vector of bit64::integer64 (list column) OFTReal: numeric value OFTRealList: vector of numeric (list column) OFTString: character string OFTStringList: vector of character strings (list column) OFTDate: numeric value of class "Date" OFTDateTime: numeric value of class "POSIXct" (millisecond accuracy) OFTTime: character string ("HH:MM:SS") OFTBinary: raw vector (list column, NULL entries for OGR NULL values)

By default, geometries are returned as WKB raw vectors in a data frame list column (with NULL entries for OGR NULL geometries). The per-object setting $returnGeomAs can also be set to one of "WKB_ISO", "WKT", "WKT_ISO", or "NONE". Omitting the geometries (e.g., by setting lyr$returnGeomAs <- "NONE") may be beneficial for performance and memory usage when access only to feature attributes is required.

Inspect data sources Data sources can be files, relational database management systems, directories of many files, or even remote web services depending on the format driver being used. However, the data source name (DSN) is always a single string which might be the file path, database connection string, URL, etc.

The following examples use a GeoPackage file included in gdalraster. The file ynp_features.gpkg is compressed using SOZip and will be read directly from the .zip archive without decompressing first. A prefix is added to the file path (/vsizip/ in this case) which specifies a GDAL Virtual File System handler. A file system handler provides access to less standard file types such as in-memory, compressed (.zip, .gz, .tar, .tar.gz), encrypted, standard input and output (STDIN, STDOUT), files stored on network (publicly accessible, or in private buckets of commercial cloud services), etc.