Docs – hyperverse
Sharing data across shiny modules, an update - Rtask
The R task Force - R experts for all your needs
I thought the moment was perfect to give y’all an update on the latest approaches I’ve been using to share data across {shiny} modules, along with some thoughts and comments on the "stratégie du petit r".
But I’m trying to keep things easy to maintain. Given the current size of the codebase, adding more layers or going deeper would make the code far more complex and harder to maintain with no real benefit. So yes, some of these modules are not perfect, and they might not be doing “just one thing”.
You know what they say: “perfect is the enemy of good.”
I’ve come to terms with this idea for two reasons:
Data frames are lists, and I don’t see any good reason to forbid passing a data.frame as an argument to a function.
JavaScript is full of functions that take scalar values and a list of parameters, and it works well.
For example, making an HTTP request in JS looks like this:
fetch(
"/api/users",
{
method: "GET",
headers: {
"Content-Type": "application/json",
"Accept": "application/json",
"Authorization": "Bearer YOUR_TOKEN",
}
}
)
Modules usually live in two scopes:
They do things within themselves
They do things that need to be passed to other modules
Doing things within themselves is pretty standard and doesn’t require a lot of thought (as long as you don’t forget the ns() 😅), but sharing things from one module to another in a reactive context can be more challenging
Passing reactive objects
One thing I’ve learned over the years is that what works for example apps can be a nightmare in a production context. The official Shiny docs recommend the following pattern: return one or more reactive() objects that can be passed to other modules.
If you feel like it’s a mess and complex to reason about, that’s because it is. And we’re in a simple case where data travels at the same depth in the stack.
As a side note, I think reactive() objects are conceptually neat, but I don’t think they should be your go-to building block.
It’s built in React, and it works just like {shiny} does (well, from a conceptual point of view): you have stateful objects, and when these objects change, they trigger another part of the app to be recomputed. In our case, whenever you interact with the first tab, the second tab (with the visualization) is updated.
To sum up, some objects are created at the top level and used to share data and trigger reactivity from one “module” to the other.
Note: my colleague Arthur pointed that Vue.js has something called store in Pinia. I’m not exactly sure how it works but apparently it’s more or less the same as reactiveValues. And Claude confirmed it 😄
THE “STRATÉGIE DU PETIT R”
One strategy we recommended is what we called the “stratégie du petit r”. Looking back, I can admit that it was a poor choice of name, but you know, sh*t happens.
The principle is quite simple: instead of returning and passing reactive() objects as arguments, you create one or more reactiveValues() at an upper level, which you then pass downstream to lower-level modules. reactiveValues() behave a lot like environments, meaning that values set down the stack are available everywhere.
I still think this is a valid way to share data, but only if you avoid applying it too literally and focus on how to work with it in practice.
The main criticism I’ve read about this approach is that you’ll end up with a huge r object with 300 entries in it, creating a monster that’s impossible to debug.
So yes, these monsters exist. But I don’t think the idea itself is the problem. It’s always easier to blame the tool than to acknowledge the lack of understanding behind its misuse. Or, as Beckett wrote, “Voilà l’homme tout entier, s’en prenant à sa chaussure alors que c’est son pied le coupable.” (“There’s man all over for you, blaming on his boots the faults of his feet.”)
Here are some random thoughts:
The corollary of the last point is simple: you need several reactiveValues(), operating at different scopes in your application.
STORAGE USING AN R6 OBJECT
One downside I can think of when using the reactiveValues() strategy I just described is that, well, it’s reactive, meaning it can lead to uncontrolled reactivity if things aren’t scoped correctly.
One pattern I’ve used in an app is combining an R6 object, used to store and process data, with the trigger mechanism from {gargoyle}. Basically, the idea behind {gargoyle} is simple: instead of relying on the reactive graph to invalidate itself, you init flags that are triggered in the code, and when a flag is triggered, the context where the flag is watched is invalidated.
It’s a bit longer to implement, but you get better control over what is happening.
Combined with this, you can use an R6 object that is passed along the modules, and that gets transformed to store, process, and serve the data.
You can read more about this in “15.1.3 Building triggers and watchers” and “15.1.4 Using R6 as data storage” in Chapter 15 of the Engineering Shiny book.
SESSION$USERDATA
This one should be used with a lot of caution, but it can be very effective if you know what you’re doing (and if you don’t have too many things to share).
The session object is an environment available everywhere in your Shiny app. It represents the current interaction between each user and the R session (i.e., each user has their own). This environment has a special slot called userData that can be populated with data, and it is scoped to the session.
The way I’ve used it in the past is via wrappers, which would look like:
set_this <- function(value, session = shiny::getDefaultReactiveDomain()){
session$userData$this <- compute_this(value)
}
get_this <- function(session = shiny::getDefaultReactiveDomain()){
session$userData$this
}
So anywhere I need it, I’ll use the wrapper function instead of session$userData$this. I would generally use it to define things at the top level that need to be accessible everywhere downstream, but I feel it might be a bit complex to manage if you need to pass data from mod_3_a to mod_3_g.
The documentation says it can be used “to store whatever session-specific data (we) want”, but my gut feeling is that it’s best not to shove too much into it. But I don’t have any rationale reason and I’d be happy to be proven wrong.
AN ENVIRONMENT IN THE SCOPE OF THE PACKAGE/TOP LEVEL OF THE APP
This is something a lot of R developers do: define an environment inside the package namespace so that, when the package is loaded, you can CRUD into it. For example, there are some (well, several) in {shiny}:
> shiny:::.globals
The function shinyOptions() writes to it, and getShinyOption() reads from it.
This pattern can be used as global storage, but be careful: it’s not session-scoped, so whatever is in this environment is shared across sessions.
AN EXTERNAL DATABASE OR STORAGE SYSTEM
Another solution is to store values in an external database, and query that DB inside modules.
If you try to implement this solution, two things to keep in mind are:
Make the data session-scoped, i.e., use session$token to identify the current session, and remove the data when the session ends.
You’ll need to handle reactivity manually, for example with {gargoyle}.
For example, with {storr}:
# Mimicking a session
session <- shiny::MockShinySession$new()
# In module 1
st <- storr::storr_rds(here::here())
st$set("dataset", mtcars, namespace = session$token)
# In module 2
st <- storr::storr_rds(here::here())
st$get("dataset", namespace = session$token)
Of course, this is a short piece of code and you’ll need more engineering, but you get the idea.
S7 & Options objects | Josiah Parry
Update on mocking for testing R packages - R-hub blog
This blog featured a post on mocking, the art of replacing a function with whatever fake we need for testing, years ago. Since then, we’ve entered a new decade, the second edition of Hadley Wickham’s and Jenny Bryan’s R packages book was published, and mocking returned to testthat, so it’s time for a new take/resources roundup!
Thanks a lot to Hannah Frick for useful feedback on this post!
Taming gnarly nested data with purrr::modify_tree – Joe Kirincic
A tutorial for using purrr::modify_tree effectively.
Igloo
Discover how the sitrep() pattern simplifies R package maintenance and surfaces configuration errors in one go.
Stop playing “diagnostics ping-pong” with your users. This post explores why the _sitrep() (situation report) pattern — popularized by the usethis package — is a game-changer for R packages wrapping APIs or external software. Learn how to build structured validation functions that power both early error-handling and comprehensive system reports, featuring real-world implementation examples from the meetupr and freesurfer packages.
Welcome to blockr – blockr
altdoc
Package Development – Data Science
Data Table Back-End for dplyr
Provides a data.table backend for dplyr. The goal of dtplyr is to allow you to write dplyr code that is automatically translated to the equivalent, but usually much faster, data.table code.
Wrappers for GDAL Utilities Executables
Rs sf package ships with self-contained GDAL
executables, including a bare bones interface to several
GDAL-related utility programs collectively known as the GDAL
utilities. For each of those utilities, this package provides an
R wrapper whose formal arguments closely mirror those of the
GDAL command line interface. The utilities operate on data
stored in files and typically write their output to other
files. Therefore, to process data stored in any of Rs more common
spatial formats (i.e. those supported by the sf and terra
packages), first write them to disk, then process them with the
package's wrapper functions before reading the outputted results
back into R. GDAL function arguments introduced in GDAL version
3.5.2 or earlier are supported.
elipousson/sfext: ✂️🌐 A R package with extra options for simple features and spatial data
✂️🌐 A R package with extra options for simple features and spatial data - elipousson/sfext
Chapter 9 Making maps with R | Geocomputation with R
Prerequisites This chapter requires the following packages that we have already been using: library(sf) library(terra) library(dplyr) library(spData) library(spDataLarge) The main package used in...
Understanding Your Topographic Map Maker
The first-ever topographic maps were said to have been formed by the British in the late 18th century, and soon after, the US followed suit. Back then, the US had a department called the “Topographical Bureau of the Army,” which used these maps to plan tactical strategies during the War of 1812. However, even when the war ended, our interest in topography remained.
The term “topography” comes from a combination of two Greek words: “topo,” which means place, and “graphia,” which means writing. It is used to describe the study of a region’s forms and features, primarily to show their relative positions and elevations. Topography could refer to the forms and features themselves or a depiction of them (such as a map).
Unlike traditional maps which only represent the land horizontally, one made with a topographic map maker will represent the land vertically as well. These maps, also referred to as topo maps, show the form and elevation of an area, including the location and shape of hills, valleys, mountains, streams, and other natural or human-made features.
Contour Lines
Contour lines are the primary way in which topographic maps depict elevation. These imaginary lines connect points of equal elevation in order to present three-dimensional information on a two-dimensional map. This allows the viewer to visualize the height and shape of mountains, the depth of canyons, and the location of flat plains.
To determine the exact elevation of a location, you’ll need to know the contour interval – the difference in elevation between two contour lines. This will vary depending on the map, but regardless, you can calculate them yourself fairly easily. First, find the bolded contour lines that contain a number. These are the index contours, and the number is the elevation at the line.
Then, count the number of contour lines between each index contour, and divide the difference in elevation by that number. For instance, if you had one index contour with an elevation of 7,800 five contour lines apart from another index with an elevation of 8,000, the contour interval would be 40 ( (8,000 - 7,800) / 5 = 40 ).
The contour lines produced by your topographic map maker are often used to determine the slope or steepness of an area. The lines will be spaced farther apart when the slope is gentle, and closer together when the slope is steep. This is because, in steep areas, the elevation will increase at a greater frequency, so the lines will appear closer together. A completely flat meadow will have no contour lines, while a vertical cliff will have contour lines that are stacked on top of one another.
Features
You can also use contour lines to identify features of the land.
Peaks and Depressions: The innermost ring at the center of several other rings will typically represent a peak, but in some cases, it could represent a depression.
Valleys: A valley is a type of depression in which water could flow down (if water is present), and they can be identified by their V or U shaped contour lines that point towards higher elevation (the peak).
Cliffs: When two or more lines join together to form a single line, they represent a cliff. However, if the change in elevation isn’t great enough to call for another contour line, the cliff may not appear on the map.
Ridges: A ridge is a chain of mountains or hills that create a continuous summit for an extended distance. These can be identified by V or U shaped contours that point towards lower elevation.
Saddles: A saddle is a low spot between two higher points of elevation, and on a topographic map, they appear as hourglass shaped contour lines.
USEPA/elevatr: An R package for accessing elevation data
An R package for accessing elevation data
Spatial data with terra — R Spatial
401 Unauthorized vs 403 Forbidden
Find the key differences between HTTP status codes 401 Unauthorized and 403 Forbidden with tabular comparison including when to use each in API development, with practical examples.
401 Unauthorized vs 403 Forbidden
In web development, ensuring access control is essential in safely and efficiently managing APIs. The meanings of 401 Unauthorized and 403 Forbidden are sometimes confused. Nonetheless, both codes have to do with restricted resources, but they serve different purposes. In this article, we will explain the codes and instruct you on which one to use.
401 Unauthorized?
The response is an HTTP error code for a request lacking valid authentication credentials from a client is referred to as the 401 Unauthorized status code. That being said, it means that before accessing the requested resource, it’s necessary for the server to authenticate itself to the client. If no credentials are provided or if wrong ones are given by the client, then what follows is a 401 status code.
When to Use 401 Unauthorized
Use 401 Unauthorized when:
No authentication details have been received yet from the client.
The authentication information supplied – username and password/token – is not valid/has expired.
There is no authorization header present in your requests like “Authorization.”
For instance, if an API demands Bearer token for access but this token has not been included in any request or is incorrect it will issue back a response having HTTP status code 401 Unauthorized (the most common case).
403 Forbidden?
The reason for using a 403 Forbidden status code is when the server recognizes the request, the client has been authenticated, but the client does not have permission to access the requested resource. It means that in this case, a client is known while a server intentionally turns down fulfilling the request because of inadequate privileges.
When to Use 403 Forbidden
Use 403 Forbidden when:
Authenticated clientele lack sufficient permissions to reach given resources.
Server denies resource access irrespective of client’s authentication state.
Client’s access to resources is prohibited by any form of an access control system.
For instance, an authorized user may try accessing an admin only page without having adequate role. Even if one gets logged in, the response will indicate 403 Forbidden if they do not have sufficient rights.
blockr/inst/shinylive/tools.R at main · BristolMyersSquibb/blockr
Composable, extensible no-code UI
How to Easily Capture and Test Code Output in R
Learn methods to capture and test code output in R, including snapshot testing, dput, and constructive package.
Databot is not a flotation device - Posit
Databot is an exciting new LLM tool for exploratory data analysis, but to use it safely and effectively, you still need the critical skills of a data scientist.
Lightweight Object-Relational Mapper for R
oRm is a lightweight Object-Relational Mapper (ORM) for R. It simplifies database interactions by allowing users to define table models, insert and query records, and establish relationships between models without writing raw SQL. oRm uses a combination of DBI, dbplyr, and R6 to provide compatibility with most database dialects.
34 🏗 R6 – Shiny App-Packages
Getting your app into an R package
Distributing Extensions – Quarto
Lua API Reference – Quarto
Lua Development – Quarto
Creating Shortcodes – Quarto
Shortcodes – Quarto
Guide – Quarto
Comprehensive guide to using Quarto. If you are just starting out, you may want to explore the tutorials to learn the basics.
Client for 'toxiproxy'
Create chaotic proxies for services to test behaviour of
your clients and servers in the face of unreliable network
connections. Provides a client to the 'toxiproxy' API.
A personal history of the tidyverse