Package 'epinowcast'

Description

This function calculates and adds the maximum observed delay for each group and reference date in the provided dataset. It first checks the validity of the observation indicator and then computes the maximum delay. If an observation indicator is provided, it further adjusts the maximum observed delay for unobserved data to be negative 1 (indicating no maximum observed).

Usage

add_max_observed_delay(new_confirm, observation_indicator = NULL)

Arguments

Value

A data.table with the original columns of new_confirm and an additional "max_obs_delay" column representing the maximum observed delay for each group and reference date. If an observation indicator is provided, unobserved data will have a "max_obs_delay" value of -1.

See Also

Helper functions for model modules add_pmfs(), convolution_matrix(), enw_dayofweek_structural_reporting(), enw_reference_by_report(), enw_reps_with_complete_refs(), enw_structural_reporting_metadata(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix()

Description

This function allows the addition of probability mass functions (PMFs) to produce a new PMF. This is useful for example in the context of reporting delays where the PMF of the sum of two Poisson distributions is the convolution of the PMFs.

Usage

add_pmfs(pmfs)

Arguments

Value

A vector describing the probability mass function of the sum of the

See Also

Helper functions for model modules add_max_observed_delay(), convolution_matrix(), enw_dayofweek_structural_reporting(), enw_reference_by_report(), enw_reps_with_complete_refs(), enw_structural_reporting_metadata(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix()

Examples

# Sample and analytical PMFs for two Poisson distributions
x <- rpois(10000, 5)
xpmf <- dpois(0:20, 5)
y <- rpois(10000, 7)
ypmf <- dpois(0:20, 7)
# Add sampled Poisson distributions up to get combined distribution
z <- x + y
# Analytical convolution of PMFs
conv_pmf <- add_pmfs(list(xpmf, ypmf))
conv_cdf <- cumsum(conv_pmf)
# Empirical convolution of PMFs
cdf <- ecdf(z)(0:42)
# Compare sampled and analytical CDFs
plot(conv_cdf)
lines(cdf, col = "black")

Autoregressive alias for arima()

Description

Thin wrapper around arima() that fixes d = 0 and q = 0. Matches the in-formula ar() helper that brms users will be familiar with. Equivalent to arima(time, by, p = p, d = 0, q = 0).

Usage

ar(time, by, p = 1)

Arguments

Value

An enw_arima_term interpretable by construct_arima().

See Also

Functions used to help convert formulas into model designs arima(), arima_terms(), arma(), as_string_formula(), construct_arima(), construct_gp(), construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

ar(time)
ar(time, location, p = 2)

Description

A call to arima() can be used in the formula argument of model construction functions in the epinowcast package such as enw_formula(). It declares an ARIMA(p, d, q) latent series indexed by time (and optionally a grouping variable by) whose value at each observation is added to the linear predictor. As with rw(), arguments are not evaluated; they are passed by name for use in model construction. Setting p = d = q = 0 is not allowed; use rw() (equivalent to arima(time, d = 1)) for a random walk.

Usage

arima(time, by, p = 1, d = 0, q = 0)

Arguments

Value

A list of class enw_arima_term describing the ARIMA term, interpretable by construct_arima(). Each group draws an independent shock series; phi, theta, and sigma are shared across groups (per-group parameters are a planned extension).

See Also

Functions used to help convert formulas into model designs ar(), arima_terms(), arma(), as_string_formula(), construct_arima(), construct_gp(), construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

arima(time)
arima(time, location)
arima(time, location, p = 2, d = 1, q = 1)

Description

This function extracts ARIMA terms from a formula so that they can be processed on their own. Matches all four user-facing helpers that produce an enw_arima_term: arima(), plus the convenience aliases ar(), ma(), and arma().

Usage

arima_terms(formula)

Arguments

Value

A character vector containing the ARIMA terms identified in the supplied formula.

See Also

Functions used to help convert formulas into model designs ar(), arima(), arma(), as_string_formula(), construct_arima(), construct_gp(), construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

epinowcast:::arima_terms(~ 1 + age_group + arima(week))
epinowcast:::arima_terms(~ 1 + ar(week, p = 2))
epinowcast:::arima_terms(~ 1 + arma(week, location, p = 1, q = 1))

ARMA alias for arima()

Description

Thin wrapper around arima() that fixes d = 0. Equivalent to arima(time, by, p = p, d = 0, q = q). For an integrated (random-walk) series use rw() or arima(time, by, p = 0, d = 1, q = 0) directly.

Usage

arma(time, by, p = 1, q = 1)

Arguments

Value

An enw_arima_term interpretable by construct_arima().

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), as_string_formula(), construct_arima(), construct_gp(), construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

arma(time)
arma(time, location, p = 1, q = 1)

Description

This function is used to convert an epinowcast as returned by epinowcast() object to a forecast_sample object which can be used for scoring using the scoringutils package.

Usage

## S3 method for class 'epinowcast'
as_forecast_sample(data, latest_obs, ...)

Arguments

Value

A forecast_sample object as returned by scoringutils::as_forecast_sample()

Examples

library(scoringutils)

nowcast <- enw_example("nowcast")
latest_obs <- enw_example("observations")
as_forecast_sample(nowcast, latest_obs)

Description

Converts formulas to strings

Usage

as_string_formula(formula)

Arguments

Value

A character string of the supplied formula

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), arma(), construct_arima(), construct_gp(), construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

epinowcast:::as_string_formula(~ 1 + age_group)

Description

Build the ord_obs data.table.

Usage

build_ord_obs(obs, max_delay, internal_timestep, timestep, nowcast = NULL)

Arguments

Value

A data.table.

See Also

Functions used for postprocessing of model fits enw_add_latest_obs_to_nowcast(), enw_nowcast_samples(), enw_nowcast_summary(), enw_posterior(), enw_pp_summary(), enw_quantiles_to_long(), enw_summarise_samples(), subset_obs()

Description

This function checks the sparsity of a design matrix and provides a recommendation if the matrix is considered sparse.

Usage

check_design_matrix_sparsity(
  matrix,
  sparsity_threshold = 0.9,
  min_matrix_size = 50,
  name = "checked"
)

Arguments

Value

This function is used for its side effect of providing an informational message if the matrix is sparse. It returns NULL invisibly.

See Also

Functions used for checking inputs check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

Check observations for reserved grouping variables

Usage

check_group(obs)

Arguments

Value

The obs object, which will be modifiable in place.

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

This function checks that the input data is stratified by reference_date, report_date, and .group. It does this by counting the number of observations for each combination of these variables, and throwing a warning if any combination has more than one observation.

Usage

check_group_date_unique(obs)

Arguments

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

Check if maximum delay specified by the user is long enough and raise potential warnings. This is achieved by computing the share of reference dates where the cumulative case count is below some aspired coverage.

Usage

check_max_delay(
  data,
  max_delay = data$max_delay,
  cum_coverage = 0.8,
  maxdelay_quantile_outlier = 0.97,
  warn = TRUE,
  warn_internal = FALSE
)

Arguments

Details

When data is very sparse (e.g., predominantly zero counts), the function may not be able to compute meaningful coverage statistics. In such cases, a warning is issued and the function treats the data as having no coverage issues. This typically occurs when groups have very few non-zero observations or when the specified max_delay is too large relative to available data.

The coverage is with respect to the maximum observed case count for the corresponding reference date. As the maximum observed case count is likely smaller than the true overall case count for not yet fully observed reference dates (due to right truncation), only reference dates that are more than the maximum observed delay ago are included. Still, because we can only use the maximum observed delay, not the unknown true maximum delay, the computed coverage values should be interpreted with care, as they are only proxies for the true coverage.

Value

A data.table with the share of reference dates where the cumulative case count is below cum_coverage, stratified by group.

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Examples

pobs <- enw_example(type = "preprocessed_observations")
check_max_delay(pobs, max_delay = 20, cum_coverage = 0.8)

Description

Check a model module contains the required components

Usage

check_module(module)

Arguments

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

Check that model modules have compatible specifications

Usage

check_modules_compatible(modules)

Arguments

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

This function verifies if the difference in numeric dates in the provided observations corresponds to the provided timestep.

Usage

check_numeric_timestep(dates, date_var, timestep, exact = TRUE)

Arguments

Value

This function is used for its side effect of stopping if the check fails. If the check passes, the function returns invisibly.

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

This function verifies if the observation_indicator within the provided new_confirm observations is logical. The check is performed to ensure that the observation_indicator is of the correct type.

Usage

check_observation_indicator(new_confirm, observation_indicator = NULL)

Arguments

Value

This function is used for its side effect of checking the observation indicator in new_confirm. If the check passes, the function returns invisibly. Otherwise, it stops and returns an error message.

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_quantiles(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

Check required quantiles are present

Usage

check_quantiles(posterior, req_probs = c(0.5, 0.95, 0.2, 0.8))

Arguments

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_timestep(), check_timestep_by_date(), check_timestep_by_group()

Description

This function verifies if the difference in dates in the provided observations corresponds to the provided timestep. If the exact argument is set to TRUE, the function checks if all differences exactly match the timestep; otherwise, it checks if the sum of the differences modulo the timestep equals zero. If the check fails, the function stops and returns an error message.

Usage

check_timestep(
  obs,
  date_var,
  timestep = "day",
  exact = TRUE,
  check_nrow = TRUE
)

Arguments

Value

This function is used for its side effect of stopping if the check fails. If the check passes, the function returns invisibly.

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group()

Description

This function verifies if the difference in dates within each date in the provided observations corresponds to the provided timestep. This check is performed for both report_date and reference_date and for each group in obs.

Usage

check_timestep_by_date(obs, timestep = "day", exact = TRUE)

Arguments

Value

This function is used for its side effect of checking the timestep by date in obs. If the check passes for all dates, the function returns invisibly. Otherwise, it stops and returns an error message.

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_group()

Description

This function verifies if the difference in dates within each group in the provided observations corresponds to the provided timestep. This check is performed for the specified date_var and for each group in obs.

Usage

check_timestep_by_group(obs, date_var, timestep = "day", exact = TRUE)

Arguments

Value

This function is used for its side effect of checking the timestep by group in obs. If the check passes for all groups, the function returns invisibly. Otherwise, it stops and returns an error message.

See Also

Functions used for checking inputs check_design_matrix_sparsity(), check_group(), check_group_date_unique(), check_max_delay(), check_module(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep(), check_timestep_by_date()

Description

Provides consistent coercion of inputs to IDate with error handling

Usage

coerce_date(dates = NULL)

Arguments

Details

If any of the elements of dates cannot be coerced, this function will result in an error, indicating all indices which cannot be coerced to IDate.

Internal methods of epinowcast assume dates are represented as IDate.

Value

An IDate vector.

See Also

Utility functions coerce_dt(), date_to_numeric_modulus(), enw_get_data(), enw_rolling_sum(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Examples

# works
coerce_date(c("2020-05-28", "2020-05-29"))
# does not, indicates index 2 is problem
tryCatch(
  coerce_date(c("2020-05-28", "2020-o5-29")),
  error = function(e) {
    print(e)
  }
)

Description

Provides consistent coercion of inputs to data.table with error handling, column checking, and optional selection.

Usage

coerce_dt(
  data,
  select = NULL,
  required_cols = select,
  forbidden_cols = NULL,
  group = FALSE,
  dates = FALSE,
  copy = TRUE,
  msg_required = "The following columns are required: ",
  msg_forbidden = "The following columns are forbidden: "
)

Arguments

Details

This function provides a single-point function for getting a "local" version of data provided by the user, in the internally used data.table format. It also enables selectively copying versus not, as well as checking for the presence and/or absence of various columns.

While it is intended to address garbage in from the user, it does not generally attempt to address garbage in from the developer - e.g. if asking for overlapping required and forbidden columns (though that will lead to an always-error condition).

When dates = TRUE, this function ensures that report_date and reference_date columns are coerced to IDate class with integer storage mode. This is necessary because some operations (such as dplyr::filter()) can convert IDate columns to double storage mode whilst preserving the class, which violates data.table's requirements and causes errors in subsequent date arithmetic operations.

Value

A data.table; the returned object will be a copy, unless copy = FALSE, in which case modifications are made in-place

See Also

Utility functions coerce_date(), date_to_numeric_modulus(), enw_get_data(), enw_rolling_sum(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Description

Takes an ARIMA term as defined by arima() and returns the metadata required to wire the term into a Stan model. Unlike construct_rw(), this does not modify the data or produce design matrix columns; ARIMA latent residuals enter the linear predictor through a parameter-dependent kernel applied to unit-normal shocks (see inst/stan/functions/arima_kernel.stan).

Usage

construct_arima(arima, data)

Arguments

Value

A list with the following elements:

• time, by, p, d, q: passed through from the arima() term.

• T: number of distinct time points in the series.

• G: number of groups (1 if by is unspecified).

• time_idx: integer vector mapping each row of data to a time index in 1:T.

• group_idx: integer vector mapping each row of data to a group index in 1:G.

• time_vals, group_levels: lookup vectors so the indices can be inverted.

• name: a label for the term, suitable as a parameter prefix.

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), arma(), as_string_formula(), construct_gp(), construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

data <- enw_example("preproc")$metareference[[1]]
epinowcast:::construct_arima(arima(week), data)
epinowcast:::construct_arima(
  arima(week, day_of_week, p = 2, d = 1), data
)

Description

Takes a Gaussian process term as defined by gp() and returns the metadata required to wire the term into a Stan model. Like construct_arima(), this does not modify the data or produce design matrix columns; the Gaussian process enters the linear predictor through a Hilbert-space reduced-rank approximation (see inst/stan/functions/gaussian_process.stan).

Usage

construct_gp(gp, data)

Arguments

Value

A list with the following elements:

• time, by, kernel, gp_type, nu, d, basis_prop, boundary_scale: passed through from the gp() term.

• T: number of distinct time points in the integrated series.

• G: number of groups (1 if by is unspecified).

• M: number of basis functions, ceiling(basis_prop * (T - d)).

• PHI: the ⁠(T - d) x M⁠ basis matrix. For d >= 1 the basis is built on the T - d free values that are integrated d times in Stan; the first d values of the realisation are anchored to zero.

• time_idx, group_idx: per-observation lookup indices.

• time_vals, group_levels: lookup vectors so the indices can be inverted.

• name: a label for the term, suitable as a parameter prefix.

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), arma(), as_string_formula(), construct_arima(), construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

data <- enw_example("preproc")$metareference[[1]]
epinowcast:::construct_gp(gp(week), data)
epinowcast:::construct_gp(gp(week, day_of_week, kernel = "se"), data)

Description

Constructs random effect terms

Usage

construct_re(re, data)

Arguments

Value

A list containing the transformed data ("data"), fixed effects terms ("terms") and a data.frame specifying the random effect structure between these terms (effects). Note that if the specified random effect was not a factor it will have been converted into one.

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), arma(), as_string_formula(), construct_arima(), construct_gp(), construct_rw(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

# Simple examples
form <- epinowcast:::parse_formula(~ 1 + (1 | day_of_week))
data <- enw_example("prepr")$metareference[[1]]
random_effect <- re(form$random[[1]])
epinowcast:::construct_re(random_effect, data)

# A more complex example
form <- epinowcast:::parse_formula(
  ~ 1 + disp + (1 + gear | cyl) + (0 + wt | am)
)
random_effect <- re(form$random[[1]])
epinowcast:::construct_re(random_effect, mtcars)

random_effect2 <- re(form$random[[2]])
epinowcast:::construct_re(random_effect2, mtcars)

Description

This function takes random walks as defined by rw(), produces the required additional variables (denoted using a "c" prefix and constructed using enw_add_cumulative_membership()), and then returns the extended data.frame along with the new fixed effects and the random effect structure.

Usage

construct_rw(rw, data)

Arguments

Value

A list containing the following:

• data: The input data.frame with the addition of the new variables required by the specified random walk. These are added using enw_add_cumulative_membership(). -terms: A character vector of new fixed effects terms to add to a model formula.

• effects: A data.frame describing the random effect structure of the new effects.

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), arma(), as_string_formula(), construct_arima(), construct_gp(), construct_re(), enw_formula(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

data <- enw_example("preproc")$metareference[[1]]

epinowcast:::construct_rw(rw(week), data)

epinowcast:::construct_rw(rw(week, day_of_week), data)

Description

This function allows the construction of convolution matrices which can be be combined with a vector of primary events to produce a vector of secondary events for example in the form of a renewal equation or to simulate reporting delays. Time-varying delays are supported as well as distribution padding (to allow for use in renewal equation like approaches).

Usage

convolution_matrix(dist, t, include_partial = FALSE)

Arguments

Value

A matrix with each column indicating a primary event and each row indicating a secondary event.

See Also

Helper functions for model modules add_max_observed_delay(), add_pmfs(), enw_dayofweek_structural_reporting(), enw_reference_by_report(), enw_reps_with_complete_refs(), enw_structural_reporting_metadata(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix()

Examples

# Simple convolution matrix with a static distribution
convolution_matrix(c(1, 2, 3), 10)
# Include partially reported convolutions
convolution_matrix(c(1, 2, 3), 10, include_partial = TRUE)
# Use a list of distributions
convolution_matrix(rep(list(c(1, 2, 3)), 10), 10)
# Use a time-varying list of distributions
convolution_matrix(c(rep(list(c(1, 2, 3)), 10), list(c(4, 5, 6))), 11)

Description

This function processes a date column in a data.table, converting it to a numeric representation and then computing the modulus with the provided timestep.

Usage

date_to_numeric_modulus(dt, date_column, timestep)

Arguments

Value

A modified data.table with two new columns: one for the numeric representation of the date minus the minimum date and another for its modulus with the timestep.

See Also

Utility functions coerce_date(), coerce_dt(), enw_get_data(), enw_rolling_sum(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Description

Calculate cumulative reported cases from incidence of new reports

Usage

enw_add_cumulative(obs, by = NULL, copy = TRUE)

Arguments

Value

The input data.frame with a new variable confirm.

See Also

Data converters enw_add_incidence(), enw_aggregate_cumulative(), enw_incidence_to_linelist(), enw_linelist_to_incidence()

Examples

# Default reconstruct incidence
dt <- germany_covid19_hosp[location == "DE"][age_group == "00+"]
dt <- enw_filter_reference_dates_by_report_start(dt)
dt <- enw_add_incidence(dt)
dt <- dt[, confirm := NULL]
enw_add_cumulative(dt)

# Make use of maximum reported to calculate empirical daily reporting
enw_add_cumulative(dt)

Description

This function adds a cumulative membership effect to a data frame. This is useful for specifying models such as random walks (using rw()) where these features can be used in the design matrix with the appropriate formula. Supports grouping via the optional .group column. Note that cumulative membership is indexed to start with zero (i.e. the first observation is assigned a cumulative membership of zero).

Usage

enw_add_cumulative_membership(metaobs, feature, copy = TRUE)

Arguments

Value

A data.frame with a new columns ⁠cfeature$⁠ that contain the cumulative membership effect for each value of feature. For example if the original feature was week (with numeric entries ⁠1, 2, 3⁠) then the new columns will be cweek1, cweek2, and cweek3.

See Also

Functions used to formulate models enw_add_pooling_effect(), enw_design(), enw_effects_metadata(), enw_one_hot_encode_feature()

Examples

metaobs <- data.frame(week = 1:2)
enw_add_cumulative_membership(metaobs, "week")

metaobs <- data.frame(week = 1:3, .group = c(1,1,2))
enw_add_cumulative_membership(metaobs, "week")

Description

This helper function takes a data.frame or data.table of observations and adds the delay (numeric, in days) between reference_date and report_date for each observation.

Usage

enw_add_delay(obs, timestep = "day", copy = TRUE)

Arguments

Value

A data.table of observations with a new column delay.

See Also

Preprocessing functions enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

obs <- data.frame(report_date = as.Date("2021-01-01") + -2:0)
obs$reference_date <- as.Date("2021-01-01")
enw_add_delay(obs)

Description

Computes incident counts from cumulative reports. Users should typically call enw_filter_reference_dates_by_report_start() before this function to remove reference dates that precede the earliest report date, which would otherwise produce spurious leading entries.

Usage

enw_add_incidence(obs, set_negatives_to_zero = TRUE, by = NULL, copy = TRUE)

Arguments

Value

The input data.frame with a new variable new_confirm. If max_confirm is present in the data.frame, then the proportion reported on each day (prop_reported) will also be added.

See Also

Data converters enw_add_cumulative(), enw_aggregate_cumulative(), enw_incidence_to_linelist(), enw_linelist_to_incidence()

Examples

# Default reconstruct incidence
dt <- germany_covid19_hosp[location == "DE"][age_group == "00+"]
dt <- enw_filter_reference_dates_by_report_start(dt)
enw_add_incidence(dt)

# Make use of maximum reported to calculate empirical
# daily reporting
dt <- germany_covid19_hosp[location == "DE"][
  age_group == "00+"
]
dt <- enw_add_max_reported(dt)
dt <- enw_filter_reference_dates_by_report_start(dt)
enw_add_incidence(dt)

Description

Add the latest observations to the nowcast output. This is useful for plotting the nowcast against the latest observations.

Usage

enw_add_latest_obs_to_nowcast(nowcast, obs)

Arguments

Value

A data.frame of nowcast output with the latest observations added.

See Also

Functions used for postprocessing of model fits build_ord_obs(), enw_nowcast_samples(), enw_nowcast_summary(), enw_posterior(), enw_pp_summary(), enw_quantiles_to_long(), enw_summarise_samples(), subset_obs()

Examples

fit <- enw_example("nowcast")
obs <- enw_example("obs")
nowcast <- summary(fit, type = "nowcast")
enw_add_latest_obs_to_nowcast(nowcast, obs)

Description

This is a helper function which adds the maximum (in the sense of latest observed) number of reported cases for each reference_date and computes the proportion of already reported cases for each combination of reference_date and report_date.

Usage

enw_add_max_reported(obs, copy = TRUE)

Arguments

Value

A data.table with new columns max_confirm and cum_prop_reported. max_confirm is the maximum number of cases reported for a certain reference_date. cum_prop_reported is the proportion of cases for a certain reference_date that are reported until a given report_day, relative to all cases so far observed for this reference_date.

See Also

Preprocessing functions enw_add_delay(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

obs <- data.frame(report_date = as.Date("2021-01-01") + 0:2)
obs$reference_date <- as.Date("2021-01-01")
obs$confirm <- 1:3
enw_add_max_reported(obs)

Description

If not already present, annotates time series data with metadata commonly used in models: day of week, and days, weeks, and months since start of time series.

Usage

enw_add_metaobs_features(
  metaobs,
  holidays = NULL,
  holidays_to = "Sunday",
  datecol = "date"
)

Arguments

Details

Effects models often need to include covariates for time-based features, such as day of the week (e.g. to reflect different care-seeking and/or reporting behaviour).

This function is called from within enw_preprocess_data() to systematically annotate metaobs with these commonly used metadata, if not already present.

However, it can also be used directly on other data.

Value

A copy of the metaobs input, with additional columns:

• day_of_week, a factor of values as output from weekdays() and possibly as holiday_to if distinct from weekdays values

• day, numeric, 0 based from start of time series

• week, numeric, 0 based from start of time series

• month, numeric, 0 based from start of time series

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

# make some example date
nat_germany_hosp <- subset(
  germany_covid19_hosp,
  location == "DE" & age_group == "80+"
)[1:40]

basemeta <- enw_add_metaobs_features(
  nat_germany_hosp,
  datecol = "report_date"
)
basemeta

# with holidays - n.b.: holidays not found are silently ignored
holidaymeta <- enw_add_metaobs_features(
  nat_germany_hosp,
  datecol = "report_date",
  holidays = c(
    "2021-04-04", "2021-04-05",
    "2021-05-01", "2021-05-13",
    "2021-05-24"
  ),
  holidays_to = "Holiday"
)
holidaymeta
subset(holidaymeta, day_of_week == "Holiday")

Description

This function adds a pooling effect to the metadata returned by enw_effects_metadata(). It does this updating the fixed column to 0 for the effects that match the string argument and adding a new column var_name that is 1 for the effects that match the string argument and 0 otherwise.

Usage

enw_add_pooling_effect(effects, var_name = "sd", finder_fn = startsWith, ...)

Arguments

Value

A data.table with the following columns:

• effects: the name of the effect

• fixed: a logical indicating whether the effect is fixed (1) or random (0).

• Argument supplied to var_name: a logical indicating whether the effect should be pooled (1) or not (0).

See Also

Functions used to formulate models enw_add_cumulative_membership(), enw_design(), enw_effects_metadata(), enw_one_hot_encode_feature()

Examples

data <- data.frame(a = 1:3, b = as.character(1:3), c = c(1,1,2))
design <- enw_design(a ~ b + c, data)$design
effects <- enw_effects_metadata(design)
enw_add_pooling_effect(effects, prefix = "b")

Description

This function aggregates observations over a specified timestep, ensuring alignment on the same day of week for report and reference dates. It is useful for aggregating data to a weekly timestep, for example which may be desirable if testing using a weekly timestep or if you are very concerned about runtime. Note that the start of the timestep will be determined by min_date + a single timestep (i.e. the first timestep will be "2022-10-23" if the minimum reference date is "2022-10-16"). Observations where the report dates do not form a complete timestep will be dropped from the aggregated output.

Usage

enw_aggregate_cumulative(
  obs,
  timestep = "day",
  by = NULL,
  min_reference_date = min(obs$reference_date, na.rm = TRUE),
  copy = TRUE
)

Arguments

Value

A data.table with aggregated observations.

See Also

Data converters enw_add_cumulative(), enw_add_incidence(), enw_incidence_to_linelist(), enw_linelist_to_incidence()

Examples

nat_hosp <- germany_covid19_hosp[location == "DE"][age_group == "00+"]
enw_aggregate_cumulative(nat_hosp, timestep = "week")

Description

Assign a group to each row of a data.table. If by is specified, then each unique combination of the columns in by will be assigned a unique group. If by is not specified, then all rows will be assigned to the same group.

Usage

enw_assign_group(obs, by = NULL, copy = TRUE)

Arguments

Value

A data.table with a .group column added ordered by .group and the existing key of obs.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

obs <- data.frame(x = 1:3, y = 1:3)
enw_assign_group(obs)
enw_assign_group(obs, by = "x")

Description

Ensures that all reference and report dates are present for all groups based on the maximum and minimum dates found in the data. This function may be of use to users when preprocessing their data. In general all features that you may consider using as grouping variables or as covariates need to be included in the by variable.

Usage

enw_complete_dates(
  obs,
  by = NULL,
  max_delay,
  min_date = min(obs$reference_date, na.rm = TRUE),
  max_date = max(obs$report_date, na.rm = TRUE),
  timestep = "day",
  missing_reference = TRUE,
  completion_beyond_max_report = FALSE,
  flag_observation = FALSE
)

Arguments

Value

A data.table with completed entries for all combinations of reference dates, groups and possible report dates.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

obs <- data.frame(
  report_date = c("2021-10-01", "2021-10-03"), reference_date = "2021-10-01",
  confirm = 1
)
enw_complete_dates(obs)

# Allow completion beyond the maximum date found in the data
enw_complete_dates(obs, completion_beyond_max_report = TRUE, max_delay = 10)

Description

This function is used internally by enw_preprocess_data() to combine various pieces of processed observed data into a single object. It is exposed to the user in order to allow for modular data preprocessing though this is not currently recommended. See documentation and code of enw_preprocess_data() for more on the expected inputs.

Usage

enw_construct_data(
  obs,
  new_confirm,
  latest,
  missing_reference,
  reporting_triangle,
  metareport,
  metareference,
  metadelay,
  max_delay,
  timestep,
  by
)

Arguments

Value

A data.table containing processed observations as a series of nested data.frames as well as variables containing metadata. These are:

• obs: (observations with the addition of empirical reporting proportions and restricted to the specified maximum delay).

• new_confirm: Incidence of notifications by reference and report date. Empirical reporting distributions are also added.

• latest: The latest available observations.

• missing_reference: Observations missing reference dates.

• reporting_triangle: Incident observations by report and reference date in the standard reporting triangle matrix format.

• metareference: Metadata reference dates derived from observations.

• metrareport: Metadata for report dates.

• metadelay: Metadata for reporting delays produced using enw_metadata_delay().

• max_delay: Maximum delay to be modelled by epinowcast.

• time: Numeric, number of timepoints in the data.

• snapshots: Numeric, number of available data snapshots to use for nowcasting.

• groups: Numeric, Number of groups/strata in the supplied observations (set using by).

• max_date: The maximum available report date.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

pobs <- enw_example("preprocessed")
enw_construct_data(
  obs = pobs$obs[[1]],
  new_confirm = pobs$new_confirm[[1]],
  latest = pobs$latest[[1]],
  missing_reference = pobs$missing_reference[[1]],
  reporting_triangle = pobs$reporting_triangle[[1]],
  metareport = pobs$metareport[[1]],
  metareference = pobs$metareference[[1]],
  metadelay = pobs$metadelay[[1]],
  max_delay = pobs$max_delay,
  timestep = pobs$timestep[[1]],
  by = c()
)

Description

Creates a structural reporting pattern for cases where reporting only occurs on specific days of the week (e.g., Wednesday-only reporting). This is a convenience function that builds on enw_structural_reporting_metadata().

Usage

enw_dayofweek_structural_reporting(pobs, day_of_week)

Arguments

Value

A data.table with columns:

• .group: Group identifier

• date: Reference date

• report_date: Report date

• report: Binary indicator (1 = reporting occurs, 0 = no reporting)

See Also

Helper functions for model modules add_max_observed_delay(), add_pmfs(), convolution_matrix(), enw_reference_by_report(), enw_reps_with_complete_refs(), enw_structural_reporting_metadata(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix()

Examples

## Not run: 
pobs <- enw_preprocess_data(obs, max_delay = 30)

# Wednesday-only reporting
enw_dayofweek_structural_reporting(
  pobs, day_of_week = "Wednesday"
)

# Multiple reporting days
enw_dayofweek_structural_reporting(
  pobs, day_of_week = c("Monday", "Wednesday", "Friday")
)

## End(Not run)

Description

Categorises incidence by delay group with empirical reporting proportions. Intended for use with the plot.enw_preprocess_data() visualisation types that show reporting patterns by delay.

Usage

enw_delay_categories(pobs, delay_group_thresh)

Arguments

Value

A data.table of notification incidence by reference date and delay group, including columns prop_reported and cum_prop_reported.

See Also

Plotting functions enw_delay_quantiles(), enw_plot_delay_counts(), enw_plot_delay_cumulative(), enw_plot_delay_fraction(), enw_plot_delay_quantiles(), enw_plot_nowcast_quantiles(), enw_plot_obs(), enw_plot_pp_quantiles(), enw_plot_quantiles(), enw_plot_theme(), plot.enw_preprocess_data(), plot.epinowcast()

Examples

pobs <- enw_example("preprocessed_observations")
enw_delay_categories(pobs, delay_group_thresh = c(0, 2, 5, 10, 21))

Description

Computes empirical quantiles of the reporting delay distribution for each reference date. Intended for use with the "delay_quantiles" plot type in plot.enw_preprocess_data().

Usage

enw_delay_quantiles(pobs, quantiles = c(0.1, 0.5, 0.9))

Arguments

Value

A data.table with columns for each quantile by reference date.

See Also

Plotting functions enw_delay_categories(), enw_plot_delay_counts(), enw_plot_delay_cumulative(), enw_plot_delay_fraction(), enw_plot_delay_quantiles(), enw_plot_nowcast_quantiles(), enw_plot_obs(), enw_plot_pp_quantiles(), enw_plot_quantiles(), enw_plot_theme(), plot.enw_preprocess_data(), plot.epinowcast()

Examples

pobs <- enw_example("preprocessed_observations")
enw_delay_quantiles(pobs)

Description

This function is a wrapper around stats::model.matrix() that can optionally return a sparse design matrix defined as the unique number of rows in the design matrix and an index vector that allows the full design matrix to be reconstructed. This is useful for models that have many repeated rows in the design matrix and that are computationally expensive to fit. This function also allows for the specification of contrasts for categorical variables.

Usage

enw_design(formula, data, no_contrasts = FALSE, sparse = TRUE, ...)

Arguments

Value

A list containing the formula, the design matrix, and the index.

See Also

Functions used to formulate models enw_add_cumulative_membership(), enw_add_pooling_effect(), enw_effects_metadata(), enw_one_hot_encode_feature()

Examples

data <- data.frame(a = 1:3, b = as.character(1:3), c = c(1,1,2))
enw_design(a ~ b + c, data)
enw_design(a ~ b + c, data, no_contrasts = TRUE)
enw_design(a ~ b + c, data, no_contrasts = c("b"))
enw_design(a ~ c, data, sparse = TRUE)
enw_design(a ~ c, data, sparse = FALSE)

Description

This function extracts metadata from a design matrix and returns a data.table with the following columns:

• effects: the name of the effect

• fixed: a logical indicating whether the effect is fixed (1) or random (0).

It automatically drops the intercept (defined as "(Intercept)").

This function is useful for constructing a model design object for random effects when used in combination with ewn_add_pooling_effect.

Usage

enw_effects_metadata(design)

Arguments

Value

A data.table with the following columns:

• effects: the name of the effect

• fixed: a logical indicating whether the effect is fixed (1) or random (0)

See Also

Functions used to formulate models enw_add_cumulative_membership(), enw_add_pooling_effect(), enw_design(), enw_one_hot_encode_feature()

Examples

data <- data.frame(a = 1:3, b = as.character(1:3), c = c(1,1,2))
design <- enw_design(a ~ b + c, data)$design
enw_effects_metadata(design)

Description

Loads examples of nowcasts produced using example scripts. Used to streamline examples, in package tests and to enable users to explore package functionality without needing to install cmdstanr.

Usage

enw_example(
  type = c("nowcast", "preprocessed_observations", "observations", "script")
)

Arguments

Value

Depending on type, a data.table of the requested output OR the file name(s) to generate these outputs (type = "script")

See Also

Package data sets germany_covid19_hosp

Examples

# Load the nowcast
enw_example(type = "nowcast")

# Load the preprocessed observations
enw_example(type = "preprocessed_observations")

# Load the latest observations
enw_example(type = "observations")

# Load the script used to generate these examples
# Optionally source this script to regenerate the example
readLines(enw_example(type = "script"))

Description

Expectation model module

Usage

enw_expectation(
  r = ~0 + (1 | day:.group),
  generation_time = 1,
  observation = ~1,
  latent_reporting_delay = 1,
  data,
  ...
)

Arguments

Value

A list containing the supplied formulas, data passed into a list describing the models, a data.frame describing the priors used, and a function that takes the output data and priors and returns a function that can be used to sample from a tightened version of the prior distribution.

See Also

Model modules enw_fit_opts(), enw_missing(), enw_obs(), enw_reference(), enw_report()

Examples

enw_expectation(data = enw_example("preprocessed"))

Description

Extend a time series with additional dates. This is useful when extending the report dates of a time series to include future dates for nowcasting purposes or to include additional dates for backcasting when using a renewal process as the expectation model.

Usage

enw_extend_date(
  metaobs,
  days = 20,
  direction = c("end", "start"),
  timestep = "day"
)

Arguments

Value

A data.table with the same columns as metaobs but with additional rows for each date in the range of date to date + days (or date - days if direction = "start"). An additional variable observed is added with a value of FALSE for all new dates and TRUE for all existing dates.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

metaobs <- data.frame(date = as.Date("2021-01-01") + 0:4)
enw_extend_date(metaobs, days = 2)
enw_extend_date(metaobs, days = 2, direction = "start")

Description

Filter observations to have a consistent maximum delay period

Usage

enw_filter_delay(obs, max_delay, timestep = "day")

Arguments

Value

A data.frame filtered so that dates by report are less than or equal the reference date plus the maximum delay.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

obs <- enw_example("preprocessed")$obs[[1]]
enw_filter_delay(obs, max_delay = 2)

Description

This is a helper function which allows users to filter datasets by reference date. This is useful, for example, when evaluating nowcast performance against fully observed data. Users may wish to combine this function with enw_filter_report_dates(). Note that by definition it is assumed that report dates must be equal or greater than the corresponding reference date (i.e a report cannot happen before the event being reported occurs). This means that this function will also filter out any report dates that are earlier than their corresponding reference date.

Usage

enw_filter_reference_dates(
  obs,
  earliest_date,
  include_days,
  latest_date,
  remove_days
)

Arguments

Details

The include_days parameter filters to include exactly the specified number of most recent reference dates. For example, if the latest reference date is 2021-10-20 and include_days = 10, the filtered data will contain reference dates from 2021-10-11 to 2021-10-20 (10 days inclusive).

Value

A data.table filtered by report date

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

# Filter by date
enw_filter_reference_dates(
  germany_covid19_hosp,
  earliest_date = "2021-09-01",
  latest_date = "2021-10-01"
)
#
# Filter by days
enw_filter_reference_dates(
  germany_covid19_hosp,
  include_days = 10, remove_days = 10
)

Description

Removes observations where the reference_date is earlier than the minimum report_date within each group. Rows with missing reference_date are retained. This is useful for ensuring that observations are only included from the first available report date onwards.

This function is typically called before enw_add_incidence() so that the incidence calculation starts from a valid reporting window. Without this step, reference dates that predate any report date produce spurious leading entries in the incidence output.

Usage

enw_filter_reference_dates_by_report_start(obs, by = NULL, copy = TRUE)

Arguments

Value

A data.table filtered so that each reference_date is on or after the minimum report_date in its group. Rows with NA reference_date are kept.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

library(data.table)
obs <- data.table(
  reference_date = as.IDate(c(
    "2021-10-01", "2021-10-02", "2021-10-03"
  )),
  report_date = as.IDate(c(
    "2021-10-02", "2021-10-02", "2021-10-03"
  ))
)
# The first row has reference_date before the minimum
# report_date, so it is removed
enw_filter_reference_dates_by_report_start(obs)

Description

This is a helper function which allows users to create truncated data sets at past time points from a given larger data set. This is useful when evaluating nowcast performance against fully observed data. Users may wish to combine this function with enw_filter_reference_dates().

Usage

enw_filter_report_dates(obs, latest_date, remove_days)

Arguments

Value

A data.table filtered by report date

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

# Filter by date
enw_filter_report_dates(germany_covid19_hosp, latest_date = "2021-09-01")

# Filter by days
enw_filter_report_dates(germany_covid19_hosp, remove_days = 10)

Description

Format model fitting options for use with stan

Usage

enw_fit_opts(
  sampler = epinowcast::enw_sample,
  nowcast = TRUE,
  pp = FALSE,
  likelihood = TRUE,
  likelihood_aggregation = c("snapshots", "groups"),
  threads_per_chain = 1L,
  debug = FALSE,
  output_loglik = FALSE,
  sparse_design = FALSE,
  ...
)

Arguments

Value

A list containing the specified sampler function, data as a list specifying the fitting options to use, and additional arguments to pass to the sampler function when it is called.

See Also

Model modules enw_expectation(), enw_missing(), enw_obs(), enw_reference(), enw_report()

Examples

# Default options along with settings to pass to enw_sample
enw_fit_opts(iter_sampling = 1000, iter_warmup = 1000)

Description

Flags observations based on the 'confirm' column. If the '.observed' column does not exist, it is created. Observations are flagged as observed (TRUE) if 'confirm' is not NA.

Usage

enw_flag_observed_observations(obs, copy = TRUE)

Arguments

Value

A data.table with an additional column '.observed' indicating observed observations.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

dt <- data.frame(id = 1:3, confirm = c(NA, 1, 2))
enw_flag_observed_observations(dt)

Description

This function allows models to be defined using a flexible formula interface that supports fixed effects, random effects (using lme4 syntax), and random walks. The formula syntax builds on standard R formula notation and extends it with lme4 style random effects and custom random walk terms. Users familiar with mixed models in lme4 or brms will recognise the syntax. Note that the returned fixed effects design matrix is sparse and so the index supplied is required to link observations to the appropriate design matrix row.

Usage

enw_formula(formula, data, sparse = TRUE)

Arguments

Details

Formula syntax overview

The formula interface supports three types of model components:

Fixed effects: Standard R formula syntax as used in stats::lm() and similar functions. For example:

• ~ 1: intercept only

• ~ age_group: intercept plus categorical predictor

• ~ age_group + location: multiple predictors

• ~ 0 + age_group: no intercept (contrasts)

Random effects: Uses lme4 syntax with vertical bar notation. Random effects allow parameters to vary by group whilst sharing information across groups through partial pooling. Note that epinowcast assumes independent standard deviations for random effects rather than correlated random effects as supported by lme4. For example:

• ~ 1 + (1 | location): random intercepts by location

• ~ 1 + age_group + (1 | location): fixed age effect with random location intercepts

• ~ (age_group | location): random slopes for age within each location

• ~ (1 + week | location:month): random intercepts and week effects for each location-month combination (using interaction to create independent random effects per strata)

Interactions (e.g., location:month) can be used on the right-hand side of the vertical bar to specify independent random effects for each combination of the interacting variables.

See the lme4 package documentation for more details on random effects syntax.

Random walks: Uses the rw() helper function to specify time-varying effects that evolve smoothly over time. For example:

• ~ rw(week): a random walk over weeks

• ~ rw(week, location): independent random walks for each location

• ~ rw(week, location): random walks with shared variance across locations (per-group variance is a planned extension)

ARIMA residuals: Uses the arima() helper to add an ARIMA(p, d, q) latent residual series to the linear predictor. Unlike rw(), the kernel that maps unit-normal shocks to the latent series depends on the autoregressive and moving-average parameters, so the term does not produce design-matrix columns; it carries lookup metadata that the Stan layer uses with the kernel from inst/stan/functions/arima_kernel.stan. For example:

• ~ arima(week): AR(1) on weekly residuals

• ~ arima(week, location, p = 2, d = 1, q = 1): ARIMA(2, 1, 1) driven by independent shocks per location, with phi, theta, and sigma shared across locations (per-group parameters are a planned extension)

• arima(time, d = 1, p = 0, q = 0) is equivalent to rw(time)

Convenience aliases match brms's in-formula vocabulary:

• ar(time, by, p) is arima(time, by, p = p, d = 0, q = 0)

• ma(time, by, q) is arima(time, by, p = 0, d = 0, q = q)

• arma(time, by, p, q) is arima(time, by, p = p, d = 0, q = q)

These four types of effects can be combined in a single formula, for example: ~ 1 + age_group + (1 | location) + rw(week, location) specifies fixed age effects, random location intercepts, and location-specific random walks over time.

Turning off model components

In epinowcast model specification functions (such as enw_reference(), enw_report(), enw_expectation()), formula arguments can be set to ~0 to disable that model component entirely. This is a package-specific convention. Note that when a formula is specified as ~0, it is typically converted internally to ~1 (intercept only) to ensure valid model structure, but the component is flagged as inactive.

How formulas map to the model

The formula you specify controls which covariates and effects enter the linear predictor of the model. For instance, in the reference date model (enw_reference()), the formula determines how reporting delay parameters vary by covariates and groups. The formula is converted to design matrices: a fixed effects matrix (which may be sparse for computational efficiency) and a random effects matrix that defines the hierarchical structure.

Value

A list containing the following:

• formula: The user supplied formula

• parsed_formula: The formula as parsed by parse_formula()

• expanded_formula: The flattened version of the formula with both user supplied terms and terms added for the user supplied complex model components.

• fixed: A list containing the fixed effect formula, sparse design matrix, and the index linking the design matrix with observations.

• random: A list containing the random effect formula, sparse design matrix, and the index linking the design matrix with random effects.

References

For users new to formula syntax in R:

• Fixed effects: See ?formula and the "Statistical Models in R" chapter of "An Introduction to R" at the URL:

https://cran.r-project.org/doc/manuals/r-release/R-intro.html#Statistical-models-in-R # nolint: line_length_linter

• Random effects: See the lme4 package documentation and vignettes.

• Mixed models: Bates et al. (2015) "Fitting Linear Mixed-Effects Models Using lme4". Journal of Statistical Software, 67(1), 1-48. doi:10.18637/jss.v067.i01

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), arma(), as_string_formula(), construct_arima(), construct_gp(), construct_re(), construct_rw(), enw_manual_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

# Use meta data for references dates from the Germany COVID-19
# hospitalisation data.
obs <- enw_filter_report_dates(
  germany_covid19_hosp[location == "DE"],
  remove_days = 40
)
obs <- enw_filter_reference_dates(obs, include_days = 40)
pobs <- enw_preprocess_data(
  obs, by = c("age_group", "location"), max_delay = 20
  )
data <- pobs$metareference[[1]]

# Intercept only
enw_formula(~ 1, data)

# Fixed effect
enw_formula(~ 1 + age_group, data)

# Random intercepts
enw_formula(~ 1 + (1 | age_group), data)

# Random walk
enw_formula(~ 1 + rw(week), data)

# Model with a random effect for age group and a random walk
enw_formula(~ 1 + (1 | age_group) + rw(week), data)

# Model defined without a sparse fixed effects design matrix
enw_formula(~1, data[1:20, ], sparse = FALSE)

# Model using an interaction in the right hand side of a random effect
# to specify an independent random effect per strata.
enw_formula(~ (1 + day | week:month), data = data)

Description

Format formula data for use with stan

Usage

enw_formula_as_data_list(formula, prefix, drop_intercept = FALSE)

Arguments

Value

A list defining the model formula. This includes:

• ⁠prefix_fintercept:⁠ Is an intercept present for the fixed effects design matrix.

• prefix_fdesign: The fixed effects design matrix

• prefix_fnrow: The number of rows of the fixed design matrix

• prefix_findex: The index linking design matrix rows to observations

• prefix_fnindex: The length of the index

• prefix_fncol: The number of columns (i.e effects) in the fixed effect design matrix (minus 1 if an intercept is present).

• prefix_rdesign: The random effects design matrix

• prefix_rncol: The number of columns (i.e random effects) in the random effect design matrix (minus 1 as the intercept is dropped).

• prefix_arima_present: 1 if the formula contains an arima() term, 0 otherwise.

• prefix_arima_T, prefix_arima_G: ARIMA series length and group count.

• prefix_arima_p, prefix_arima_d, prefix_arima_q: ARIMA orders.

• prefix_arima_flat_idx: per-observation column-major index into a ⁠(T x G)⁠ ARIMA residual matrix, used by Stan to gather residuals with to_vector(eps)[flat_idx].

• prefix_arima_n_obs: length of the lookup vectors.

See Also

Functions used to help convert models into the format required for stan enw_get_cache(), enw_model(), enw_pathfinder(), enw_priors_as_data_list(), enw_replace_priors(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()

Examples

f <- enw_formula(~ 1 + (1 | cyl), mtcars)
enw_formula_as_data_list(f, "mtcars")

# A missing formula produces the default list
enw_formula_as_data_list(prefix = "missing")

Description

Retrieves the user set cache location for Stan models. This path can be set through the enw_cache_location function call. If no environmental variable is available the output from tempdir() will be returned.

Usage

enw_get_cache()

Value

A string representing the file path for the cache location

See Also

Functions used to help convert models into the format required for stan enw_formula_as_data_list(), enw_model(), enw_pathfinder(), enw_priors_as_data_list(), enw_replace_priors(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()

Description

Extracts a named component from an enw_preprocess_data() or epinowcast() object. List columns are unwrapped automatically so you get the underlying data.table or value directly.

Usage

enw_get_data(x, name)

Arguments

Value

The extracted component. For list columns this is the first element (typically a data.table); for scalar columns the value is returned as-is.

See Also

Utility functions coerce_date(), coerce_dt(), date_to_numeric_modulus(), enw_rolling_sum(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Examples

pobs <- enw_example("preprocessed_observations")
enw_get_data(pobs, "obs")
enw_get_data(pobs, "max_delay")

Description

Imputes NA values in the 'confirm' column. NA values are replaced with the last available observation or 0.

Usage

enw_impute_na_observations(obs, by = NULL, copy = TRUE)

Arguments

Value

A data.table with imputed 'confirm' column where NA values have been replaced with zero.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

dt <- data.frame(
  id = 1:3, confirm = c(NA, 1, 2),
  reference_date = as.Date("2021-01-01")
)
enw_impute_na_observations(dt)

Description

This function takes a data.table of aggregate counts or something coercible to a data.table (such as a data.frame) and converts it to a line list where each row represents a case.

Usage

enw_incidence_to_linelist(
  obs,
  reference_date = "reference_date",
  report_date = "report_date"
)

Arguments

Value

A data.table with the following variables: id, reference_date, report_date, and any other variables in the obs object. Rows in obs will be duplicated based on the new_confirm column. reference_date and report_date may be renamed if reference_date and report_date are supplied.

See Also

Data converters enw_add_cumulative(), enw_add_incidence(), enw_aggregate_cumulative(), enw_linelist_to_incidence()

Examples

incidence <- enw_filter_reference_dates_by_report_start(
  germany_covid19_hosp
)
incidence <- enw_add_incidence(incidence)
incidence <- enw_filter_reference_dates(
  incidence[location == "DE"], include_days = 10
)
enw_incidence_to_linelist(incidence, reference_date = "onset_date")

Description

Filter observations for the latest available reported data for each reference date. Note this is not the same as filtering for the maximum report date in all cases as data may only be updated up to some maximum number of days.

Usage

enw_latest_data(obs)

Arguments

Value

A data.table of observations filtered for the latest available data for each reference date.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_metadata(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

# Filter for latest reported data
enw_latest_data(germany_covid19_hosp)

Description

This function takes a line list (i.e. tabular data where each row represents a case) and aggregates to a count (new_confirm) of cases by user-specified reference_dates and report_dates. This is enables the use of enw_preprocess_data() and other epinowcast() preprocessing functions.

Usage

enw_linelist_to_incidence(
  linelist,
  reference_date = "reference_date",
  report_date = "report_date",
  by = NULL,
  max_delay,
  completion_beyond_max_report = FALSE,
  copy = TRUE
)

Arguments

Value

A data.table with the following variables: reference_date, report_date, new_confirm, confirm, delay, and any variables specified in by.

See Also

Data converters enw_add_cumulative(), enw_add_incidence(), enw_aggregate_cumulative(), enw_incidence_to_linelist()

Examples

linelist <- data.frame(
  onset_date = as.Date(c("2021-01-02", "2021-01-03", "2021-01-02")),
  report_date = as.Date(c("2021-01-03", "2021-01-05", "2021-01-04"))
)
enw_linelist_to_incidence(linelist, reference_date = "onset_date")

# Specify a custom maximum delay and allow completion beyond the maximum
# observed delay
enw_linelist_to_incidence(
 linelist, reference_date = "onset_date", max_delay = 5,
 completion_beyond_max_report = TRUE
)

Description

For most typical use cases enw_formula() should provide sufficient flexibility to allow models to be defined. However, there may be some instances where more manual model specification is required. This function supports this by allowing the user to supply vectors of fixed, random, and customised random effects (where they are not first treated as fixed effect terms). Prior to ⁠1.0.0⁠ this was the main interface for specifying models and it is still used internally to handle some parts of the model specification process.

Usage

enw_manual_formula(
  data,
  fixed = NULL,
  random = NULL,
  custom_random = NULL,
  no_contrasts = FALSE,
  add_intercept = TRUE
)

Arguments

Value

A list specifying the fixed effects (formula, design matrix, and design matrix index), and random effects (formula and design matrix).

See Also

Functions used to help convert formulas into model designs ar(), arima(), arima_terms(), arma(), as_string_formula(), construct_arima(), construct_gp(), construct_re(), construct_rw(), enw_formula(), gp(), gp_terms(), ma(), parse_formula(), re(), remove_arima_terms(), remove_gp_terms(), remove_rw_terms(), rw(), rw_terms(), split_formula_to_terms()

Examples

data <- enw_example("prep")$metareference[[1]]
enw_manual_formula(data, fixed = "week", random = "day_of_week")

Description

Extract metadata from raw data, either by reference or by report date. For the target date chosen (reference or report), confirm, ⁠max_confirm``, and ⁠cum_prop_reported' are dropped and the first observation for each group and date is retained.

Usage

enw_metadata(obs, target_date = c("reference_date", "report_date"))

Arguments

Value

A data.table with columns:

• date, a Date column

• .group, a grouping column

and the first observation for each group and date. The data.table is sorted by .group and date.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

obs <- data.frame(
  reference_date = as.Date("2021-01-01"),
  report_date = as.Date("2022-01-01"), x = 1:10
)
enw_metadata(obs, target_date = "reference_date")

Description

Calculate delay metadata based on the supplied maximum delay and independent of other metadata or date indexing. These data are meant to be used in conjunction with metadata on the date of reference. Users can build additional features with this data.frame or regenerate it using this function in the output of enw_preprocess_data().

Usage

enw_metadata_delay(max_delay = 20, breaks = 4, timestep = "day")

Arguments

Value

A data.frame of delay metadata. This includes:

• delay: The numeric delay from reference date to report.

• delay_cat: The categorised delay. This may be useful for model building.

• delay_week: The numeric week since the delay was reported. This again may be useful for model building.

• delay_head: A logical variable defining if the delay is in the lower 25% of the potential delays. This may be particularly useful when building models that assume a parametric distribution in order to increase the weight of the head of the reporting distribution in a pragmatic way.

• delay_tail: A logical variable defining if the delay is in the upper 75% of the potential delays. This may be particularly useful when building models that assume a parametric distribution in order to increase the weight of the tail of the reporting distribution in a pragmatic way.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_missing_reference(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

enw_metadata_delay(max_delay = 20, breaks = 4)

Description

Missing reference data model module

Usage

enw_missing(formula = ~1, data)

Arguments

Value

A list containing the supplied formulas, data passed into a list describing the models, a data.frame describing the priors used, and a function that takes the output data and priors and returns a function that can be used to sample from a tightened version of the prior distribution.

See Also

Model modules enw_expectation(), enw_fit_opts(), enw_obs(), enw_reference(), enw_report()

Examples

# Missingness model with a fixed intercept only
enw_missing(data = enw_example("preprocessed"))

# No missingness model specified
enw_missing(~0, data = enw_example("preprocessed"))

Description

Returns reports with missing reference dates as well as calculating the proportion of reports for a given reference date that were missing.

Usage

enw_missing_reference(obs)

Arguments

Value

A data.table of missing counts and proportions by report date and group.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_reference_dates_by_report_start(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_metadata_delay(), enw_obs_at_delay(), enw_preprocess_data(), enw_reporting_triangle(), enw_reporting_triangle_to_long(), enw_retrospective()

Examples

obs <- data.frame(
  report_date = c("2021-10-01", "2021-10-03"), reference_date = "2021-10-01",
  confirm = 1
)
obs <- rbind(
  obs,
  data.frame(report_date = "2021-10-04", reference_date = NA, confirm = 4)
)
obs <- enw_complete_dates(obs)
obs <- enw_assign_group(obs)
obs <- enw_filter_reference_dates_by_report_start(obs)
obs <- enw_add_incidence(obs)
enw_missing_reference(obs)

Description

Load and compile the nowcasting model

Usage

enw_model(
  model = system.file("stan", "epinowcast.stan", package = "epinowcast"),
  include = system.file("stan", package = "epinowcast"),
  compile = TRUE,
  threads = TRUE,
  profile = FALSE,
  target_dir = epinowcast::enw_get_cache(),
  stanc_options = list(),
  cpp_options = list(),
  verbose = TRUE,
  ...
)

Arguments

Value

A cmdstanr model.

See Also

Functions used to help convert models into the format required for stan enw_formula_as_data_list(), enw_get_cache(), enw_pathfinder(), enw_priors_as_data_list(), enw_replace_priors(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()