{ "_id": "6a1b0d2c1d7bb097a0a0841e", "Package": "scoringutils", "Title": "Utilities for Scoring and Assessing Predictions", "Version": "2.2.0.9000", "Language": "en-GB", "Authors@R": "c(\nperson(given = \"Nikos\",\nfamily = \"Bosse\",\nrole = c(\"aut\", \"cre\"),\nemail = \"nikosbosse@gmail.com\",\ncomment = c(ORCID = \"0000-0002-7750-5280\")),\nperson(given = \"Sam\",\nfamily = \"Abbott\",\nrole = c(\"aut\"),\nemail = \"contact@samabbott.co.uk\",\ncomment = c(ORCID = \"0000-0001-8057-8037\")),\nperson(given = \"Hugo\",\nfamily = \"Gruson\",\nrole = c(\"aut\"),\nemail = \"hugo.gruson+R@normalesup.org\",\ncomment = c(ORCID = \"0000-0002-4094-1476\")),\nperson(given = \"Johannes\",\nfamily = \"Bracher\",\nrole = c(\"ctb\"),\nemail = \"johannes.bracher@kit.edu\",\ncomment = c(ORCID = \"0000-0002-3777-1410\")),\nperson(given = \"Toshiaki Asakura\",\nrole = c(\"ctb\"),\nemail = \"toshiaki.asa9ra@gmail.com\",\ncomment = c(ORCID = \"0000-0001-8838-785X\")),\nperson(given = \"James Mba\",\nfamily = \"Azam\",\nrole = c(\"ctb\"),\nemail = \"james.azam@lshtm.ac.uk\",\ncomment = c(ORCID = \"0000-0001-5782-7330\")),\nperson(\"Sebastian\", \"Funk\",\nemail = \"sebastian.funk@lshtm.ac.uk\",\nrole = c(\"aut\")),\nperson(given = \"Michael\",\nfamily = \"Chirico\",\nrole = c(\"ctb\"),\nemail = \"michaelchirico4@gmail.com\",\ncomment = c(ORCID = \"0000-0003-0787-087X\")))", "Description": "Facilitate the evaluation of forecasts in a convenient\nframework based on data.table. It allows user to to check their\nforecasts and diagnose issues, to visualise forecasts and\nmissing data, to transform data before scoring, to handle\nmissing forecasts, to aggregate scores, and to visualise the\nresults of the evaluation. The package mostly focuses on the\nevaluation of probabilistic forecasts and allows evaluating\nseveral different forecast types and input formats. Find more\ninformation about the package in the Vignettes as well as in\nthe accompanying paper, .", "License": "MIT + file LICENSE", "Encoding": "UTF-8", "LazyData": "true", "Config/Needs/website": "r-lib/pkgdown, amirmasoudabdol/preferably", "Config/testthat/edition": "3", "RoxygenNote": "7.3.3", "URL": "https://doi.org/10.48550/arXiv.2205.07090,\nhttps://epiforecasts.io/scoringutils/,\nhttps://github.com/epiforecasts/scoringutils", "BugReports": "https://github.com/epiforecasts/scoringutils/issues", "VignetteBuilder": "knitr", "Roxygen": "list(markdown = TRUE)", "Repository": "https://bisaloo.r-universe.dev", "Date/Publication": "2026-05-30 09:40:10 UTC", "RemoteUrl": "https://github.com/epiforecasts/scoringutils", "RemoteRef": "HEAD", "RemoteSha": "0e152773ea4e15fccc4e46abbac233938e740099", "NeedsCompilation": "no", "Packaged": { "Date": "2026-05-30 11:23:49 UTC", "User": "root" }, "Author": "Nikos Bosse [aut, cre] (ORCID: ),\nSam Abbott [aut] (ORCID: ),\nHugo Gruson [aut] (ORCID: ),\nJohannes Bracher [ctb] (ORCID: ),\nToshiaki Asakura [ctb] (ORCID: ),\nJames Mba Azam [ctb] (ORCID: ),\nSebastian Funk [aut],\nMichael Chirico [ctb] (ORCID: )", "Maintainer": "Nikos Bosse ", "MD5sum": "53d217f1cb3a3b72a18a5d8f74b44e9d", "_user": "bisaloo", "_type": "src", "_file": "scoringutils_2.2.0.9000.tar.gz", "_fileid": "6ae8c0df7e373c1b93514d50adae2fe10d096920526c818e5447947b0f93f57a", "_filesize": 6209349, "_sha256": "6ae8c0df7e373c1b93514d50adae2fe10d096920526c818e5447947b0f93f57a", "_created": "2026-05-30T11:23:49.000Z", "_published": "2026-05-30T16:15:40.122Z", "_distro": "noble", "_jobs": [ { "job": 78645385534, "time": 180, "config": "linux-devel-x86_64", "r": "4.7.0", "check": "NOTE", "artifact": "7307840975" }, { "job": 78645385535, "time": 199, "config": "linux-release-x86_64", "r": "4.6.0", "check": "NOTE", "artifact": "7307842520" }, { "job": 78645385541, "time": 173, "config": "macos-oldrel-arm64", "r": "4.5.3", "check": "NOTE", "artifact": "7309543974" }, { "job": 78645385557, "time": 143, "config": "macos-release-arm64", "r": "4.6.0", "check": "NOTE", "artifact": "7309538115" }, { "job": 78645228693, "time": 197, "config": "source", "r": "4.6.0", "check": "OK", "artifact": "7307825148" }, { "job": 78645385530, "time": 119, "config": "wasm-release", "r": "4.6.0", "check": "OK", "artifact": "7307835841" }, { "job": 78645385564, "time": 188, "config": "windows-devel", "r": "4.7.0", "check": "NOTE", "artifact": "7307841735" }, { "job": 78645385571, "time": 180, "config": "windows-oldrel", "r": "4.5.3", "check": "NOTE", "artifact": "7307840982" }, { "job": 78645385570, "time": 166, "config": "windows-release", "r": "4.6.0", "check": "NOTE", "artifact": "7307839820" } ], "_buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830", "_status": "success", "_host": "GitHub-Actions", "_upstream": "https://github.com/epiforecasts/scoringutils", "_commit": { "id": "0e152773ea4e15fccc4e46abbac233938e740099", "author": "Nikos Bosse <37978797+nikosbosse@users.noreply.github.com>", "committer": "GitHub ", "message": "Fix duplicate column names from summarise_scores() with empty metrics (#1179) (#1180)\n\n* fix(summarise_scores): use exact metric-column matching\n\n`summarise_scores()` selected the columns to summarise via\n`colnames(scores) %like% paste(metrics, collapse = \"|\")`. When the\n`metrics` attribute is empty (which happens when every metric passed\nto `score()` warned and returned nothing), the pattern becomes the\nempty string, which `%like%` matches against every column. The `by`\ncolumn was then passed to the summary function, producing a duplicate\n`by` column in the output and the spurious \"argument is not numeric or\nlogical\" warning.\n\nSwitch to exact column-name matching via `intersect()` and error early\nwhen there is nothing to summarise. This also incidentally fixes a\nlatent issue where a metric named e.g. \"wis\" would have matched any\ncolumn whose name contained \"wis\" (such as \"wis_relative_skill\").\n\nCloses #1179.\n\nCo-Authored-By: Claude Opus 4.7 (1M context) \n\n* test(summarise_scores): add end-to-end reprex regression test for #1179\n\nAdds a regression test exercising the exact reprex from #1179: scoring\nexample_quantile with only `interval_coverage_55` warns and produces no\nscore columns, after which `summarise_scores()` must error rather than\nreturn a data.table with a duplicate `by` column.\n\n---------\n\nCo-authored-by: Claude Opus 4.7 (1M context) \nCo-authored-by: seabbs-bot ", "time": 1780134010 }, "_maintainer": { "name": "Nikos Bosse", "email": "nikosbosse@gmail.com", "login": "nikosbosse", "twitter": "@nikosbosse", "description": "Research Engineer", "uuid": 37978797, "orcid": "0000-0002-7750-5280" }, "_registered": true, "_dependencies": [ { "package": "R", "version": ">= 4.1", "role": "Depends" }, { "package": "checkmate", "role": "Imports" }, { "package": "cli", "role": "Imports" }, { "package": "data.table", "version": ">= 1.17.0", "role": "Imports" }, { "package": "ggplot2", "version": ">= 3.4.0", "role": "Imports" }, { "package": "lifecycle", "role": "Imports" }, { "package": "methods", "role": "Imports" }, { "package": "purrr", "role": "Imports" }, { "package": "scoringRules", "version": ">= 1.1.3", "role": "Imports" }, { "package": "stats", "role": "Imports" }, { "package": "ggdist", "role": "Suggests" }, { "package": "kableExtra", "role": "Suggests" }, { "package": "knitr", "role": "Suggests" }, { "package": "Metrics", "role": "Suggests" }, { "package": "rmarkdown", "role": "Suggests" }, { "package": "testthat", "version": ">= 3.1.9", "role": "Suggests" }, { "package": "vdiffr", "role": "Suggests" } ], "_owner": "epiforecasts", "_selfowned": false, "_usedby": 7, "_updates": [ { "week": "2025-25", "n": 2 }, { "week": "2025-26", "n": 6 }, { "week": "2025-27", "n": 3 }, { "week": "2025-29", "n": 3 }, { "week": "2025-34", "n": 2 }, { "week": "2025-35", "n": 2 }, { "week": "2025-37", "n": 5 }, { "week": "2025-39", "n": 1 }, { "week": "2025-44", "n": 1 }, { "week": "2025-48", "n": 3 }, { "week": "2025-51", "n": 2 }, { "week": "2026-05", "n": 4 }, { "week": "2026-07", "n": 5 }, { "week": "2026-08", "n": 2 }, { "week": "2026-09", "n": 2 }, { "week": "2026-10", "n": 3 }, { "week": "2026-12", "n": 4 }, { "week": "2026-14", "n": 17 }, { "week": "2026-17", "n": 5 }, { "week": "2026-20", "n": 3 }, { "week": "2026-22", "n": 5 } ], "_tags": [ { "name": "v2.1.1", "date": "2025-06-25" }, { "name": "v2.1.2", "date": "2025-08-25" }, { "name": "v2.2.0", "date": "2026-04-05" } ], "_topics": [ "forecast-evaluation", "forecasting" ], "_stars": 61, "_contributors": [ { "user": "nikosbosse", "count": 2148, "uuid": 37978797 }, { "user": "seabbs", "count": 295, "uuid": 20317649 }, { "user": "sbfnk", "count": 87, "uuid": 1156307 }, { "user": "jamesmbaazam", "count": 75, "uuid": 43169674 }, { "user": "seabbs-bot", "count": 70, "uuid": 256795758 }, { "user": "bisaloo", "count": 70, "uuid": 10783929 }, { "user": "toshiakiasakura", "count": 14, "uuid": 20488884 }, { "user": "michaelchirico", "count": 9, "uuid": 7606389 }, { "user": "jcken95", "count": 4, "uuid": 57638616 }, { "user": "athowes", "count": 4, "uuid": 29818116 }, { "user": "nickreich", "count": 3, "uuid": 1280767 }, { "user": "damonbayer", "count": 1, "uuid": 8095080 } ], "_userbio": { "uuid": 10783929, "type": "user", "name": "Hugo Gruson", "description": "Evolutionary Biologist turned Research Software Engineer in R." }, "_downloads": { "count": 1131, "source": "https://cranlogs.r-pkg.org/downloads/total/last-month/scoringutils" }, "_devurl": "https://github.com/epiforecasts/scoringutils", "_pkgdown": "https://epiforecasts.io/scoringutils/", "_searchresults": 445, "_rbuild": "4.6.0", "_assets": [ "extra/citation.cff", "extra/citation.html", "extra/citation.json", "extra/citation.txt", "extra/contents.json", "extra/NEWS.html", "extra/NEWS.txt", "extra/readme.html", "extra/readme.md", "extra/scoringutils.html", "manual.pdf" ], "_homeurl": "https://github.com/epiforecasts/scoringutils", "_realowner": "epiforecasts", "_cranurl": true, "_releases": [ { "version": "0.1.0", "date": "2020-06-14" }, { "version": "0.1.4", "date": "2020-11-17" }, { "version": "0.1.7", "date": "2021-07-14" }, { "version": "0.1.7.2", "date": "2021-07-21" }, { "version": "1.0.0", "date": "2022-05-13" }, { "version": "1.0.1", "date": "2022-08-16" }, { "version": "1.1.0", "date": "2023-01-30" }, { "version": "1.2.2", "date": "2023-11-29" }, { "version": "2.0.0", "date": "2024-10-31" }, { "version": "2.1.0", "date": "2025-03-03" }, { "version": "2.1.1", "date": "2025-06-25" }, { "version": "2.1.2", "date": "2025-08-25" }, { "version": "2.2.0", "date": "2026-04-05" } ], "_exports": [ "add_relative_skill", "ae_median_quantile", "ae_median_sample", "as_forecast_binary", "as_forecast_multivariate_point", "as_forecast_multivariate_sample", "as_forecast_nominal", "as_forecast_ordinal", "as_forecast_point", "as_forecast_quantile", "as_forecast_sample", "assert_forecast", "bias_quantile", "bias_sample", "brier_score", "crps_sample", "dispersion_quantile", "dispersion_sample", "dss_sample", "energy_score_multivariate", "get_correlations", "get_coverage", "get_duplicate_forecasts", "get_forecast_counts", "get_forecast_type_ids", "get_forecast_unit", "get_grouping", "get_metrics", "get_pairwise_comparisons", "get_pit_histogram", "interval_coverage", "is_forecast", "is_forecast_binary", "is_forecast_multivariate_point", "is_forecast_multivariate_sample", "is_forecast_nominal", "is_forecast_ordinal", "is_forecast_point", "is_forecast_quantile", "is_forecast_sample", "log_shift", "logs_binary", "logs_categorical", "logs_sample", "mad_sample", "new_forecast", "overprediction_quantile", "overprediction_sample", "pit_histogram_sample", "plot_correlations", "plot_discrimination", "plot_forecast_counts", "plot_heatmap", "plot_interval_coverage", "plot_pairwise_comparisons", "plot_quantile_coverage", "plot_wis", "quantile_score", "rps_ordinal", "score", "se_mean_sample", "select_metrics", "summarise_scores", "summarize_scores", "theme_scoringutils", "transform_forecasts", "underprediction_quantile", "underprediction_sample", "variogram_score_multivariate", "variogram_score_multivariate_point", "wis" ], "_datasets": [ { "name": "example_binary", "title": "Binary forecast example data", "object": "example_binary", "class": [ "forecast_binary", "forecast", "data.table", "data.frame" ], "fields": [ "location", "location_name", "target_end_date", "target_type", "forecast_date", "model", "horizon", "predicted", "observed" ], "rows": 1031, "table": true, "tojson": true }, { "name": "example_multivariate_sample", "title": "Multivariate forecast example data", "object": "example_multivariate_sample", "class": [ "forecast_multivariate_sample", "forecast", "data.table", "data.frame" ], "fields": [ "location", "location_name", "target_end_date", "target_type", "forecast_date", "model", "horizon", "predicted", "sample_id", "observed", ".mv_group_id" ], "rows": 35624, "table": true, "tojson": true }, { "name": "example_nominal", "title": "Nominal example data", "object": "example_nominal", "class": [ "forecast_nominal", "forecast", "data.table", "data.frame" ], "fields": [ "location", "location_name", "target_end_date", "target_type", "forecast_date", "horizon", "model", "observed", "predicted_label", "predicted" ], "rows": 3093, "table": true, "tojson": true }, { "name": "example_ordinal", "title": "Ordinal example data", "object": "example_ordinal", "class": [ "forecast_ordinal", "forecast", "data.table", "data.frame" ], "fields": [ "location", "location_name", "target_end_date", "target_type", "forecast_date", "horizon", "model", "observed", "predicted_label", "predicted" ], "rows": 3093, "table": true, "tojson": true }, { "name": "example_point", "title": "Point forecast example data", "object": "example_point", "class": [ "forecast_point", "forecast", "data.table", "data.frame" ], "fields": [ "location", "target_end_date", "target_type", "observed", "location_name", "forecast_date", "predicted", "model", "horizon" ], "rows": 1031, "table": true, "tojson": true }, { "name": "example_quantile", "title": "Quantile example data", "object": "example_quantile", "class": [ "forecast_quantile", "forecast", "data.table", "data.frame" ], "fields": [ "location", "target_end_date", "target_type", "observed", "location_name", "forecast_date", "quantile_level", "predicted", "model", "horizon" ], "rows": 20545, "table": true, "tojson": true }, { "name": "example_sample_continuous", "title": "Continuous forecast example data", "object": "example_sample_continuous", "class": [ "forecast_sample", "forecast", "data.table", "data.frame" ], "fields": [ "location", "location_name", "target_end_date", "target_type", "forecast_date", "model", "horizon", "predicted", "sample_id", "observed" ], "rows": 35624, "table": true, "tojson": true }, { "name": "example_sample_discrete", "title": "Discrete forecast example data", "object": "example_sample_discrete", "class": [ "forecast_sample", "forecast", "data.table", "data.frame" ], "fields": [ "location", "location_name", "target_end_date", "target_type", "forecast_date", "model", "horizon", "predicted", "sample_id", "observed" ], "rows": 35624, "table": true, "tojson": true } ], "_help": [ { "page": "add_relative_skill", "title": "Add relative skill scores based on pairwise comparisons", "topics": [ "add_relative_skill" ] }, { "page": "ae_median_quantile", "title": "Absolute error of the median (quantile-based version)", "topics": [ "ae_median_quantile" ] }, { "page": "ae_median_sample", "title": "Absolute error of the median (sample-based version)", "topics": [ "ae_median_sample" ] }, { "page": "as_forecast_binary", "title": "Create a 'forecast' object for binary forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_binary", "as_forecast_binary.default" ] }, { "page": "as_forecast_doc_template", "title": "General information on creating a 'forecast' object", "topics": [ "as_forecast_doc_template" ] }, { "page": "as_forecast_generic", "title": "Common functionality for as_forecast_ functions", "topics": [ "as_forecast_generic" ] }, { "page": "as_forecast_multivariate_point", "title": "Create a 'forecast' object for multivariate point forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_multivariate_point", "as_forecast_multivariate_point.default" ] }, { "page": "as_forecast_multivariate_sample", "title": "Create a 'forecast' object for sample-based multivariate forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_multivariate_sample", "as_forecast_multivariate_sample.default" ] }, { "page": "as_forecast_nominal", "title": "Create a 'forecast' object for nominal forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_nominal", "as_forecast_nominal.default" ] }, { "page": "as_forecast_ordinal", "title": "Create a 'forecast' object for ordinal forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_ordinal", "as_forecast_ordinal.default" ] }, { "page": "as_forecast_point", "title": "Create a 'forecast' object for point forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_point", "as_forecast_point.default", "as_forecast_point.forecast_quantile" ] }, { "page": "as_forecast_quantile", "title": "Create a 'forecast' object for quantile-based forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_quantile", "as_forecast_quantile.default", "as_forecast_quantile.forecast_sample" ] }, { "page": "as_forecast_sample", "title": "Create a 'forecast' object for sample-based forecasts", "concept": [ "functions to create forecast objects" ], "topics": [ "as_forecast_sample", "as_forecast_sample.default" ] }, { "page": "assert_dims_ok_scalar", "title": "Assert Inputs Have Matching Dimensions", "topics": [ "assert_dims_ok_scalar" ] }, { "page": "assert_forecast_generic", "title": "Validation common to all forecast types", "topics": [ "assert_forecast_generic" ] }, { "page": "assert_forecast_type", "title": "Assert that forecast type is as expected", "topics": [ "assert_forecast_type" ] }, { "page": "assert_forecast", "title": "Assert that input is a forecast object and passes validations", "topics": [ "assert_forecast", "assert_forecast.default", "assert_forecast.forecast_binary", "assert_forecast.forecast_multivariate_point", "assert_forecast.forecast_multivariate_sample", "assert_forecast.forecast_point", "assert_forecast.forecast_quantile", "assert_forecast.forecast_sample" ] }, { "page": "assert_input_binary", "title": "Assert that inputs are correct for binary forecast", "topics": [ "assert_input_binary" ] }, { "page": "assert_input_categorical", "title": "Assert that inputs are correct for categorical forecasts", "topics": [ "assert_input_categorical" ] }, { "page": "assert_input_interval", "title": "Assert that inputs are correct for interval-based forecast", "topics": [ "assert_input_interval" ] }, { "page": "assert_input_multivariate_sample", "title": "Assert that inputs are correct for sample-based forecast", "topics": [ "assert_input_multivariate_sample" ] }, { "page": "assert_input_nominal", "title": "Assert that inputs are correct for nominal forecasts", "topics": [ "assert_input_nominal" ] }, { "page": "assert_input_ordinal", "title": "Assert that inputs are correct for ordinal forecasts", "topics": [ "assert_input_ordinal" ] }, { "page": "assert_input_point", "title": "Assert that inputs are correct for point forecast", "topics": [ "assert_input_point" ] }, { "page": "assert_input_quantile", "title": "Assert that inputs are correct for quantile-based forecast", "topics": [ "assert_input_quantile" ] }, { "page": "assert_input_sample", "title": "Assert that inputs are correct for sample-based forecast", "topics": [ "assert_input_sample" ] }, { "page": "bias_quantile", "title": "Determines bias of quantile forecasts", "topics": [ "bias_quantile" ] }, { "page": "bias_sample", "title": "Determine bias of forecasts", "topics": [ "bias_sample" ] }, { "page": "check_dims_ok_scalar", "title": "Check Inputs Have Matching Dimensions", "topics": [ "check_dims_ok_scalar" ] }, { "page": "check_duplicates", "title": "Check that there are no duplicate forecasts", "topics": [ "check_duplicates" ] }, { "page": "check_input_binary", "title": "Check that inputs are correct for binary forecast", "topics": [ "check_input_binary" ] }, { "page": "check_input_point", "title": "Check that inputs are correct for point forecast", "topics": [ "check_input_point" ] }, { "page": "check_number_per_forecast", "title": "Check that all forecasts have the same number of rows", "topics": [ "check_number_per_forecast" ] }, { "page": "check_numeric_vector", "title": "Check whether an input is an atomic vector of mode 'numeric'", "topics": [ "check_numeric_vector" ] }, { "page": "check_try", "title": "Helper function to convert assert statements into checks", "topics": [ "check_try" ] }, { "page": "crps_sample", "title": "(Continuous) ranked probability score", "topics": [ "crps_sample", "dispersion_sample", "overprediction_sample", "underprediction_sample" ] }, { "page": "dss_sample", "title": "Dawid-Sebastiani score", "topics": [ "dss_sample" ] }, { "page": "energy_score_multivariate", "title": "Energy score for multivariate forecasts", "topics": [ "energy_score_multivariate" ] }, { "page": "example_binary", "title": "Binary forecast example data", "topics": [ "example_binary" ] }, { "page": "example_multivariate_sample", "title": "Multivariate forecast example data", "topics": [ "example_multivariate_sample" ] }, { "page": "example_nominal", "title": "Nominal example data", "topics": [ "example_nominal" ] }, { "page": "example_ordinal", "title": "Ordinal example data", "topics": [ "example_ordinal" ] }, { "page": "example_point", "title": "Point forecast example data", "topics": [ "example_point" ] }, { "page": "example_quantile", "title": "Quantile example data", "topics": [ "example_quantile" ] }, { "page": "example_sample_continuous", "title": "Continuous forecast example data", "topics": [ "example_sample_continuous" ] }, { "page": "example_sample_discrete", "title": "Discrete forecast example data", "topics": [ "example_sample_discrete" ] }, { "page": "get_correlations", "title": "Calculate correlation between metrics", "topics": [ "get_correlations" ] }, { "page": "get_coverage", "title": "Get quantile and interval coverage values for quantile-based forecasts", "topics": [ "get_coverage" ] }, { "page": "get_duplicate_forecasts", "title": "Find duplicate forecasts", "topics": [ "get_duplicate_forecasts" ] }, { "page": "get_forecast_counts", "title": "Count number of available forecasts", "topics": [ "get_forecast_counts" ] }, { "page": "get_forecast_type", "title": "Get forecast type from forecast object", "topics": [ "get_forecast_type" ] }, { "page": "get_forecast_type_ids", "title": "Get type-specific ID columns for a forecast", "topics": [ "get_forecast_type_ids", "get_forecast_type_ids.forecast_multivariate_sample", "get_forecast_type_ids.forecast_nominal", "get_forecast_type_ids.forecast_ordinal", "get_forecast_type_ids.forecast_quantile", "get_forecast_type_ids.forecast_sample" ] }, { "page": "get_forecast_unit", "title": "Get unit of a single forecast", "topics": [ "get_forecast_unit" ] }, { "page": "get_grouping", "title": "Get grouping for a multivariate forecast", "topics": [ "get_grouping" ] }, { "page": "get_metrics", "title": "Get metrics", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics" ] }, { "page": "get_metrics.forecast_binary", "title": "Get default metrics for binary forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_binary" ] }, { "page": "get_metrics.forecast_multivariate_point", "title": "Get default metrics for multivariate point forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_multivariate_point" ] }, { "page": "get_metrics.forecast_multivariate_sample", "title": "Get default metrics for sample-based forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_multivariate_sample" ] }, { "page": "get_metrics.forecast_nominal", "title": "Get default metrics for nominal forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_nominal" ] }, { "page": "get_metrics.forecast_ordinal", "title": "Get default metrics for nominal forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_ordinal" ] }, { "page": "get_metrics.forecast_point", "title": "Get default metrics for point forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_point" ] }, { "page": "get_metrics.forecast_quantile", "title": "Get default metrics for quantile-based forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_quantile" ] }, { "page": "get_metrics.forecast_sample", "title": "Get default metrics for sample-based forecasts", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.forecast_sample" ] }, { "page": "get_metrics.scores", "title": "Get names of the metrics that were used for scoring", "concept": [ "get_metrics functions" ], "topics": [ "get_metrics.scores" ] }, { "page": "get_pairwise_comparisons", "title": "Obtain pairwise comparisons between models", "topics": [ "get_pairwise_comparisons" ] }, { "page": "get_pit_histogram", "title": "Probability integral transformation histogram", "topics": [ "get_pit_histogram", "get_pit_histogram.default", "get_pit_histogram.forecast_quantile", "get_pit_histogram.forecast_sample" ] }, { "page": "get_type", "title": "Get type of a vector or matrix of observed values or predictions", "topics": [ "get_type" ] }, { "page": "interval_coverage", "title": "Interval coverage (for quantile-based forecasts)", "topics": [ "interval_coverage" ] }, { "page": "interval_score", "title": "Interval score", "topics": [ "interval_score" ] }, { "page": "is_forecast", "title": "Test whether an object is a forecast object", "topics": [ "is_forecast", "is_forecast_binary", "is_forecast_multivariate_point", "is_forecast_multivariate_sample", "is_forecast_nominal", "is_forecast_ordinal", "is_forecast_point", "is_forecast_quantile", "is_forecast_sample" ] }, { "page": "log_shift", "title": "Log transformation with an additive shift", "topics": [ "log_shift" ] }, { "page": "scoring-functions-nominal", "title": "Log score for categorical outcomes", "concept": [ "log score functions" ], "topics": [ "logs_categorical" ] }, { "page": "logs_sample", "title": "Logarithmic score (sample-based version)", "concept": [ "log score functions" ], "topics": [ "logs_sample" ] }, { "page": "mad_sample", "title": "Determine dispersion of a probabilistic forecast", "topics": [ "mad_sample" ] }, { "page": "pit_histogram_sample", "title": "Probability integral transformation for counts", "topics": [ "pit_histogram_sample" ] }, { "page": "plot_correlations", "title": "Plot correlation between metrics", "topics": [ "plot_correlations" ] }, { "page": "plot_discrimination", "title": "Plot discrimination for binary forecasts", "topics": [ "plot_discrimination" ] }, { "page": "plot_forecast_counts", "title": "Visualise the number of available forecasts", "topics": [ "plot_forecast_counts" ] }, { "page": "plot_heatmap", "title": "Create a heatmap of a scoring metric", "topics": [ "plot_heatmap" ] }, { "page": "plot_interval_coverage", "title": "Plot interval coverage", "topics": [ "plot_interval_coverage" ] }, { "page": "plot_pairwise_comparisons", "title": "Plot heatmap of pairwise comparisons", "topics": [ "plot_pairwise_comparisons" ] }, { "page": "plot_quantile_coverage", "title": "Plot quantile coverage", "topics": [ "plot_quantile_coverage" ] }, { "page": "plot_wis", "title": "Plot contributions to the weighted interval score", "topics": [ "plot_wis" ] }, { "page": "print.forecast", "title": "Print information about a forecast object", "topics": [ "print.forecast" ] }, { "page": "print.scores", "title": "Print a scores object", "topics": [ "print.scores" ] }, { "page": "quantile_score", "title": "Quantile score", "topics": [ "quantile_score" ] }, { "page": "rps_ordinal", "title": "Ranked Probability Score for ordinal outcomes", "concept": [ "scoring functions" ], "topics": [ "rps_ordinal" ] }, { "page": "score", "title": "Evaluate forecasts", "topics": [ "score", "score.forecast_binary", "score.forecast_multivariate_point", "score.forecast_multivariate_sample", "score.forecast_nominal", "score.forecast_ordinal", "score.forecast_point", "score.forecast_quantile", "score.forecast_sample" ] }, { "page": "scoring-functions-binary", "title": "Metrics for binary outcomes", "concept": [ "log score functions" ], "topics": [ "brier_score", "logs_binary", "scoring-functions-binary" ] }, { "page": "se_mean_sample", "title": "Squared error of the mean (sample-based version)", "topics": [ "se_mean_sample" ] }, { "page": "select_metrics", "title": "Select metrics from a list of functions", "topics": [ "select_metrics" ] }, { "page": "set_forecast_unit", "title": "Set unit of a single forecast manually", "topics": [ "set_forecast_unit" ] }, { "page": "summarise_scores", "title": "Summarise scores as produced by 'score()'", "topics": [ "summarise_scores", "summarize_scores" ] }, { "page": "theme_scoringutils", "title": "Scoringutils ggplot2 theme", "topics": [ "theme_scoringutils" ] }, { "page": "transform_forecasts", "title": "Transform forecasts and observed values", "topics": [ "transform_forecasts", "transform_forecasts.default", "transform_forecasts.forecast", "transform_forecasts.forecast_multivariate_point", "transform_forecasts.forecast_multivariate_sample" ] }, { "page": "validate_metrics", "title": "Validate metrics", "topics": [ "validate_metrics" ] }, { "page": "variogram_score_multivariate", "title": "Variogram score for multivariate forecasts", "topics": [ "variogram_score_multivariate" ] }, { "page": "variogram_score_multivariate_point", "title": "Variogram score for multivariate point forecasts", "topics": [ "variogram_score_multivariate_point" ] }, { "page": "wis", "title": "Weighted interval score (WIS)", "topics": [ "dispersion_quantile", "overprediction_quantile", "underprediction_quantile", "wis" ] } ], "_pkglogo": "https://github.com/epiforecasts/scoringutils/raw/HEAD/man/figures/logo.png", "_readme": "https://github.com/epiforecasts/scoringutils/raw/HEAD/README.md", "_rundeps": [ "backports", "checkmate", "cli", "cpp11", "data.table", "evaluate", "farver", "ggplot2", "glue", "gtable", "highr", "isoband", "knitr", "labeling", "lifecycle", "magrittr", "MASS", "purrr", "R6", "RColorBrewer", "Rcpp", "RcppArmadillo", "rlang", "S7", "scales", "scoringRules", "vctrs", "viridisLite", "withr", "xfun", "yaml" ], "_vignettes": [ { "source": "scoring-multivariate-forecasts.Rmd", "filename": "scoring-multivariate-forecasts.html", "title": "Scoring multivariate forecasts", "engine": "knitr::rmarkdown", "headings": [ "Univariate forecasts", "Multivariate forecasts", "Multivariate point forecasts" ], "created": "2025-09-23 15:12:03", "modified": "2026-03-08 19:16:25", "commits": 4 }, { "source": "scoring-rules.Rmd", "filename": "scoring-rules.html", "title": "Scoring rules in scoringutils", "author": "Nikos Bosse", "engine": "knitr::rmarkdown", "headings": [ "Introduction", "Metrics for point forecasts", "A note of caution", "Absolute error", "Squared error", "Absolute percentage error", "Binary forecasts", "Brier score", "Logarithmic score", "Sample-based forecasts", "CRPS", "Overprediction, underprediction and dispersion", "Log score", "Dawid-Sebastiani score", "Dispersion - Median Absolute Deviation (MAD)", "Bias", "Absolute error of the median", "Squared error of the mean", "Quantile-based forecasts", "Weighted interval score (WIS)", "Interval coverage", "Interval coverage deviation", "Quantile score", "Additional metrics", "Quantile coverage" ], "created": "2024-04-08 09:36:27", "modified": "2026-03-30 16:16:20", "commits": 8 } ], "_score": 11.172983725543219, "_indexed": false, "_nocasepkg": "scoringutils", "_universes": [ "bisaloo" ], "_indexurl": "https://epiforecasts.r-universe.dev/scoringutils", "_binaries": [ { "r": "4.7.0", "os": "linux", "version": "2.2.0.9000", "date": "2026-05-30T11:26:09.000Z", "distro": "noble", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "39a436098bbdee000d2452a579e525ff4e4e669a49dc1181b4742c50eda2cc69", "status": "success", "check": "NOTE", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" }, { "r": "4.6.0", "os": "linux", "version": "2.2.0.9000", "date": "2026-05-30T11:26:15.000Z", "distro": "noble", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "00ed82a5d141bbe1392d55a5b20484e74d49de18a5297379f43a93cf429be18d", "status": "success", "check": "NOTE", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" }, { "r": "4.5.3", "os": "mac", "version": "2.2.0.9000", "date": "2026-05-30T16:14:08.000Z", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "49eb6ec6c767fdb4815c9aa74665c3fcf21a01bd25aefa3111cab1bd96957de6", "status": "success", "check": "NOTE", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" }, { "r": "4.6.0", "os": "mac", "version": "2.2.0.9000", "date": "2026-05-30T16:13:12.000Z", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "53234cf6568aa3598563ac7a84aca20b849828c6dcea42723b2c4f9eddaea43f", "status": "success", "check": "NOTE", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" }, { "r": "4.6.0", "os": "wasm", "version": "2.2.0.9000", "date": "2026-05-30T11:26:19.000Z", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "e7fee7e91f1bae04b170343f2b6d27665ba61264d2eddc2d3ee3c841186d433e", "status": "success", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" }, { "r": "4.7.0", "os": "win", "version": "2.2.0.9000", "date": "2026-05-30T11:25:31.000Z", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "aa4e697f12fa2ac060b17c9222cb801c0560156123d105241ffd9067d49e15b3", "status": "success", "check": "NOTE", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" }, { "r": "4.5.3", "os": "win", "version": "2.2.0.9000", "date": "2026-05-30T11:25:29.000Z", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "a087d7218926e700566a35ecc1c2e15a7b90c2e327665617bddfb4a91138fe07", "status": "success", "check": "NOTE", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" }, { "r": "4.6.0", "os": "win", "version": "2.2.0.9000", "date": "2026-05-30T11:25:17.000Z", "commit": "0e152773ea4e15fccc4e46abbac233938e740099", "fileid": "7ce6452a55bb9e6b84d5e077e96622dec57a97215bdf5c6ada7308ec7bb89ce2", "status": "success", "check": "NOTE", "buildurl": "https://github.com/r-universe/bisaloo/actions/runs/26682502830" } ] }