-
Notifications
You must be signed in to change notification settings - Fork 2
/
README.Rmd
175 lines (133 loc) · 6.25 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
library(ggplot2)
theme_set(theme_minimal())
```
# waywiser <a href="https://docs.ropensci.org/waywiser/"><img src="man/figures/logo.png" align="right" height="138" /></a>
<!-- badges: start -->
[](https://arxiv.org/abs/2303.11312)
[](https://github.com/ropensci/waywiser/actions/workflows/R-CMD-check.yaml)
[](https://choosealicense.com/licenses/mit/)
[](https://lifecycle.r-lib.org/articles/stages.html#maturing)
[](https://www.repostatus.org/#active)
[](https://app.codecov.io/gh/ropensci/waywiser?branch=main)
[](https://CRAN.R-project.org/package=waywiser)
[](https://github.com/ropensci/software-review/issues/571)
<!-- badges: end -->
"Waywiser" is an old-timey name for a
[surveyor's wheel](https://en.wikipedia.org/wiki/Surveyor%27s_wheel), a device
that makes measuring long distances easier than with measurement tools like a
ruler or yardstick. The waywiser R package makes it easier to measure the
performance of models fit to 2D spatial data by implementing a number of
well-established assessment methods in a consistent, ergonomic toolbox; features
include new [yardstick](https://yardstick.tidymodels.org/) metrics
for measuring agreement and spatial autocorrelation, functions to assess model
predictions across multiple scales, and methods to calculate the area of
applicability of a model.
## Installation
You can install waywiser from CRAN via:
```r
install.packages("waywiser")
```
You can install the development version of waywiser from [GitHub](https://github.com/) with:
``` r
# install.packages("remotes")
remotes::install_github("ropensci/waywiser")
# or, equivalently:
install.packages("waywiser", repos = "https://ropensci.r-universe.dev")
```
## Example
Let's say that we fit a linear model predicting crimes against people as a
function of literacy, using the `guerry` data included in waywiser:
```{r}
library(waywiser)
set.seed(123)
split_idx <- sample(seq_len(nrow(guerry)), nrow(guerry) * 0.8)
guerry_train <- guerry[split_idx, ]
guerry_test <- guerry[-split_idx, ]
crime_model <- lm(Crm_prs ~ Litercy, guerry_train)
```
We want to assess this model, to better understand how well it predicts crime
rates across 1830s France. One method to do so is to evaluate our predictions at
multiple levels of aggregation, as suggested by Riemann et al. (2010)
(<doi: 10.1016/j.rse.2010.05.010>). This approach is focused on
aggregating point predictions, so we'll convert our data to points and then see
how well our predictions perform when aggregated to two different scales:
```{r}
guerry_points <- data.frame(
truth = guerry$Crm_prs,
estimate = predict(crime_model, guerry),
geometry = sf::st_centroid(sf::st_geometry(guerry))
)
guerry_points <- sf::st_as_sf(guerry_points)
guerry_multi_scale <- ww_multi_scale(
guerry_points,
truth,
estimate,
n = list(c(5, 5), c(2, 2))
)
guerry_multi_scale
```
More information about multi-scale assessment is included in `vignette("multi-scale-assessment", package = "waywiser")`.
We could also assess the spatial dependence of our model residuals, to identify
any potential "hot spots" where our model is consistently less accurate than
we'd expect by chance:
```{r}
guerry_predicted <- guerry
guerry_predicted$predictions <- predict(crime_model, guerry)
ww_local_moran_i(guerry_predicted, Crm_prs, predictions)
```
More information about multi-scale assessment is included in `vignette("residual-autocorrelation", package = "waywiser")`.
Lastly, we can also see if there's any areas in our data that are too different
from our training data for us to safely predict on, which fall outside the
"area of applicability" defined by Meyer and Pebesma (2021)
(<doi: 10.1111/2041-210X.13650>):
```{r}
crime_model_aoa <- ww_area_of_applicability(
Crm_prs ~ Litercy,
guerry_train,
guerry_test,
importance = vip::vi_model(crime_model)
)
guerry_aoa <- cbind(
guerry,
predict(crime_model_aoa, guerry)
)
plot(guerry_aoa["aoa"])
```
We can see that two areas are outside our model's area of applicability, meaning
that we probably can't trust our model when extrapolating into those regions!
For more information, check out [the documentation website!](https://docs.ropensci.org/waywiser/)
## Citing waywiser
To cite waywiser in publications please use:
Mahoney M. J. (2023). waywiser: Ergonomic Methods for Assessing Spatial Models. arXiv:2303.11312 [cs.MS]. https://doi.org/10.48550/arXiv.2303.11312
A BibTeX entry for LaTeX users is
```bibtex
@Misc{,
title = {waywiser: Ergonomic Methods for Assessing Spatial Models},
author = {Michael J Mahoney},
year = {2023},
eprint = {2303.11312},
archiveprefix = {arXiv},
primaryclass = {cs.MS},
doi = {10.48550/arXiv.2303.11312},
url = {https://arxiv.org/abs/2303.11312},
}
```
See `citation("waywiser")` for the most up-to-date citation information.
## Contributing
Please note that this package is released with a
[Contributor Code of Conduct](https://ropensci.org/code-of-conduct/).
By contributing to this project, you agree to abide by its terms.
- If you think you have encountered a bug, please [submit an issue](https://github.com/ropensci/waywiser).
- Please include a [reprex](https://reprex.tidyverse.org/articles/articles/learn-reprex.html) (a minimal, reproducible example) to clearly communicate about your code.
[](https://ropensci.org)