summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorBirte Kristina Friesel <birte.friesel@uos.de>2024-01-11 14:42:13 +0100
committerBirte Kristina Friesel <birte.friesel@uos.de>2024-01-11 14:42:13 +0100
commit9705d6687def65a82193e60aa69137ab5bd1d047 (patch)
tree0db05398ee800131a55001050fae1ba95e83ddd9 /doc
parent59e35e5112db018a3ba150c4a661292a154398e3 (diff)
add model assessment documentation
Diffstat (limited to 'doc')
-rw-r--r--doc/model-assessment.md22
1 files changed, 22 insertions, 0 deletions
diff --git a/doc/model-assessment.md b/doc/model-assessment.md
new file mode 100644
index 0000000..f468f42
--- /dev/null
+++ b/doc/model-assessment.md
@@ -0,0 +1,22 @@
+# Assessing Model Quality
+
+## Tabular Comparison
+
+`--show-quality=table` outputs a table that shows LUT error, model error, and
+static error. In general, the model is suitable if its prediction error is
+close to LUT error and far away from static error.
+
+The error metric can be selected via `--error-metric`.
+
+It is generally a good idea to combine `--show-quality=table` with
+`--cross-validate=kfold:10` and, depending on application,
+`--parameter-aware-cross-validation`. Note that LUT error serves as baseline
+("lowest achievable prediction error" / underlying measurement uncertainty)
+and is always reported without cross-validation.
+
+## Dataref export
+
+`--export-dref=filename.tex` exports model statistics and all available error
+metrics to a [dataref](https://ctan.org/pkg/dataref) file. Again, it may be a
+good idea to also specify `--cross-validate=kfold:10` and possibly
+`--parameter-aware-cross-validation`.