diff options
author | Birte Kristina Friesel <birte.friesel@uos.de> | 2024-01-11 14:42:13 +0100 |
---|---|---|
committer | Birte Kristina Friesel <birte.friesel@uos.de> | 2024-01-11 14:42:13 +0100 |
commit | 9705d6687def65a82193e60aa69137ab5bd1d047 (patch) | |
tree | 0db05398ee800131a55001050fae1ba95e83ddd9 /doc | |
parent | 59e35e5112db018a3ba150c4a661292a154398e3 (diff) |
add model assessment documentation
Diffstat (limited to 'doc')
-rw-r--r-- | doc/model-assessment.md | 22 |
1 files changed, 22 insertions, 0 deletions
diff --git a/doc/model-assessment.md b/doc/model-assessment.md new file mode 100644 index 0000000..f468f42 --- /dev/null +++ b/doc/model-assessment.md @@ -0,0 +1,22 @@ +# Assessing Model Quality + +## Tabular Comparison + +`--show-quality=table` outputs a table that shows LUT error, model error, and +static error. In general, the model is suitable if its prediction error is +close to LUT error and far away from static error. + +The error metric can be selected via `--error-metric`. + +It is generally a good idea to combine `--show-quality=table` with +`--cross-validate=kfold:10` and, depending on application, +`--parameter-aware-cross-validation`. Note that LUT error serves as baseline +("lowest achievable prediction error" / underlying measurement uncertainty) +and is always reported without cross-validation. + +## Dataref export + +`--export-dref=filename.tex` exports model statistics and all available error +metrics to a [dataref](https://ctan.org/pkg/dataref) file. Again, it may be a +good idea to also specify `--cross-validate=kfold:10` and possibly +`--parameter-aware-cross-validation`. |