diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/model-assessment.md | 22 |
1 files changed, 22 insertions, 0 deletions
diff --git a/doc/model-assessment.md b/doc/model-assessment.md new file mode 100644 index 0000000..f468f42 --- /dev/null +++ b/doc/model-assessment.md @@ -0,0 +1,22 @@ +# Assessing Model Quality + +## Tabular Comparison + +`--show-quality=table` outputs a table that shows LUT error, model error, and +static error. In general, the model is suitable if its prediction error is +close to LUT error and far away from static error. + +The error metric can be selected via `--error-metric`. + +It is generally a good idea to combine `--show-quality=table` with +`--cross-validate=kfold:10` and, depending on application, +`--parameter-aware-cross-validation`. Note that LUT error serves as baseline +("lowest achievable prediction error" / underlying measurement uncertainty) +and is always reported without cross-validation. + +## Dataref export + +`--export-dref=filename.tex` exports model statistics and all available error +metrics to a [dataref](https://ctan.org/pkg/dataref) file. Again, it may be a +good idea to also specify `--cross-validate=kfold:10` and possibly +`--parameter-aware-cross-validation`. |