The package supports a variety of metrics that help to evaluate and compare estimated models.
Internal metrics: metrics that assess the adequacy of the model with respect to the data.
External metrics: metrics that compare two models.
Users can implement new metrics through defineInternalMetric() and defineExternalMetric().
Custom-defined metrics are accessible using the same by-name mechanism as the other metrics.
| Metric name | Description | Function / Reference |
AIC | Akaike information criterion. A goodness-of-fit estimator that adjusts for model complexity (i.e., the number of parameters). Only available for models that support the computation of the model log-likelihood through logLik. | stats::AIC(), (Akaike 1974) |
APPA.mean | Mean of the average posterior probability of assignment (APPA) across clusters. A measure of the precision of the trajectory classifications. A score of 1 indicates perfect classification. | APPA(), (Nagin 2005) |
APPA.min | Lowest APPA among the clusters | APPA(), (Nagin 2005) |
ASW | Average silhouette width based on the Euclidean distance | (Rousseeuw 1987) |
BIC | Bayesian information criterion. A goodness-of-fit estimator that corrects for the degrees of freedom (i.e., the number of parameters) and sample size. Only available for models that support the computation of the model log-likelihood through logLik. | stats::BIC(), (Schwarz 1978) |
CAIC | Consistent Akaike information criterion | (Bozdogan 1987) |
CLC | Classification likelihood criterion | (McLachlan and Peel 2000) |
converged | Whether the model converged during estimation | converged() |
deviance | The model deviance | stats::deviance() |
Dunn | The Dunn index | (Dunn 1974) |
entropy | Entropy of the posterior probabilities | |
estimationTime | The time needed for fitting the model | estimationTime() |
ED | Euclidean distance between the cluster trajectories and the assigned observed trajectories | |
ED.fit | Euclidean distance between the cluster trajectories and the assigned fitted trajectories | |
ICL.BIC | Integrated classification likelihood (ICL) approximated using the BIC | (Biernacki et al. 2000) |
logLik | Model log-likelihood | stats::logLik() |
MAE | Mean absolute error of the fitted trajectories (assigned to the most likely respective cluster) to the observed trajectories | |
Mahalanobis | Mahalanobis distance between the cluster trajectories and the assigned observed trajectories | (Mahalanobis 1936) |
MSE | Mean squared error of the fitted trajectories (assigned to the most likely respective cluster) to the observed trajectories | |
relativeEntropy, RE | A measure of the precision of the trajectory classification. A value of 1 indicates perfect classification, whereas a value of 0 indicates a non-informative uniform classification. It is the normalized version of entropy, scaled between [0, 1]. | (Ramaswamy et al. 1993) , (Muthén 2004) |
RMSE | Root mean squared error of the fitted trajectories (assigned to the most likely respective cluster) to the observed trajectories | |
RSS | Residual sum of squares under most likely cluster allocation | |
scaledEntropy | See relativeEntropy | |
sigma | The residual standard deviation | stats::sigma() |
ssBIC | Sample-size adjusted BIC | (Sclove 1987) |
SED | Standardized Euclidean distance between the cluster trajectories and the assigned observed trajectories | |
SED.fit | The cluster-weighted standardized Euclidean distance between the cluster trajectories and the assigned fitted trajectories | |
WMAE | MAE weighted by cluster-assignment probability | |
WMSE | MSE weighted by cluster-assignment probability | |
WRMSE | RMSE weighted by cluster-assignment probability | |
WRSS | RSS weighted by cluster-assignment probability |
| Metric name | Description | Function / Reference |
adjustedRand | Adjusted Rand index. Based on the Rand index, but adjusted for agreements occurring by chance. A score of 1 indicates a perfect agreement, whereas a score of 0 indicates an agreement no better than chance. | mclustcomp::mclustcomp(), (Hubert and Arabie 1985) |
CohensKappa | Cohen's kappa. A partitioning agreement metric correcting for random chance. A score of 1 indicates a perfect agreement, whereas a score of 0 indicates an agreement no better than chance. | psych::cohen.kappa(), (Cohen 1960) |
F | F-score | mclustcomp::mclustcomp() |
F1 | F1-score, also referred to as the Sørensen–Dice Coefficient, or Dice similarity coefficient | mclustcomp::mclustcomp() |
FolkesMallows | Fowlkes-Mallows index | mclustcomp::mclustcomp() |
Hubert | Hubert index | clusterCrit::extCriteria() |
Jaccard | Jaccard index | mclustcomp::mclustcomp() |
jointEntropy | Joint entropy between model assignments | mclustcomp::mclustcomp() |
Kulczynski | Kulczynski index | clusterCrit::extCriteria() |
MaximumMatch | Maximum match measure | mclustcomp::mclustcomp() |
McNemar | McNemar statistic | clusterCrit::extCriteria() |
MeilaHeckerman | Meila-Heckerman measure | mclustcomp::mclustcomp() |
Mirkin | Mirkin metric | mclustcomp::mclustcomp() |
MI | Mutual information | mclustcomp::mclustcomp() |
NMI | Normalized mutual information | igraph::compare() |
NSJ | Normalized version of splitJoin. The proportion of edits relative to the maximum changes (twice the number of ids) | |
NVI | Normalized variation of information | mclustcomp::mclustcomp() |
Overlap | Overlap coefficient, also referred to as the Szymkiewicz–Simpson coefficient | mclustcomp::mclustcomp() (M K and K 2016) |
PD | Partition difference | mclustcomp::mclustcomp() |
Phi | Phi coefficient. | clusterCrit::extCriteria() |
precision | precision | clusterCrit::extCriteria() |
Rand | Rand index | mclustcomp::mclustcomp() |
recall | recall | clusterCrit::extCriteria() |
RogersTanimoto | Rogers-Tanimoto dissimilarity | clusterCrit::extCriteria() |
RusselRao | Russell-Rao dissimilarity | clusterCrit::extCriteria() |
SMC | Simple matching coefficient | mclustcomp::mclustcomp() |
splitJoin | total split-join index | igraph::split_join_distance() |
splitJoin.ref | Split-join index of the first model to the second model. In other words, it is the edit-distance between the two partitionings. | |
SokalSneath1 | Type-1 Sokal-Sneath dissimilarity | clusterCrit::extCriteria() |
SokalSneath2 | Type-2 Sokal-Sneath dissimilarity | clusterCrit::extCriteria() |
VI | Variation of information | mclustcomp::mclustcomp() |
Wallace1 | Type-1 Wallace criterion | mclustcomp::mclustcomp() |
Wallace2 | Type-2 Wallace criterion | mclustcomp::mclustcomp() |
WMSSE | Weighted minimum sum of squared errors between cluster trajectories | |
WMMSE | Weighted minimum mean of squared errors between cluster trajectories | |
WMMAE | Weighted minimum mean of absolute errors between cluster trajectories |