Compute external model metric(s)

Compute one or more external metrics for two or more objects.

Note that there are many external metrics available, and there exists no external metric that works best in all scenarios. It is recommended to carefully consider which metric is most appropriate for your use case.

Many of the external metrics depend on implementations in other packages:

• clusterCrit (Desgraupes 2018)

• mclustcomp (You 2018)

• igraph (Csardi and Nepusz 2006)

• psych (Revelle 2019)

See mclustcomp::mclustcomp() for a grouped overview of similarity metrics.

Call getInternalMetricNames() to retrieve the names of the defined internal metrics. Call getExternalMetricNames() to retrieve the names of the defined internal metrics.

# S4 method for lcModel,lcModel
externalMetric(
  object,
  object2,
  name = getOption("latrend.externalMetric"),
  ...
)

# S4 method for lcModels,missing
externalMetric(object, object2, name = "adjustedRand")

# S4 method for lcModels,character
externalMetric(object, object2 = "adjustedRand")

# S4 method for lcModels,lcModel
externalMetric(object, object2, name, drop = TRUE)

# S4 method for list,lcModel
externalMetric(object, object2, name, drop = TRUE)

Arguments

object

The object to compare to the second object

object2

The second object

name

The name(s) of the external metric(s) to compute. If no names are given, the names specified in the latrend.externalMetric option (none by default) are used.

...

Additional arguments.

drop

Whether to return a numeric vector instead of a data.frame in case of a single metric.

Value

For externalMetric(lcModel, lcModel): A numeric vector of the computed metrics.

For externalMetric(lcModels): A distance matrix of class dist representing the pairwise comparisons.

For externalMetric(lcModels, name): A distance matrix of class dist representing the pairwise comparisons.

For externalMetric(lcModels, lcModel): A named numeric vector or data.frame

containing the computed model metrics.

For externalMetric(list, lcModel): A named numeric vector or data.frame

containing the computed model metrics.

Supported external metrics

Metric name	Description	Function / Reference
adjustedRand	Adjusted Rand index. Based on the Rand index, but adjusted for agreements occurring by chance. A score of 1 indicates a perfect agreement, whereas a score of 0 indicates an agreement no better than chance.	mclustcomp::mclustcomp(), (Hubert and Arabie 1985)
CohensKappa	Cohen's kappa. A partitioning agreement metric correcting for random chance. A score of 1 indicates a perfect agreement, whereas a score of 0 indicates an agreement no better than chance.	psych::cohen.kappa(), (Cohen 1960)
F	F-score	mclustcomp::mclustcomp()
F1	F1-score, also referred to as the Sørensen–Dice Coefficient, or Dice similarity coefficient	mclustcomp::mclustcomp()
FolkesMallows	Fowlkes-Mallows index	mclustcomp::mclustcomp()
Hubert	Hubert index	clusterCrit::extCriteria()
Jaccard	Jaccard index	mclustcomp::mclustcomp()
jointEntropy	Joint entropy between model assignments	mclustcomp::mclustcomp()
Kulczynski	Kulczynski index	clusterCrit::extCriteria()
MaximumMatch	Maximum match measure	mclustcomp::mclustcomp()
McNemar	McNemar statistic	clusterCrit::extCriteria()
MeilaHeckerman	Meila-Heckerman measure	mclustcomp::mclustcomp()
Mirkin	Mirkin metric	mclustcomp::mclustcomp()
MI	Mutual information	mclustcomp::mclustcomp()
NMI	Normalized mutual information	igraph::compare()
NSJ	Normalized version of splitJoin. The proportion of edits relative to the maximum changes (twice the number of ids)
NVI	Normalized variation of information	mclustcomp::mclustcomp()
Overlap	Overlap coefficient, also referred to as the Szymkiewicz–Simpson coefficient	mclustcomp::mclustcomp() (M K and K 2016)
PD	Partition difference	mclustcomp::mclustcomp()
Phi	Phi coefficient.	clusterCrit::extCriteria()
precision	precision	clusterCrit::extCriteria()
Rand	Rand index	mclustcomp::mclustcomp()
recall	recall	clusterCrit::extCriteria()
RogersTanimoto	Rogers-Tanimoto dissimilarity	clusterCrit::extCriteria()
RusselRao	Russell-Rao dissimilarity	clusterCrit::extCriteria()
SMC	Simple matching coefficient	mclustcomp::mclustcomp()
splitJoin	total split-join index	igraph::split_join_distance()
splitJoin.ref	Split-join index of the first model to the second model. In other words, it is the edit-distance between the two partitionings.
SokalSneath1	Type-1 Sokal-Sneath dissimilarity	clusterCrit::extCriteria()
SokalSneath2	Type-2 Sokal-Sneath dissimilarity	clusterCrit::extCriteria()
VI	Variation of information	mclustcomp::mclustcomp()
Wallace1	Type-1 Wallace criterion	mclustcomp::mclustcomp()
Wallace2	Type-2 Wallace criterion	mclustcomp::mclustcomp()
WMSSE	Weighted minimum sum of squared errors between cluster trajectories
WMMSE	Weighted minimum mean of squared errors between cluster trajectories
WMMAE	Weighted minimum mean of absolute errors between cluster trajectories

Implementation

See the documentation of the defineExternalMetric() function for details on how to define your own external metrics.

References

Cohen J (1960). “A Coefficient of Agreement for Nominal Scales.” Educational and Psychological Measurement, 20(1), 37-46.

Csardi G, Nepusz T (2006). “The igraph software package for complex network research.” InterJournal, Complex Systems, 1695. https://igraph.org.

Desgraupes B (2018). clusterCrit: Clustering Indices. R package version 1.2.8, https://CRAN.R-project.org/package=clusterCrit.

Hubert L, Arabie P (1985). “Comparing Partitions.” Journal of Classification, 2(1), 193--218. ISSN 1432-1343, doi:10.1007/BF01908075 .

M K V, K K (2016). “A Survey on Similarity Measures in Text Mining.” Machine Learning and Applications: An International Journal, 3, 19-28. doi:10.5121/mlaij.2016.3103 .

Revelle W (2019). psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, Illinois. R package version 1.9.12, https://CRAN.R-project.org/package=psych.

You K (2018). mclustcomp: Measures for Comparing Clusters. R package version 0.3.1, https://CRAN.R-project.org/package=mclustcomp.

See also

metric

Other metric functions: defineExternalMetric(), defineInternalMetric(), getExternalMetricDefinition(), getExternalMetricNames(), getInternalMetricDefinition(), getInternalMetricNames(), metric()

Other lcModel functions: clusterNames(), clusterProportions(), clusterSizes(), clusterTrajectories(), coef.lcModel(), converged(), deviance.lcModel(), df.residual.lcModel(), estimationTime(), fitted.lcModel(), fittedTrajectories(), getCall.lcModel(), getLcMethod(), ids(), lcModel-class, metric(), model.frame.lcModel(), nClusters(), nIds(), nobs.lcModel(), plot-lcModel-method, plotClusterTrajectories(), plotFittedTrajectories(), postprob(), predict.lcModel(), predictAssignments(), predictForCluster(), predictPostprob(), qqPlot(), residuals.lcModel(), sigma.lcModel(), strip(), time.lcModel(), trajectoryAssignments()

Examples

data(latrendData)
method <- lcMethodLMKM(Y ~ Time, id = "Id", time = "Time")
model2 <- latrend(method, latrendData, nClusters = 2)
model3 <- latrend(method, latrendData, nClusters = 3)

if (require("mclustcomp")) {
  externalMetric(model2, model3, "adjustedRand")
}
#> Loading required package: mclustcomp
#> adjustedRand 
#>    0.6227153