Supervised NDRLM: latent-variable regression and optimization
What ndrlm() adds
ndrlm() combines network-based dimensionality reduction
with one supervised model per dependent variable. It can reduce the
input side, the output side, both sides, or neither; retain excluded
indicators as direct connections; fit OLS, ridge, elastic-net, or LOESS
models; and tune NDA weights and thresholds with NSGA-II.
NDRLM is predictive and exploratory. A fitted path is not, by itself, evidence of a causal effect. Its role as a data-driven benchmark for structural models is developed in Kosztyán (2026a).
Published latent-input example
The example follows the article’s low-sample-size mtcars
analysis. The ten input indicators form two communities. The first
groups cyl, hp, qsec,
vs, and carb; the second groups
disp, drat, wt, am,
and gear. The two resulting input latent variables predict
mpg.
X_car <- mtcars[, setdiff(names(mtcars), "mpg")]
Y_car <- mtcars["mpg"]
input_membership <- c(
cyl = 1, disp = 2, hp = 1, drat = 2, wt = 2,
qsec = 1, vs = 1, am = 2, gear = 2, carb = 1
)
fit_latent <- ndrlm(
Y_car, X_car,
latents = "in",
optimize = FALSE,
membership.X = input_membership,
regression = "lm",
seed = 42
)
fit_latent
#> Network-based dimensionality reduction and regression
#> Call:
#> ndrlm(Y = Y_car, X = X_car, latents = "in", optimize = FALSE,
#> seed = 42, regression = "lm", membership.X = input_membership)
#>
#> Regression: lm | Latent mode: in | Optimized: no
#> Target: adj.r.square | Objective: 0.4508713
#> Independent variables: NDAin1, NDAin2
#> Dependent variables: mpg
#>
#> response metric
#> mpg 0.4508713
#>
#> Fit for mpg
#>
#> Call:
#> stats::lm(formula = lm_formula, data = lm_train)
#>
#> Residuals:
#> Min 1Q Median 3Q Max
#> -6.2287304 -2.7541419 -0.9093334 2.5742679 10.4369019
#>
#> Coefficients:
#> Estimate Std. Error t value Pr(>|t|)
#> (Intercept) 20.0906250 0.7895134 25.44684 < 2.22e-16 ***
#> NDAin1 -4.2225851 0.8065667 -5.23526 1.3191e-05 ***
#> NDAin2 0.2710497 0.8065667 0.33605 0.73925
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 4.466162 on 29 degrees of freedom
#> Multiple R-squared: 0.486299, Adjusted R-squared: 0.4508713
#> F-statistic: 13.72653 on 2 and 29 DF, p-value: 6.387067e-05
summary(fit_latent)
#> Network-based dimensionality reduction and regression
#> Call:
#> ndrlm(Y = Y_car, X = X_car, latents = "in", optimize = FALSE,
#> seed = 42, regression = "lm", membership.X = input_membership)
#>
#> Regression: lm | Latent mode: in | Optimized: no
#> Target: adj.r.square | Objective: 0.4508713
#> Independent variables: NDAin1, NDAin2
#> Dependent variables: mpg
#>
#> response metric
#> mpg 0.4508713
#>
#> Fit for mpg
#>
#> Call:
#> stats::lm(formula = lm_formula, data = lm_train)
#>
#> Residuals:
#> Min 1Q Median 3Q Max
#> -6.2287304 -2.7541419 -0.9093334 2.5742679 10.4369019
#>
#> Coefficients:
#> Estimate Std. Error t value Pr(>|t|)
#> (Intercept) 20.0906250 0.7895134 25.44684 < 2.22e-16 ***
#> NDAin1 -4.2225851 0.8065667 -5.23526 1.3191e-05 ***
#> NDAin2 0.2710497 0.8065667 0.33605 0.73925
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 4.466162 on 29 degrees of freedom
#> Multiple R-squared: 0.486299, Adjusted R-squared: 0.4508713
#> F-statistic: 13.72653 on 2 and 29 DF, p-value: 6.387067e-05The NDA object remains available inside the supervised model, so the measurement structure is inspectable rather than hidden.
data.frame(
indicator = colnames(fit_latent$NDAin$R),
community = unname(fit_latent$NDAin$membership)
)
#> indicator community
#> 1 cyl 1
#> 2 disp 2
#> 3 hp 1
#> 4 drat 2
#> 5 wt 2
#> 6 qsec 1
#> 7 vs 1
#> 8 am 2
#> 9 gear 2
#> 10 carb 1
round(fit_latent$NDAin$loadings, 3)
#> NDA1 NDA2
#> cyl 0.620 0.034
#> disp 0.618 0.075
#> hp 0.735 0.378
#> drat -0.468 0.490
#> wt 0.719 0.008
#> qsec -0.144 -0.526
#> vs -0.233 -0.278
#> am -0.499 0.612
#> gear -0.250 0.680
#> carb 0.627 0.590
coef(fit_latent$fits[[1]])
#> (Intercept) NDAin1 NDAin2
#> 20.0906250 -4.2225851 0.2710497
fit_latent$fit_stats
#> response metric
#> 1 mpg 0.4508713The latent scores are computed only from their own community
indicators. The OLS coefficients then describe the fitted relationship
between each input latent variable and mpg.
Predictions and residual diagnostics
prediction <- predict(fit_latent, X_car)
diagnostic <- data.frame(
observed = Y_car$mpg,
fitted = fitted(fit_latent)$mpg,
residual = residuals(fit_latent)$mpg
)
head(prediction)
#> mpg
#> Mazda RX4 24.76310
#> Mazda RX4 Wag 24.02762
#> Datsun 710 24.07228
#> Hornet 4 Drive 18.79777
#> Hornet Sportabout 20.59865
#> Valiant 17.83850
head(diagnostic)
#> observed fitted residual
#> 1 21.0 24.76310 -3.7630984
#> 2 21.0 24.02762 -3.0276225
#> 3 22.8 24.07228 -1.2722817
#> 4 21.4 18.79777 2.6022336
#> 5 18.7 20.59865 -1.8986459
#> 6 18.1 17.83850 0.2614997op <- par(mfrow = c(1, 2), mar = c(4, 4, 3, 1))
plot(diagnostic$observed, diagnostic$fitted,
pch = 19, col = "#0072B2",
xlab = "Observed mpg", ylab = "Fitted mpg",
main = "Observed versus fitted")
abline(0, 1, col = "grey45", lty = 2)
plot(diagnostic$fitted, diagnostic$residual,
pch = 19, col = "#009E73",
xlab = "Fitted mpg", ylab = "Residual",
main = "Residual pattern")
abline(h = 0, col = "grey45", lty = 2)
Observed versus fitted fuel efficiency and residual diagnostic for the latent-input model.
These plots are descriptive in-sample diagnostics. A predictive study should evaluate the complete reduction-and-regression pipeline using held-out data or resampling.
Input, output, and full latent models
latents |
Predictors used by regression | Responses used by regression |
|---|---|---|
"none" |
observed X |
observed Y |
"in" |
input NDA scores | observed Y |
"out" |
observed X |
output NDA scores |
"both" |
input NDA scores | output NDA scores |
The next executable example reduces both sides. Two observed response
indicators form one output latent variable. With
dircon = TRUE, membership zero would retain an excluded
indicator as a direct observed variable.
X_both <- mtcars[, c("cyl", "disp", "hp", "wt", "qsec", "vs", "am", "gear")]
Y_both <- mtcars[, c("mpg", "drat")]
fit_both <- ndrlm(
Y_both, X_both,
latents = "both",
optimize = FALSE,
membership.X = c(1, 2, 1, 2, 1, 1, 2, 2),
membership.Y = c(1, 1),
regression = "lm",
seed = 42
)
fit_both$NDAin$membership
#> [1] 1 2 1 2 1 1 2 2
fit_both$NDAout$membership
#> [1] 1 1
fit_both$indep_names
#> [1] "NDAin1" "NDAin2"
fit_both$dep_names
#> [1] "NDAout1"
head(fitted(fit_both))
#> NDAout1
#> Mazda RX4 0.97433368
#> Mazda RX4 Wag 0.79549782
#> Datsun 710 0.33047165
#> Hornet 4 Drive -0.32859439
#> Hornet Sportabout -0.01532464
#> Valiant -0.47118761Regression engines with visible results
The enclosing NDRLM interface is the same for every engine. OLS provides coefficient tests and intervals. Ridge stabilizes correlated predictors. Elastic net combines shrinkage with coefficient selection. LOESS provides a small-predictor nonlinear model but no linear-path significance tests.
fit_ols <- fit_latent
if (requireNamespace("glmnet", quietly = TRUE)) {
fit_ridge <- ndrlm(
Y_car, X_car,
latents = "in", optimize = FALSE,
membership.X = input_membership,
regression = "ridge", lambda = 0.05,
seed = 42
)
fit_elastic <- ndrlm(
Y_car, X_car,
latents = "in", optimize = FALSE,
membership.X = input_membership,
regression = "elastic_net", alpha = 0.5, lambda = 0.05,
seed = 42
)
engine_results <- rbind(
OLS = fit_ols$fit_stats,
Ridge = fit_ridge$fit_stats,
Elastic_net = fit_elastic$fit_stats
)
engine_results
round(cbind(
ridge = fit_ridge$fits[[1]]$coefficients,
elastic_net = fit_elastic$fits[[1]]$coefficients
), 3)
} else {
cat("Install the suggested package 'glmnet' to run ridge and elastic net.\n")
}
#> ridge elastic_net
#> NDAin1 -4.187 -4.176
#> NDAin2 0.265 0.240fit_loess <- ndrlm(
Y_car,
X_car["wt"],
latents = "none",
regression = "loess",
loess_span = 0.80
)
head(predict(fit_loess, X_car["wt"]))
#> mpg
#> Mazda RX4 22.18677
#> Mazda RX4 Wag 21.07864
#> Datsun 710 24.99968
#> Hornet 4 Drive 19.40155
#> Hornet Sportabout 17.93234
#> Valiant 17.81902
fit_loess$fit_stats
#> response metric
#> 1 mpg 0.7931395Raw coefficients from different engines or differently scaled predictors are not standardized effect-size comparisons. Compare prediction metrics on a common validation design when prediction is the scientific goal.
NSGA-II optimization and convergence history
When optimize = TRUE, NSGA-II tunes indicator weights
and the four NDA thresholds. This deliberately small run demonstrates
the complete interface. Larger populations, more generations, and
repeated runs are appropriate for a final analysis.
fit_optimized <- ndrlm(
Y_car, X_car,
latents = "in",
membership.X = input_membership,
optimize = TRUE,
target = "adj.r.square",
rel_weight = TRUE,
lower.bounds.x = rep(0.5, ncol(X_car)),
upper.bounds.x = rep(1.5, ncol(X_car)),
lower.bounds.latentx = c(0, 0, 0, 0),
upper.bounds.latentx = c(0.01, 0.01, 0.01, 0.01),
popsize = 4,
generations = 3,
restarts = 1,
regression = "lm",
seed = 42
)
fit_optimized$optimized
#> [1] TRUE
fit_optimized$fval
#> [1] 0.6360032
round(fit_optimized$NDAin_weight, 3)
#> [1] 0.971 1.314 1.269 1.364 0.911 1.106 0.517 1.367 0.573 0.609
tail(fit_optimized$NSGA_trace)
#> restart evaluation generation target_value cost best_target
#> 11 1 11 2 0.6134205 -0.6134205 0.6248127
#> 12 1 12 2 0.6167337 -0.6167337 0.6248127
#> 13 1 13 3 0.6360032 -0.6360032 0.6360032
#> 14 1 14 3 0.6293834 -0.6293834 0.6360032
#> 15 1 15 3 0.6259359 -0.6259359 0.6360032
#> 16 1 16 3 0.6185751 -0.6185751 0.6360032trace <- fit_optimized$NSGA_trace
plot(
trace$evaluation,
trace$best_target,
type = "s", lwd = 2, col = "#0072B2",
xlab = "Objective evaluation",
ylab = "Best adjusted R-squared",
main = "NSGA-II search history"
)
points(trace$evaluation, trace$target_value,
pch = 16, cex = 0.65, col = grDevices::adjustcolor("grey35", 0.45))
lines(trace$evaluation, trace$best_target, lwd = 2, col = "#0072B2")
Best adjusted R-squared encountered during the compact NSGA-II run.
The grey points are evaluated candidates; the blue step line is the best target value encountered so far. A short flat segment is not proof of global convergence, but the trace makes search progress transparent and helps compare longer runs and restarts.
Latent path network
path_network <- plot(
fit_latent,
sig = 1,
interactive = FALSE,
show_edge_labels = TRUE,
shape = "automatic"
)
path_network
Latent-input NDRLM graph. Rectangles are observed indicators, circles are latent variables, and path colour carries sign.
Observed variables are rectangles and latent variables are circles, following standard structural-model notation. Each latent variable and its observed indicators share one community colour. Measurement loadings are grey. Positive regression paths are green, negative paths are red, and absolute magnitude is mapped to width separately within loadings and structural paths. All lines are solid by default. Colours, line types, shapes, and the minimum/maximum width can be overridden explicitly when a publication requires another convention.
References
The references connect the original and generalized NDA methods to the package implementation and the supervised network-based structural extension (Kosztyán et al. 2022, 2024; Kosztyán 2023; Kosztyán 2026b, 2026a).