![](bg8.png)
sharing, Air quality, Boston house data sets. It can
be noticed that in general FS-ELM manages to select
the variables better compared to the other approaches.
Precisely:
• On Bike sharing data set, FS-ELM performs well
in the variable selection compared to Lasso and
NFSN.
• On Air quality data set, the two first important
variables selected by FS-ELM and NFSN can es-
timate well the target variable.
• On the Boston house data set, FS-ELM performs
well compared to NFSN. Indeed, FS-ELM has the
minimum MSE for any number of selected vari-
ables. There are some variances in the MSE be-
cause there are only 508 samples.
Figure 6 shows the estimated value of the mean of
the MSE by 5-fold cross-validation between Y and its
estimate versus log(C) for each approach on the data
sets with several target variables. It can be noticed
that FS-ELM has greater stability for regularization
parameters than the other methods. For each approach
and each data set, the chosen C
∗
is described below:
• On Enb data set, C
∗
= 10
−4
for Multi-task Lasso,
C
∗
= 1, λ
∗
= 10
−3
for FS-ELM, C
∗
= 10
−4
for
NFSN.
• On Atp1d data set, C
∗
= 10
−2
for Multi-task
Lasso, C
∗
= 1, λ
∗
= 10
−2
for FS-ELM, C
∗
= 10
−2
for NFSN.
Once the regularization parameters have been
determined for each approach, the variables are
ranked for each approach. The number of important
variables taken successively is {1, 2, . . . , 8} on Enb
data set and {50, 100, 150, . . . , 400} on Atp1d data
set. Figure 7 shows the estimated value of the mean
of the MSE by 5-fold cross-validation between Y and
its estimate versus the number of important variables
taken successively for each approach and on the data
sets Enb, Atp1d. It can be noticed that in general,
FS-ELM manages to select well the relevant variables
and reach the best performance with Atp1d which is
the most challenging case.
The proposed method successfully selects the rel-
evant variables on regression problems for one and
several target variables. In addition, it can be no-
ticed that generally, FS-ELM selects better compared
to NFSN and Multi-task Lasso.
5 CONCLUSIONS
In this paper, starting from an approach that was ini-
tially proposed for a classification problem with a
single target variable, we first showed its feasibility
for regression problems with a single target variable,
then proposed an extension in the framework of multi-
output regression for variable selection with several
target variables. Finally, many experiments made on
synthetic data and real data confirm the effectiveness
of the proposed approach.
The future works would be to:
• Calculate the partial derivative of L
λ,C
(Θ) with re-
spect to the α
i
since it was not calculated in the
initial formulation and this work to improve the
optimization algorithm.
• Propose an approximation of the matrix division
made in Equation 8 to reduce the complexity of
the optimization.
• Apply the proposed extension to the unsupervised
nonlinear variable selection problems for contin-
uous variables.
ACKNOWLEDGEMENT
This work was supported by Labcom-DiTeX, a joint
research group in Textile Data Innovation between In-
stitut Franc¸ais du Textile et de l’Habillement (IFTH)
and Universit
´
e de Technologie de Troyes (UTT).
REFERENCES
Challita, N., Khalil, M., and Beauseroy, P. (2016). New
feature selection method based on neural network and
machine learning.
De Vito, S., Massera, E., Piga, M., Martinotto, L., and
Francia, G. (2008). On field calibration of an elec-
tronic nose for benzene estimation in an urban pol-
lution monitoring scenario. Sensors and Actuators B
Chemical.
Ding, C., Zhou, D., He, X., and Zha, H. (2006). R 1-
pca: Rotational invariant l 1-norm principal compo-
nent analysis for robust subspace factorization. In
ICML 2006 - Proceedings of the 23rd International
Conference on Machine Learning.
Fanaee-T, H. and Gama, J. (2013). Event labeling combin-
ing ensemble detectors and background knowledge.
Progress in Artificial Intelligence.
Gretton, A., Bousquet, O., Smola, A., and Sch
¨
olkopf, B.
(2005). Measuring statistical dependence with hilbert-
schmidt norms.
Harrison, D. and Rubinfeld, D. (1978). Hedonic housing
prices and the demand for clean air. Journal of Envi-
ronmental Economics and Management.
He, X., Cai, D., and Niyogi, P. (2005). Laplacian score for
feature selection.
NCTA 2023 - 15th International Conference on Neural Computation Theory and Applications
438