Table 7: Fine-Tuning D
UNSW
with Five Examples Per
Class.
Fine-Tuning UNSW
Class Support Precision (%) F1 (%) Recall (%)
Bulb 12 100 100 100
Camera 203 46.87 54.98 66.50
Computer 342 98.57 75.22 80.81
Mobile 101 59.00 72.51 94.05
Motion Sensor 134 91.66 15.06 8.20
Printer 22 95.65 97.77 100
Router 80 96.72 83.68 73.75
Speaker 70 100 93.95 88.60
Switch 50 33.89 47.61 80
Macro Avg 1023 72.23 64.08 67.19
Weighted Avg 1023 80.26 64.59 63.73
Total Accuracy 63.73
Table 8: Fine-Tuning D
ZBW
with Five Examples Per Class.
Fine-Tuning ZBW
Class Support Precision (%) F1 (%) Recall (%)
Bulb 1 100 100 100
Camera 115 33.33 1.69 0.86
Computer 0 0 0 0
Mobile 23 90.90 58.82 43.47
Motion Sensor 0 0 0 0
Printer 0 0 0 0
Router 860 100 0.46 0.23
Speaker 9 5.14 9.65 77.77
Switch 4 0 0 0
Macro Avg 1012 36.58 18.95 24.70
Weighted Avg 1012 90.97 2.10 2.07
Total Accuracy 2.07
We successfully classify devices on these three
networking protocols; using prompt tuning, the LLM
successfully classified devices on a smaller dataset
reaching an accuracy of 79.44%. Through fine tuning,
the LLM successfully classified devices on a larger
dataset reaching an accuracy of 63.73%. The F1 score
for these two instances are also in-line with them, dif-
fering by no more than two. The LLM outperforms
the traditional models with the same data distribution.
For future work, we plan to transition to us-
ing LLMs entirely for this task by generating wider
datasets and further incorporating the knowledge em-
bedded in pre-trained language models. Furthermore,
we plan to evaluate this and similar tasks with other
LLMs (e.g., Llama 2) as the results of this work imply
feasibility in the use of LLMs for network classifica-
tion tasks, thus opening potential avenues for their use
in network security and privacy.
REFERENCES
Allifah, N. M. and Zualkernan, I. A. (2022). Ranking secu-
rity of iot-based smart home consumer devices. Ieee
Access, 10:18352–18369.
Banerji, S. and SinghaChowdhury, R. (2013). Wi-fi & wi-
max: A comparative study. CoRR, abs/1302.2247.
Chang, T. A. and Bergen, B. K. (2022). Word Acquisition
in Neural Language Models. Transactions of the As-
sociation for Computational Linguistics, 10:1–16.
Chataut, R., Phoummalayvane, A., and Akl, R. (2023). Un-
leashing the power of iot: A comprehensive review
of iot applications and future prospects in healthcare,
agriculture, smart homes, smart cities, and industry
4.0. Sensors, 23(16):7194.
Danbatta, S. J. and Varol, A. (2019). Comparison of zig-
bee, z-wave, wi-fi, and bluetooth wireless technolo-
gies used in home automation. In 2019 7th Interna-
tional Symposium on Digital Forensics and Security
(ISDFS), pages 1–5.
Draper-Gil., G., Lashkari., A. H., Mamun., M. S. I., and A.
Ghorbani., A. (2016). Characterization of encrypted
and vpn traffic using time-related features. In Pro-
ceedings of the 2nd International Conference on In-
formation Systems Security and Privacy - ICISSP,,
pages 407–414. INSTICC, SciTePress.
Ergen, S. C. (2004). Zigbee/ieee 802.15. 4 summary. UC
Berkeley, September, 10(17):11.
Habibi Lashkari, A., Draper Gil, G., Mamun, M., and Ghor-
bani, A. (2017). Characterization of tor traffic using
time based features. pages 253–262.
Hamdaoui, B. and Elmaghbub, A. (2022). Deep-learning-
based device fingerprinting for increased lora-iot se-
curity: Sensitivity to network deployment changes.
IEEE network, 36(3):204–210.
Hao, Q. and Rong, Z. (2023). Iottfid: An incremental iot de-
vice identification model based on traffic fingerprint.
IEEE Access.
Hayes, M. (2018). What is a network traffic flow?
Hegselmann, S., Buendia, A., Lang, H., Agrawal, M., Jiang,
X., and Sontag, D. (2023). Tabllm: Few-shot classi-
fication of tabular data with large language models.
In International Conference on Artificial Intelligence
and Statistics, pages 5549–5581. PMLR.
Horyachyy, O. (2017). Comparison of Wireless Communi-
cation Technologies used in a Smart Home: Analysis
of wireless sensor node based on Arduino in home au-
tomation scenario. PhD thesis.
Hu, T., Dubois, D. J., and Choffnes, D. (2023). Be-
haviot: Measuring smart home iot behavior using
network-inferred behavior models. In Proceedings of
the 2023 ACM on Internet Measurement Conference,
pages 421–436.
Kandpal, N., Wallace, E., and Raffel, C. (2022). Dedu-
plicating training data mitigates privacy risks in lan-
guage models. In International Conference on Ma-
chine Learning, pages 10697–10707. PMLR.
Kumar, S., Dalal, S., and Dixit, V. (2014). The osi model:
Overview on the seven layers of computer networks.
International Journal of Computer Science and Infor-
mation Technology Research, 2(3):461–466.
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mo-
hamed, A., Levy, O., Stoyanov, V., and Zettlemoyer,
L. (2020). Bart: Denoising sequence-to-sequence pre-
training for natural language generation, translation,
and comprehension. In Proceedings of the 58th An-
IoT Device Classification Using Link-Level Features for Traditional Machine Learning and Large Language Models
307