Analysis of the Capability and Training of Chat Bots in the Generation of Rules for Firewall or Intrusion Detection Systems
Sequeiros, J.
; Louro, B.
; Abreu, R.
;
Costa, J.
;
Inácio , P. R. M. I.
Analysis of the Capability and Training of Chat Bots in the Generation of Rules for Firewall or Intrusion Detection Systems, Proc ACM International Conference on Availability, Reliability and Security ARES , Viena, Austria, Vol. , pp. - , July, 2024.
Digital Object Identifier:
Download Full text PDF ( 509 KBs)
Abstract
This work explores various fine-tuning approaches to address this limitations. Three distinct approaches were done, each of them with a different objective and some more successful than others. The first approach aimed at testing how well the model was able to use the knowledge it got from the prompts when the question was structured differently and that got a success rate of 89%. The second approach aimed at testing how well the model could link the knowledge it got from two different prompts and that got a success rate of 61%. The third and final approach aimed at testing if the model could create complex rules by learning simple rules and that got a success rate of 79%. Besides these results there were diverse conclusions drawn from deeper analysis. It can be concluded that fine-tuning is a good way to approach the improvement of this problem and the future work includes the development of a model that used these and other approaches with the idea to help users with varying levels of knowledge and to include as many attacks, firewalls and IDS as possible.