![](bg9.png)
that the Sample method has better results than Sam-
pleSpace one, although the Pass Rate metric shows
inverse results. Remember that by sampling each cha-
racter according to the probability distribution in the
generative model, it has a higher rate of making in-
puts incorrect (Figure 3). One possible explanation
for this is that having a high rate of correct inputs
can make the program avoid some instructions that
were verifying the code’s correctness in more detail.
Thus, those instructions might be encountered by Tra-
cer only when the inputs given are a mix between cor-
rect and (slightly) invalid. In (Godefroid et al., 2017)
there is also a discussion about performing random
fuzzing over the inputs learned using RNN methods,
but similar to our evaluation, the results are not better
than the Sample method. The other technique pre-
sented in (H
¨
oschele and Zeller, 2016) that learns the
grammar of the input through dynamic tainting and
applicable currently only to Java programs, could not
be evaluated since the tool is not (yet) open-source
and could not be retrieved in any other way.
6 CONCLUSIONS AND FUTURE
WORK
This paper presented an open-source tool that is able
to assist users in automatic generation of test data for
evaluating programs, having as initial input a corpus
of example tests. Support for any kind of input file
formats, operating efficiently in distributed environ-
ments, online learning, and checkpoints are one of its
strongest features. The evaluation section shows the
efficiency of using recurrent neural networks to learn
generative models that are able to produce new tests,
from two main perspectives: improved instruction co-
verage over random fuzzing and the percent of cor-
rect input files produced from the learned model. As
future work, we plan to improve the clusterization
of files using autoencoders techniques that are able
to learn features from existing inputs, study the ef-
fectiveness of using Generative adversarial networks
(GANs) in improving tests coverage. Another topic is
to improve the usability of the tool by providing a vi-
sual interface for controlling parameters and injecting
expert knowledge in learning and generation proces-
ses in an easier way.
ACKNOWLEDGMENTS
This work was supported by a grant of Roma-
nian Ministry of Research and Innovation CCCDI-
UEFISCDI. project no. 17PCCDI/2018 We would
like to thank our colleagues Teodor Stoenescu and
Alexandra Sandulescu from Bitdefender, and to Alin
Stefanescu from University of Bucharest for fruitful
discussions and collaboration.
REFERENCES
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z.,
Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin,
M., Ghemawat, S., Goodfellow, I. J., Harp, A., Irving,
G., Isard, M., Jia, Y., J
´
ozefowicz, R., Kaiser, L., Kud-
lur, M., Levenberg, J., Man
´
e, D., Monga, R., Moore,
S., Murray, D. G., Olah, C., Schuster, M., Shlens, J.,
Steiner, B., Sutskever, I., Talwar, K., Tucker, P. A.,
Vanhoucke, V., Vasudevan, V., Vi
´
egas, F. B., Vinyals,
O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y.,
and Zheng, X. (2016). Tensorflow: Large-scale ma-
chine learning on heterogeneous distributed systems.
CoRR, abs/1603.04467.
Bastani, O., Sharma, R., Aiken, A., and Liang, P. (2017).
Synthesizing program input grammars. SIGPLAN
Not., 52(6):95–110.
Cho, K., van Merrienboer, B., G
¨
ulc¸ehre, C¸ ., Bougares, F.,
Schwenk, H., and Bengio, Y. (2014). Learning phrase
representations using RNN encoder-decoder for sta-
tistical machine translation. CoRR, abs/1406.1078.
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., and
Bengio, Y. (2015). Attention-based models for speech
recognition. CoRR, abs/1506.07503.
Coppit, D. and Lian, J. (2005). Yagg: An easy-to-use ge-
nerator for structured test inputs. In Proceedings of
the 20th IEEE/ACM International Conference on Au-
tomated Software Engineering, ASE ’05, pages 356–
359, New York, NY, USA. ACM.
Cui, W., Peinado, M., Chen, K., Wang, H. J., and Irun-Briz,
L. (2008). Tupni: Automatic reverse engineering of
input formats. In Proceedings of the 15th ACM Con-
ference on Computer and Communications Security,
CCS ’08, pages 391–402, New York, NY, USA. ACM.
Godefroid, P. (2007). Random testing for security: black-
box vs. whitebox fuzzing. In RT ’07.
Godefroid, P., Kiezun, A., and Levin, M. Y. (2008).
Grammar-based whitebox fuzzing. In Proceedings of
the 29th ACM SIGPLAN Conference on Programming
Language Design and Implementation, PLDI ’08, pa-
ges 206–215, New York, NY, USA. ACM.
Godefroid, P., Levin, M. Y., and Molnar, D. (2012).
Sage: Whitebox fuzzing for security testing. Queue,
10(1):20:20–20:27.
Godefroid, P., Peleg, H., and Singh, R. (2017). Learn&fuzz:
machine learning for input fuzzing. In Rosu, G.,
Penta, M. D., and Nguyen, T. N., editors, Proceedings
of the 32nd IEEE/ACM International Conference on
Automated Software Engineering, ASE 2017, Urbana,
IL, USA, October 30 - November 03, 2017, pages 50–
59. IEEE Computer Society.
ICSOFT 2018 - 13th International Conference on Software Technologies
480