encrypted h1c, which is almost always faster than h2.
The new h2 Push mechanism looks especially promis-
ing on “warm” TCP connections but seems difficult
to fine-tune when pushing files along with the initial
HTML file (see Section 4.2.2).
Finally, using the Speeder framework to run our
experiments on a large number of emulated test setup
permutations has shown to be a useful approach.
Many implementation inefficiencies were discovered
by comparing different metrics, browsers and network
conditions and looking for inconsistencies. Analyz-
ing and comparing both synthetic and realistic test
data was also effective, with the more realistic results
indeed somewhat dampening the more extreme syn-
thetic results but largely showing similar trends. Fur-
thermore, the two concrete website examples in Sec-
tion 5.1 show that individual datapoints can have sub-
tle underlying reasons for their observed values, some
having little to do with the underlying protocol or im-
plementations but originating in the HTML structure
itself. They also show that even a deep comparison
of different setups is not always enough to find im-
portant problems and that tools should also support
manual confirmation of results through a variety of
visualizations. We can conclude that the Speeder
framework is a versatile platform usable not only by
researchers but also developers to assess both global
trends for comparisons, and local optimizations for
individual websites.
7 CONCLUSION
In this work we have used the Speeder framework to
run experiments for both synthetic and realistic test
cases over a variety of different setups, allowing us
to compare h2s to h1s and h1c, Chrome to Firefox,
loadEventEnd to SpeedIndex and good to bad net-
work conditions.
Our results have shown that most of the cases in
which h2 currently underperforms are caused by un-
optimized implementations and that most of the real
inherent h2 problems are caused by the use of a sin-
gle underlying TCP connection. This can lead h2 per-
formance to deteriorate on poor networks with high
packet loss, but in practice we have seen that the dif-
ference with h1 under these conditions is usually not
that large. Globally speaking, we find that h2 is often
on a par with h1s and a little slower than h1c.
We have found that all three discussed h1 best
practices of resource concatenation, resource embed-
ding and hostname sharding perform similarly over
h1 and h2 in non-extreme circumstances. Especially
concatenation and sharding can have a large impact
on performance, while embedding and its h2 Push al-
ternative are observed to be micro-optimizations that
are difficult to fine-tune (at least for the initial page
load). These findings lead us to recommend to devel-
opers that they often do not have to significantly alter
their current setups when deciding to migrate from h1
to h2 or to support both versions in parallel.
We argue that, in time, the envisioned ideal h2
model of using many small, individually cacheable
files over a single, well filled TCP connection will
become viable with improving browser/server imple-
mentations and the QUIC protocol.
We will keep expanding and sharing the Speeder
framework (e.g., with the h2o server, Google Light-
house, sitespeed.io v4), using it to follow browser
implementation changes, as well as more deeply in-
vestigate the possibilities of h2 Push and h2 priorities
in various settings.
ACKNOWLEDGEMENTS
This work is part of the imec ICON PRO-FLOW
project. The project partners are among others Nokia
Bell Labs, Androme, Barco and VRT. Robin Marx is a
SB PhD fellow at FWO, Research Foundation - Flan-
ders, project number 1S02717N. Thanks to messrs
Goel, Wijnants, Michiels, Robyns, Menten, Bonn
´
e
and our anonymous reviewers for their help.
REFERENCES
Belshe, M., Peon, R., and Thomson, M. (2015). Hyper-
text transfer protocol version 2. https://tools.ietf.org/
html/rfc7540. Last visited: 2017-03-01.
Bergan, T., Pelchat, S., and Buettner, M. (2016).
Rules of thumb for http/2 push. https://docs.
google.com/document/d/1K0NykTXBbbbTlv60t5My
JvXjqKGsCVNYHyLEXIx-YMv0. Last visited:
2017-03-01.
Carlucci, G., De Cicco, L., and Mascolo, S. (2015). Http
over udp: an experimental investigation of quic. In
Proceedings of the 30th Annual ACM Symposium on
Applied Computing, pages 609–614. ACM.
Corbel, R., Stephan, E., and Omnes, N. (2016). Http/1.1
pipelining vs http2 in-the-clear: Performance com-
parison. In 2016 13th International Conference on
New Technologies for Distributed Systems (NOTERE),
pages 1–6.
de Oliveira, I. N., Endo, P. T., Melo, W., Sadok, D., and
Kelner, J. (2016). Should i wait or should i push? a
performance analysis of push feature in http/2 connec-
tions. In Proceedings of the 2016 workshop on Foster-
ing Latin-American Research in Data Communication
Networks, pages 7–9. ACM.
WEBIST 2017 - 13th International Conference on Web Information Systems and Technologies
172