![](bg8.png)
possible quality without violating time constraints
(e.g., deadline). We discussed several policies in or-
der to either give more priority for early applications
or to obtain a fair quality between all the applications.
6 FUTURE WORK
The future work goes into several directions. First, we
consider to define a unified DSL that combines sev-
eral frameworks and not only Spark. Second, we will
introduce interface place holders within a Spark appli-
cation to allow more expressive composition. Third,
we will support several clusters and hence efficiently
transfer data at synchronization points become a chal-
lenging task. Another direction is to extend the con-
troller to take as input a graph of Spark applications
(not only a sequence). For this, the controller should
be augmented with a scheduler to select the best next
Spark application in addition to the next quality level.
REFERENCES
Buttazzo, G. C., Lipari, G., and Abeni, L. (1998). Elastic
task model for adaptive rate control. RTSS, pages 286–
295.
Cascading (2016). Cascading. http://www.cascading.org.
Chambers, C., Raniwala, A., Perry, F., Adams, S., Henry,
R. R., Bradshaw, R., and Weizenbaum, N. (2010).
Flumejava: easy, efficient data-parallel pipelines. In
Proceedings of the 2010 ACM SIGPLAN Conference
on Programming Language Design and Implementa-
tion, PLDI 2010, Toronto, Ontario, Canada, June 5-
10, 2010, pages 363–375.
Combaz, J., Fernandez, J., Sifakis, J., and Strus, L. (2008).
Symbolic quality control for multimedia applications.
Real-Time Systems, 40(1):1–43.
Combaz, J., Fernandez, J.-C., Lepley, T., and Sifakis, J.
(2005b). Qos control for optimality and safety. Pro-
ceedings of the 5th Conference on Embedded Soft-
ware.
Dean, J. and Ghemawat, S. (2004). Mapreduce: Simplified
data processing on large clusters. In 6th Symposium on
Operating System Design and Implementation (OSDI
2004), San Francisco, California, USA, December 6-
8, 2004, pages 137–150.
Dean, J. and Ghemawat, S. (2010). Mapreduce: a flexible
data processing tool. Commun. ACM, 53(1):72–77.
Ghemawat, S., Gobioff, H., and Leung, S. (2003). The
google file system. In Proceedings of the 19th ACM
Symposium on Operating Systems Principles 2003,
SOSP 2003, Bolton Landing, NY, USA, October 19-
22, 2003, pages 29–43.
Giraph (2016). Apache giraph. http://giraph.apache.org.
Gounaris, A., Kougka, G., Tous, R., Tripiana, C., and Tor-
res, J. (2017). Dynamic configuration of partitioning
in spark applications. IEEE Transactions on Parallel
and Distributed Systems, PP(99):1–1.
Mahout (2016). Apache mahout. http://mahout.apache.org.
Malewicz, G., Austern, M. H., Bik, A. J. C., Dehnert,
J. C., Horn, I., Leiser, N., and Czajkowski, G. (2010).
Pregel: a system for large-scale graph processing. In
Proceedings of the ACM SIGMOD International Con-
ference on Management of Data, SIGMOD 2010, In-
dianapolis, Indiana, USA, June 6-10, 2010, pages
135–146.
Meng, X., Bradley, J. K., Yavuz, B., Sparks, E. R.,
Venkataraman, S., Liu, D., Freeman, J., Tsai, D. B.,
Amde, M., Owen, S., Xin, D., Xin, R., Franklin, M. J.,
Zadeh, R., Zaharia, M., and Talwalkar, A. (2015).
Mllib: Machine learning in apache spark. CoRR,
abs/1505.06807.
R.I.Davis, K.W.Tindell, and A.Burns (1993). Scheduling
slack time in fixed priority pre-emptive systems. Pro-
ceeding of the IEEE Real-Time Systems Symposium,
pages 222–231.
Spark MLlib (2016). Spark mllib.
http://spark.apache.org/mllib/.
Valiant, L. G. (2011). A bridging model for multi-core com-
puting. J. Comput. Syst. Sci., 77(1):154–166.
Wust, C. C., Steffens, L., Verhaegh, W. F., Bril, R. J., and
Hentschel, C. (2004). Qos control strategies for high-
quality video processing. Euromicro Conference on
Real-Time Systems, pages 3–12.
Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S.,
and Stoica, I. (2010). Spark: Cluster computing with
working sets. In 2nd USENIX Workshop on Hot Topics
in Cloud Computing, HotCloud’10, Boston, MA, USA,
June 22, 2010.
Reconfigurable and Adaptive Spark Applications
91