Task Allocation in Hybrid Big Data Analytics for Urban IoT Applications
Loading...
Links to Files
Permanent Link
Author/Creator
Author/Creator ORCID
Date
2020-09-14
Type of Work
Department
Program
Citation of Original Publication
Weilong Ding, Zhuofeng Zhao, Jianwu Wang, Han Li. Task Allocation in Hybrid Big Data Analytics for Urban IoT Applications. ACM/IMS Transactions on Data Science, 1(3), pages 1-22, 2020. https://doi.org/10.1145/3374751
Rights
This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
Subjects
Abstract
In urban Internet of Things (IoT) environments, data generated in real time could be processed by analytical applications in online or offline mode. In the management perspective of runtime environments, such modes can hardly be supported in a unified framework under multiple restrictions such as latency, utility, and QoS (quality of service). Meanwhile in the optimization perspective of specific applications, it is difficult for current infrastructure to efficiently allocate sufficient resources to tasks of an application, simultaneously considering multiple factors such as data size, velocity, and locality. In this article, two task allocation methods are proposed for batch and stream analytics to improve resource utility with auto-scaling guarantee when an analytical application is submitted or sudden workloads appear. Taking the highway domain as an example, the task allocation methods are implemented in a novel combined framework accordingly. Using both real-world and simulated data, extensive experiments show that our methods can improve utility efficiency with effective offload support.