Big data provenance: Challenges, state of the art and opportunities

Files

Big_Data_Provenance-2015.pdf (765.41 KB)

Links to Files

https://ieeexplore.ieee.org/document/7364047

Permanent Link

https://doi.org/10.1109/BigData.2015.7364047
 http://hdl.handle.net/11603/31616

Collections

UMBC Information Systems Department
UMBC Center for Accelerated Real Time Analysis
UMBC Center for Real-time Distributed Sensing and Autonomy
UMBC Computer Science and Electrical Engineering Department
UMBC Data Science
UMBC Faculty Collection
Load more

Author/Creator

Author/Creator ORCID

https://orcid.org/0000-0002-9933-1170

Date

2015-12-28

Type of Work

conference papers and proceedings
preprints
Text

Department

Program

Citation of Original Publication

J. Wang, D. Crawl, S. Purawat, M. Nguyen and I. Altintas, "Big data provenance: Challenges, state of the art and opportunities," 2015 IEEE International Conference on Big Data (Big Data), Santa Clara, CA, USA, 2015, pp. 2509-2516, doi: 10.1109/BigData.2015.7364047.

Rights

© 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Subjects

UMBC Big Data Analytics Lab

Abstract

Ability to track provenance is a key feature of scientific workflows to support data lineage and reproducibility. The challenges that are introduced by the volume, variety and velocity of Big Data, also pose related challenges for provenance and quality of Big Data, defined as veracity. The increasing size and variety of distributed Big Data provenance information bring new technical challenges and opportunities throughout the provenance lifecycle including recording, querying, sharing and utilization. This paper discusses the challenges and opportunities of Big Data provenance related to the veracity of the datasets themselves and the provenance of the analytical processes that analyze these datasets. It also explains our current efforts towards tracking and utilizing Big Data provenance using workflows as a programming model to analyze Big Data.