Comparison of Distributed Data-Parallelization Patterns for Big Data Analysis: A Bioinformatics Case Study

Files

Comparison_of_Distributed_Data-Parallelization_Patterns_for_Big_Data_Analysis_A_Bioinformatics_Case_Study(2013).pdf (324.49 KB)
 DDP-Comparison-Jianwu_Wang-UCSD_Slides.pdf (224.32 KB)

Links to Files

https://users.sdsc.edu/~jianwu/JianwuWang_files/Comparison_of_Distributed_Data-Parallelization_Patterns_for_Big_Data_Analysis_A_Bioinformatics_Case_Study(2013).pdf

Permanent Link

http://hdl.handle.net/11603/31627

Collections

UMBC Information Systems Department
UMBC Center for Accelerated Real Time Analysis
UMBC Center for Real-time Distributed Sensing and Autonomy
UMBC Computer Science and Electrical Engineering Department
UMBC Data Science
UMBC Joint Center for Earth Systems Technology (JCET)

Author/Creator

Author/Creator ORCID

https://orcid.org/0000-0002-9933-1170

Date

2013

Type of Work

conference papers and proceedings
presentations (communicative events)
preprints
Text

Department

Program

Citation of Original Publication

Rights

This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.

Subjects

UMBC Big Data Analytics Lab

Abstract

As a distributed data-parallelization (DDP) pattern, MapReduce has been adopted by many new big data analysis tools to achieve good scalability and performance in Cluster or Cloud environments. This paper explores how two binary DDP patterns, i.e., CoGroup and Match, could also be used in these tools. We reimplemented an existing bioinformatics tool,called CloudBurst, with three different DDP pattern combinations. We identify two factors, namely, input data balancing and value sparseness, which could greatly affect the performances using different DDP patterns. Our experiments show: (i) a simple DDP pattern switch could speed up performance by almost two times; (ii) the identified factors can explain the differences well.