| Title |
Spark Framework Based on a Heterogenous Pipeline Computing with OpenCL |
| Authors |
김대희(Daehee Kim) ; 박능수(Neungsoo Park) |
| DOI |
http://doi.org/10.5370/KIEE.2018.67.2.270 |
| Keywords |
HPC ; GPGPU ; OpenCL ; Apache spark ; Parallel computing |
| Abstract |
Apache Spark is one of the high performance in-memory computing frameworks for big-data processing. Recently, to improve the performance, general-purpose computing on graphics processing unit(GPGPU) is adapted to Apache Spark framework. Previous Spark-GPGPU frameworks focus on overcoming the difficulty of an implementation resulting from the difference between the computation environment of GPGPU and Spark framework. In this paper, we propose a Spark framework based on a heterogenous pipeline computing with OpenCL to further improve the performance. The proposed framework overlaps the Java-to-Native memory copies of CPU with CPU-GPU communications(DMA) and GPU kernel computations to hide the CPU idle time. Also, CPU-GPU communication buffers are implemented with switching dual buffers, which reduce the mapped memory region resulting in decreasing memory mapping overhead. Experimental results showed that the proposed Spark framework based on a heterogenous pipeline computing with OpenCL had up to 2.13 times faster than the previous Spark framework using OpenCL. |