SCHEDULE: NOV 10-16, 2012
When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.
High Performance RDMA-Based Design of HDFS over InfiniBand
SESSION: Grids/Clouds Networking
EVENT TYPE: Papers
TIME: 4:00PM - 4:30PM
SESSION CHAIR: Michael Lang
AUTHOR(S):Nusrat S. Islam, Md W. Rahman, Jithin Jose, Raghunath Rajachandrasekar, Hao Wang, Hari Subramoni, Chet Murthy, Dhabaleswar K. Panda
ROOM:255-EF
ABSTRACT:
Hadoop Distributed File System (HDFS) acts as primary storage of Hadoop and has been adopted by reputed organizations (Facebook, Yahoo! etc.) due to its portability and fault-tolerance. The existing implementation of HDFS uses Java-socket interface for communication which delivers suboptimal performance in terms of latency and throughput. For data-intensive applications, network performance becomes key component as the amount of data being stored and replicated to HDFS, increases. In this paper, we present a novel design of HDFS using Remote Direct Memory Access (RDMA) over InfiniBand via JNI interfaces. Experimental results show that, for 5GB HDFS file writes, the new design reduces the communication time by 87% and 30% over 1Gigabit Ethernet (1GigE) and IP-over-InfiniBand (IPoIB), respectively, on QDR platform (32Gbps). For HBase, the Put operation performance is improved by 20% with our new design. To the best of our knowledge, this is the first design of HDFS over InfiniBand networks.
Chair/Author Details:
Michael Lang (Chair) - Los Alamos National Laboratory
Nusrat S. Islam - Ohio State University
Md W. Rahman - Ohio State University
Jithin Jose - Ohio State University
Raghunath Rajachandrasekar - Ohio State University
Hao Wang - Ohio State University
Hari Subramoni - Ohio State University
Chet Murthy - IBM T.J. Watson Research Center
Dhabaleswar K. Panda - Ohio State University
Click here to download .ics calendar file
Click here to download .vcs calendar file
Click here to add event to your Google Calendar
High Performance RDMA-Based Design of HDFS over InfiniBand
SESSION: Grids/Clouds Networking
EVENT TYPE:
TIME: 4:00PM - 4:30PM
SESSION CHAIR: Michael Lang
AUTHOR(S):Nusrat S. Islam, Md W. Rahman, Jithin Jose, Raghunath Rajachandrasekar, Hao Wang, Hari Subramoni, Chet Murthy, Dhabaleswar K. Panda
ROOM:255-EF
ABSTRACT:
Hadoop Distributed File System (HDFS) acts as primary storage of Hadoop and has been adopted by reputed organizations (Facebook, Yahoo! etc.) due to its portability and fault-tolerance. The existing implementation of HDFS uses Java-socket interface for communication which delivers suboptimal performance in terms of latency and throughput. For data-intensive applications, network performance becomes key component as the amount of data being stored and replicated to HDFS, increases. In this paper, we present a novel design of HDFS using Remote Direct Memory Access (RDMA) over InfiniBand via JNI interfaces. Experimental results show that, for 5GB HDFS file writes, the new design reduces the communication time by 87% and 30% over 1Gigabit Ethernet (1GigE) and IP-over-InfiniBand (IPoIB), respectively, on QDR platform (32Gbps). For HBase, the Put operation performance is improved by 20% with our new design. To the best of our knowledge, this is the first design of HDFS over InfiniBand networks.
Chair/Author Details:
Michael Lang (Chair) - Los Alamos National Laboratory
Nusrat S. Islam - Ohio State University
Md W. Rahman - Ohio State University
Jithin Jose - Ohio State University
Raghunath Rajachandrasekar - Ohio State University
Hao Wang - Ohio State University
Hari Subramoni - Ohio State University
Chet Murthy - IBM T.J. Watson Research Center
Dhabaleswar K. Panda - Ohio State University
Click here to download .ics calendar file