SC12 Home > SC12 Schedule > SC12 Presentation - PLFS/HDFS: HPC Applications on Cloud Storage

SCHEDULE: NOV 10-16, 2012

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

PLFS/HDFS: HPC Applications on Cloud Storage

SESSION: Research Poster Reception

EVENT TYPE: Posters and Electronic Posters

TIME: 5:15PM - 7:00PM

SESSION CHAIR: Torsten Hoefler

AUTHOR(S):Chuck Cranor, Milo Polte, Garth Gibson

ROOM:East Entrance

ABSTRACT:
Long running large scale HPC applications protect themselves from failures by periodically checkpointing their state to a single file stored in a distributed network filesystem. These filesystems commonly provide a POSIX-style interface for reading and writing files. HDFS is a filesystem used in cloud computing by Apache Hadoop. HDFS is optimized for Hadoop jobs that do not require full POSIX I/O semantics. Only one process may write to an HDFS file, and all writes are appends. Our work enables multiple HPC processes to checkpoint their state into an HDFS file using PLFS. PLFS is a middleware filesystem that converts random I/O into log-based I/O. We added a new I/O store layer to PLFS that allows it to use non-POSIX filesystems like HDFS as backing store. HPC applications can now checkpoint to HDFS, allowing HPC and cloud to share the same storage systems and work with each others data.

Chair/Author Details:

Torsten Hoefler (Chair) - ETH Zurich

Chuck Cranor - Carnegie Mellon University

Milo Polte - Carnegie Mellon University

Garth Gibson - Carnegie Mellon University

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

PLFS/HDFS: HPC Applications on Cloud Storage

SESSION: Research Poster Reception

EVENT TYPE:

TIME: 5:15PM - 7:00PM

SESSION CHAIR: Torsten Hoefler

AUTHOR(S):Chuck Cranor, Milo Polte, Garth Gibson

ROOM:East Entrance

ABSTRACT:
Long running large scale HPC applications protect themselves from failures by periodically checkpointing their state to a single file stored in a distributed network filesystem. These filesystems commonly provide a POSIX-style interface for reading and writing files. HDFS is a filesystem used in cloud computing by Apache Hadoop. HDFS is optimized for Hadoop jobs that do not require full POSIX I/O semantics. Only one process may write to an HDFS file, and all writes are appends. Our work enables multiple HPC processes to checkpoint their state into an HDFS file using PLFS. PLFS is a middleware filesystem that converts random I/O into log-based I/O. We added a new I/O store layer to PLFS that allows it to use non-POSIX filesystems like HDFS as backing store. HPC applications can now checkpoint to HDFS, allowing HPC and cloud to share the same storage systems and work with each others data.

Chair/Author Details:

Torsten Hoefler (Chair) - ETH Zurich

Chuck Cranor - Carnegie Mellon University

Milo Polte - Carnegie Mellon University

Garth Gibson - Carnegie Mellon University

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar