1 / 8

Proposal for Nonblocking File Manipulation Routines in MPI

This proposal outlines nonblocking file manipulation routines for MPI, including MPI_File_iopen, MPI_File_iclose, MPI_File_isync, MPI_File_iset_size, and others. These enhancements allow applications to open files and set views without immediate blocking, enabling computations to progress while I/O operations are pending. Key features include valid but inactive file handles, the ability to batch operations, and synchronization mechanisms, designed to optimize performance in high-level file access scenarios.

vinny
Télécharger la présentation

Proposal for Nonblocking File Manipulation Routines in MPI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nonblocking File Manipulation Routines The HDF Group

  2. Proposal • MPI_File_iopen (MPI_Commcomm, char* filename, intamode, MPI_Info info, MPI_File *fh, MPI_Request *req); • MPI_File_iclose (MPI_Filefh, MPI_Request *req); • MPI_File_isync (MPI_File file, MPI_Request *req); • MPI_File_iset_view(MPI_Filefh, MPI_Offsetdisp, MPI_Datatypeetype, MPI_Datatypefiletype, char *datarep, MPI_Info info, MPI_Request *req); • MPI_File_iset_size (MPI_Filefh, MPI_Offset size, MPI_Request *req); • MPI_File_ipreallocate (MPI_Filefh, MPI_Offset size, MPI_Request *req); • MPI_File_iset_info ( MPI_Filefh,MPI_Infoinfo, MPI_Request *req); Straw Vote at July meeting: 15 – 1 – [5(need to think), 1(doesn’t care)]

  3. Usecase • Applications that Open the file (and sets view) then go to compute. File access happens later. • Hide the cost of opening the file: • Collective Communication • Actual open • Applications that just dump data to file after a certain amount of computation: • Compute, (open write close), Compute • Hide the I/O step

  4. MPI_File_IOPEN • Sets up a valid but inactive file handle and returns it to the user. • User can call anything on the file handle, however nothing will progress before the file handle becomes active. • A blocking operation will block for the file handle to become active and the operation is performed. • A nonblocking operation will return immediately. Progress will start after the file handle becomes active. The MPI library would record internally all the operations that are called on the inactive file handle.

  5. MPI_FILE_ISET_VIEW • All I/O operations issued before the iset_view will use the old view. • The iset_view will wait for all pending operations to complete before changing the view. • All I/O operations issued after the iset_view will use the new file view. • Blocking operations would block for the iset_view to complete. • Nonblocking operations would not progress before the iset_view completes.

  6. MPI_File_ISYNC • MPI_FILE_ISYNC will not sync operations called after it was called. • MPI_FILE_ISYNC will sync all write operations before it was called even if they have not completed.

  7. MPI_FILE_ISET_SIZE • I/O operations called before the iset_size will complete before the file is resized • Access to the file after the iset_size is done after the iset_size completes. • Same thing applies to MPI_FILE_ISET_INFO and MPI_FILE_IPREALLOCATE

  8. MPI_FILE_ICLOSE • In contrary to MPI_FILE_CLOSE, outstanding operations on the file handle will complete before actually closing the file.

More Related