1 / 11

nub - npoess user block tool

nub - npoess user block tool. Richard Ullman NASA GSFC. nub. nub (nŭb) n. 1. A protuberance or knob. 2. A small lump. 3. The essence; the core: the nub of a story. [Variant of knub, probably from Low German knubbe, from Middle Low German, variant of knobbe.]

zariel
Télécharger la présentation

nub - npoess user block tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. nub - npoess user block tool Richard Ullman NASA GSFC

  2. nub • nub (nŭb) n. • 1. A protuberance or knob. • 2. A small lump. • 3. The essence; the core: the nub of a story. • [Variant of knub, probably from Low German knubbe, from Middle Low German, variant of knobbe.] • (selected from http://www.answers.com/topic/nub) 2

  3. NPOESS user block • For NPOESS files, the user block is ASCII text written in XML that provides a quick-look into the metadata attributes contained in the HDF5 file. • The content of the user block includes metadata that is attached to the NPOESS: • / (root) group • <collection_shortname> group, and • <collection_shortname>_Aggr group. 3

  4. nub_Motivation • When evaluating NPOESS sample data last year, I wished to examine the user block, as a stand-alone item, but I found that the h5unjam tool to be expensive • h5unjam reads an HDF5 file and writes two output files • 1. a file containing the content of the user block • 2. a file containing the H5 data without the user block. • The second part was unnecessary for my purposes, but because the NPOESS files are large, it is expensive. • So .. I wrote nub … 4

  5. nub_Design • Design criteria: • “c” program (that’s all I know how to write) • Efficient - do no ‘EXTRA’ work. • Command line operation, with Unix-like options. • Pipe-able. (stdin/stdout) • Assume user block is text. • Work only on the front user block. • Complete set of functions to manipulate the user block: • List, Create, Overwrite, Append, Delete, Get size • Requirements creep … • Complete replacement for h5jam/h5unjam/tellub • No reliance on h5 library (no need). • Availability: • Applied for NASA Open Source release. All paperwork completed. Waiting for final approval. • Evaluation copy available to anyone interested in NPOESS. 5

  6. nub operates on the front user block ONLY signature: “\211HDF\r\n\32\n” HDF5 user block sizeof( user_block) = 2^n, where n>8, (or n=0) 6

  7. nub file.h5 signature: “\211HDF\r\n\32\n” HDF5 content,content/0/0/0/0/0/0/0/0 --tell -b --tell -a --tell -s • Tell: name comes from HDF5 internal tool “tellub” • This is the nub default operation. • Count the size of the user block content [-b], the size of the user block [-s], and the size of the complete H5 file [-a] in bytes. • No options given is equivalent to: nub --tell -bsa file.h5 • Output is similar to wc 7

  8. nub -l file.h5 signature: “\211HDF\r\n\32\n” file.txt HDF5 content,content/0/0/0/0/0/0/0/0 • List: list the content of the user block write to stdout, or if -f write to the given file 8

  9. nub -w file.h5 signature: “\211HDF\r\n\32\n” user_file.txt HDF5 content,content/0/0/0/0/0/0/0/0 • Write: write text content to the user block • append by default, or clobber (replace) • fit into existing space (by overwriting only null chars) by default, or “jam” (extend the block) • “kram” option shrinks user block into smallest that will fit the content • read from stdin, or use the file specified on the -u switch 9

  10. nub_ compatibility • nub will respond to the same command line options that are used with h5jam and h5unjam. • “nub -J” is an explicit substitute for h5jam • “nub -U” is an explicit substitute for h5unjam • “nub -T” is an explicit substitute for tellub • Or, use just “nub” as a substitute for either and the code will decide which of the two is most appropriate based on the flags provided. 10

  11. nub_test • Results of nub as a substitute for h5jam, h5unjam in the standard hdf5 tools regression tests … • 68 PASSED • 6 FAILED • fails are due to difference in concept of concatenation. • nub appends to the end of the content of an existing user block. • h5jam appends to the end of the existing user block. • User blocks are of size=2^n, where size > 256 and is big enough to fit the content. So, if user block content is 513 bytes, the size of the user block is 1024 bytes, including 511 bytes of NULL padding. • nub will overwrite the NULL padding, h5jam leaves the padding in place. 11

More Related