1 / 20

Data Management Practice: Analyzing Hurricane Pressure Data

This project focuses on data management and mining by analyzing formatted input data on hurricanes. The key objective is to create a refined list that includes 'year', 'stormid', and 'min.nonzero.pressure'. The approach involves sorting through lists to find minimum pressure values for specific storms. Despite challenges in processing raw data, the exercise of utilizing command-line scripts in CSH highlights the importance of accurate data manipulation. The insights gained from this analysis provide key information about hurricane frequency and pressure dynamics.

brenna-camacho
Télécharger la présentation

Data Management Practice: Analyzing Hurricane Pressure Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Homework 3Responses This was meant to give you practice with data management/mining given a non ideally formatted input data structure

  2. Sorting and List Management The strategy I'm testing on these shorter lists is to try and wind up with a list containing only ['year','stormid','min nonzero pressure'] - so I want to try and do it by sorting through the lists, for a given year and storm id finding the min pressure and adding to a new list... so far my attempts have been fruitless This was the typical problem encountered by some of you

  3. Some Success

  4. Many Counting Problems

  5. Producing the right file to “count” • What is the “problem” here with the raw data file?

  6. Code to Produce a new data file I could have filtered on “0” pressure but that would have reduced the actual number of Hurricanes carried forward

  7. Next Pass produces inital “counting” file

  8. Producing same counting file using csh scripts – practice this in csh! • 1. grep –i “HURR” master1.txt > new.txt • 2. sed s/NAMED//g new.txt > new1.txt • 3. **awk ‘{print $1 FS $6 FS $7}’ new1.txt > new2.txt • 4. sort –n –k 2 new2.txt > new3.txt New3.txt is then a file with 3 columns in which the FIRST occurrence of unique storm ID Is the lowest central pressure Actual files on following slides. ** could use cut –d” “ –f1 –f6 –f7 new1.txt > new2.txt but I always screw up CUT

  9. New.txt

  10. New1.txt

  11. New2.txt

  12. New3.txt Count this file anyway you like; Conditional IF statements are useful For counting.

  13. Science • An example of science that is enabled by doing list management operations on data bases • Current project, track analysis of Atlantic Basic Hurricanes – all previous analysis is only for LANDFALL events.

  14. Frequency: What detection efficiency improvement happened in 1950 relative to 1940?

  15. Lots of Information!

  16. Pressure analysis for strong storms < 965 mb; 2005 is dramatic!

  17. New Important Result: Dynamics are changing; evolution is faster

More Related