1 / 23

CrIMSS EDR Validation Team Recommendations for DR 7069 extension for QC

CrIMSS EDR Validation Team Recommendations for DR 7069 extension for QC. Susan Kizer and Xu Liu NASA Langley Research Center Mike Wilson, Murty Divakarla , Changyi Tan, Xiaozhen Xiong , Flavio Iturbide-Sanchez I.M Systems Group

jill
Télécharger la présentation

CrIMSS EDR Validation Team Recommendations for DR 7069 extension for QC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CrIMSS EDR Validation TeamRecommendations for DR 7069 extension for QC Susan Kizer andXu Liu NASA Langley Research Center Mike Wilson, MurtyDivakarla, Changyi Tan, XiaozhenXiong, Flavio Iturbide-Sanchez I.M Systems Group DeguiGu, Denise Hagan, and Xia L Ma Northrop Grumman Aerospace Systems Tony Reale, NOAA/NESDIS

  2. DR 7069 QC extension – Quality Flag Code Change(This slide provided by NGAS, slide 14 of “CrIMSS DR Status and MX 8.0 Recommendation”, CrIMSS Cal/Val Telecon 04/25/2013) • Original code IF (writeDebug) THEN CALL addLine('Overcast case, write out MW retrievals') END IF qcflag(1) = 1 qcflag(4) = 1 EXIT irLoop • Updated code IF (writeDebug) THEN CALL addLine('Overcast case, write out MW retrievals') END IF IF(.NOT. (mwOff)) THEN xgesgO3 = xgesmwsav ! Mw-only retrieval profile chisq_mw1(1) = chisqmw1 ! chisq for the first stage retrieval qcflag(1:5) = 1 crimssRetrLvlDataPtr%profDiff(currRet) = 0.0 ! qcflag(2) = 0 if(chisqmw1 <= 1.0 ) qcflag(5) = 0 ! The convergence flag for 1st stage retrieval END IF EXIT irLoop

  3. DR 7069 QC extension – Quality Flag Code Change (cont’d) (This slide provided by NGAS, slide 15 of “CrIMSS DR Status and MX 8.0 Recommendation”, CrIMSS Cal/Val Telecon 04/25/2013) • Original code xgesgO3 = xgesg xgesgO3(io3g:io3g+no3g-1) = xgesg(io3g:io3g+no3g-1) * 6.035E5 CALL getchisq_airs(ym,y,airs_err,chisqairs,kchan,nchan) CALL qc(chisq,chisqmw,chisqairs,nsurf,xgesg,xgesmwsav,plandavg, & qcflag, iflagqc) • Updated code IF(CC(ic) .eqv. .true.) THEN xgesgO3 = xgesg xgesgO3(io3g:io3g+no3g-1) = xgesg(io3g:io3g+no3g-1) * 6.035E5 CALL getchisq_airs(ym,y,airs_err,chisqairs,kchan,nchan) CALL qc(chisq,chisqmw,chisqairs,nsurf,xgesg,xgesmwsav,plandavg, & qcflag, iflagqc) END IF

  4. LaRC Proposed Changes to the QC Bug Presented by Mike and Bigyani • Original Code ! Initialize variables used during iteration chisq0 = CHISQ_INIT_VALUE chisq_ir = CHISQ_INIT_VALUE chisq_mw1 = CHISQ_INIT_VALUE chisq_mw2 = CHISQ_INIT_VALUE ! Start IR iteration: irLoop: DO iter = 1, maxIrIter • Updated Code ! Initialize variables used during iteration chisq0 = CHISQ_INIT_VALUE chisq_ir = CHISQ_INIT_VALUE chisq_mw1(1:maxIrIter) = chisqmw1 chisq_mw2 = CHISQ_INIT_VALUE ! Start IR iteration: irLoop: DO iter = 1, maxIrIter

  5. LaRC Proposed Changes to the QC Bug Presented by Mike and BigyaniCONTINUED • Original Code chisq_ir(iter) = chisq chisq_mw1(iter) = chisqmw1 chisq_mw2(iter) = chisqmw IF ( .NOT. cc(ic)) THEN ccnaf = 0.0001 END IF • Updated Code chisq_ir(iter) = chisq chisq_mw1(iter) = chisqmw1 chisq_mw2(iter) = chisqmw IF ( .NOT. cc(ic)) THEN NOTE: It is not NECESSARY that these 3 linesbeomitted. ccnaf = 0.0001This code will never bereached. END IF But ifitcosts $3000 toremoveit, then I am okay with leavingit in…

  6. LaRC Proposed Changes to the QC Bug Presented by Mike and BigyaniCONTINUED • Original Code ! Set chi-square quality flags CALL qc(chisq,chisqmw,chisqairs,nsurf,xgesg,xgesmwsav,plandavg,qcflag,iflagqc) • Updated Code ! Set chi-square quality flags CALL qc(chisq_ir(iter),chisq_mw2(iter),chisqairs,nsurf,xgesg,xgesmwsav,plandavg,qcflag,iflagqc) • Original Code ! If IR+MW retrieval good, save it as 1st guess for the next cluster ! otherwise, set the 1st guess to the previously determined best ! guess only relevant for sceneClassMode=1 IF (crimssConfigParamPtr%sceneClassMode==1) THEN IF (chisq < 2. .AND. chisqmw < 2.) THEN • Updated Code ! If IR+MW retrieval good, save it as 1st guess for the next cluster ! otherwise, set the 1st guess to the previously determined best ! guess only relevant for sceneClassMode=1 IF (crimssConfigParamPtr%sceneClassMode==1) THEN IF (chisq_ir(iter) < 2. .AND. chisq_mw2(iter) < 2.) THEN

  7. Testing LaRCand NGAS Proposed ChangesTesting Details • Chose to use data date Sept. 20, 2012, granule t0000259 (1st scan, 1st FOR) and simulate each instance that may cause an issue with reporting the proper chisq values and quality control flags. • My first thought was to run 107 granules of data date May 15, 2012 but there was no guarantee that we would be testing all the possible situations that could occur. • This granule has the following characteristics and we used its output as the baseline. • Clear Flag = FALSE, CC flag= TRUE • noIrRet = FALSE, mwOff = FALSE • For testing purposes, I reviewed the output within the IR netCDF output file. This file will contain intermediate information as well as data passed to post-processing (which creates the EDR product.)

  8. Testing LaRC Proposed ChangesTesting Details • Insertion of clrflg(ic) and cc(ic) where we test various combinations of setting these to TRUE and FALSE: • Original Code ! Start IR iteration: irLoop: DO iter = 1, maxIrIter • Modified Code for Testing clrflg(ic) = .TRUE. cc(ic) = .FALSE. print*, "cc(ic), clrflg(ic) = ", cc(ic), clrflg(ic), ic ! Start IR iteration: irLoop: DO iter = 1, maxIrIter • At the end of icLoop, I added a ‘stop’ to run only the first FOR for this granule. • Original Code mwCloud(:,currRet) = xgesmwsav(icldmwg:icldmwg+2) • Modified Code for Testing stop mwCloud(:,currRet) = xgesmwsav(icldmwg:icldmwg+2) Insertion of noIrRet and mwOffwhere we test variouscombinations of settingthese to TRUE and FALSE: OriginalCode !------------------------------------------------------------------- ! End of Stage 1 microwaveretrieval !------------------------------------------------------------------- IF ((maxIrIter == 0) .OR. noIrRet) THEN !no IR retrieval Modified Code for Testing !------------------------------------------------------------------- ! End of Stage 1 microwaveretrieval !------------------------------------------------------------------- !!noIrRet = .TRUE. !!mwOff= .TRUE. IF ((maxIrIter == 0) .OR. noIrRet) THEN !no IR retrieval ==>

  9. Possible Scenarios for Testing at LaRC

  10. Testing LaRCand NGAS Proposed Changes(NOTE: did not initialize qcFlag = FILL) qcFlagIr(1:12) cc=T,clrflg=F cc=F,clrflg=F cc=T,clrflg=T cc=F,clrflg=T noIrRet=F,mwOff=F noIrRet=F,mwOff=F noIrRet=F,mwOff=F noIrRet=F,mwOff=F (1) = 1 IR+MW retrieval chisq not within threshold 0 1 (0 IDPS) 0 1 (0 IDPS) (2) = 1, IR-MW profile difference not within 1 0 1 0 threshold (over ocean) (3) = 1 chisqAirs not within threshold 0 1 0 1 (4) = 1 2nd stage MW chisq not within threshold 1 1 (0 IDPS) 1 1 (0 IDPS) (5) = 1 MW did not converge 0 0 (1 NGAS) 0 0 (1 NGAS) (6) = 1 precipitation detected 1 1 1 1 (7) = 1 MW not available 0 0 0 0 (8) = 0 day, = 1 night 1 1 1 1 (9) = 0 clear scene, = 1 partly cloudy scene, 1 1 1 1 = 2 cloudy scene (10) = 0 ocean, = 1 land, = 2 coast 0 0 0 0 (11) = 1 ice over ocean 1 1 1 1 (12) = 1 non-LTE condition 0 0 0 0

  11. Testing LaRCand NGAS Proposed Changes(NOTE: did not initialize qcFlag = FILL) qcFlagIr(1:12) cc=T,clrflg=F cc=T,clrflg=F cc=T,clrflg=F cc=T,clrflg=F noIrRet=F,mwOff=F noIrRet=F,mwOff=T noIrRet=T,mwOff=F noIrRet=T,mwOff=T (1) = 1 IR+MW retrieval chisq not within threshold 00 (not F)0 0 (2) = 1, IR-MW profile difference not within 10 (not F) 0 0 threshold (over ocean) (3) = 1 chisqAirs not within threshold 00 (not F) 0 0 (4) = 1 2nd stage MW chisq not within threshold 1 0 (not F) 0 0 (5) = 1 MW did not converge 0 0 0 0 (6) = 1 precipitation detected 1 1 1 1 (7) = 1 MW not available 0 0 0 0 (8) = 0 day, = 1 night 1 1 1 1 (9) = 0 clear scene, = 1 partly cloudy scene, 1 1 0 0 = 2 cloudy scene (10) = 0 ocean, = 1 land, = 2 coast 0 0 0 0 (11) = 1 ice over ocean 1 1 1 1 (12) = 1 non-LTE condition 0 0 0 0

  12. Testing LaRC Proposed Changes When cc(ic) = FALSEIDPSLaRCNGAS (13) = 1 IR+MW retrieval did not converge 0 0 0 (14) = 1 MW only retrieval did not converge 0 0 0 (15) = 1 Overall Retrieval Quality – IR only 0 0 0 (16) = 1 Overall Retrieval Quality – MW only0 1 1 (17) = Overall Retrieval Quality – No Conv. 0 0 0 (18) = 1 Profile Diff Exceeds Threshold0 0 0 (19) = 1 Partly Cloudy, = 2 Cloudy111 (20) Non-LTE Flag000 (21) Precipitation Flag 1 1 1 (22) Retrieval Cell Size, 4 FOVs Used 0 0 0 (23) Retrieval Cell Size, 1 FOV Used 000 (24) Retrieval Cell Size, No Retrieval 0 0 0 (25) Temperature out of range 0 0 0 (26) Sun Glint present in retrieval 0 0 0 (27) ATMS data is not available 0 0 0 (28) Day/Night Flag = 0 Day, = 1 Night 1 1 1 (29) Ice Mask 00 0

  13. Testing LaRC Proposed ChangesWhat’s next for test completion • LaRChas tested these proposed changes in offline code. These changes have not been tested on IDPS code. • LaRC still needs to complete documentation of testing • Test results of ccNAF > 5. (make sure all instances of EXIT irLoop are represented in findings) • Need to show that the discrepancies in convergence rates (Bigyani’s findings) have been resolved • Did not test Mike’s original proposed solution to this bug

  14. NOTES The following slides contain additional notes on test results for each permutation of testing. Again, the IR netCDF output files were used for comparison of results.

  15. Testing LaRC Proposed ChangesPreliminary Results • When CC flag = FALSE • rms, chisq, chisq2 = 999999 for all iterations • noiseReduct = 0; CCFlag = 0 for all FOVs • Reports MW retrieval • NOTE: In the MW product, IR emissivity, IR reflectivity and O3 are reported as -999.5. Here the IR emissivity, IR reflectivity and O3 are reporting IR background values. • QC Flags • IR+MW retrieval chisq not within threshold = TRUE (1) • IR-MW profile difference not within threshold (over ocean) = FALSE (0) • chisqAirs not within threshold = TRUE (1) • 2nd stage MW chisq not within threshold = TRUE (1) • NOTE: No change from baseline

  16. Testing LaRC Proposed ChangesPreliminary Results • When Clrflg = TRUE • Profile: IR+MW profile with differing values from baseline • rms= values differ from baseline • chisq = values differ from baseline • chisq2 = values differ from baseline • noiseReduct = 0 • QcFlag= same as baseline • ClearFlag = 1 0 0 0 0 0 0 0 0 • CCFlag = same as baseline

  17. Testing LaRC Proposed Changes • d20120920_t0000259 • Scan = 1, FOR = 1 • Clear Flag and CC flag, F T • noIrRet = FALSE, mwOff = FALSE • Retr.ir.scene.nc • Profile – IR+ MW retrieval profile • Iteration = 4 • QcFlag = 0101F1F11F1F0010000110000000000 • Use as a baseline for this test

  18. Testing LaRC Proposed Changes • Clear Flag and CC flag, F F • noIrRet = FALSE, mwOff = FALSE • Retr.ir.scene.nc • Profile: MW retrieval (show how this differs from the reported MW retrieval) • Iteration = 1 • rms = 999999 x 4 • chisq = 999999 x 4 • chisq1 =999999 x 4 • LaRC = 1.00101 , 1.00101 , 1.00101 , 1.00101 • NGAS = 1.00101 , 999999 , 999999 , 999999 • chisq2 = 999999 x 4 • noiseReduct = 0 • QcFlag = 0010F1F11F1F0000000100000000000 • LaRC = 1011F1F11F1F0000000100000000000 • NGAS = 101111F11F1F0000000100000000000 • CCFlag = 0 x 9

  19. Testing LaRC Proposed Changes • Clear Flag and CC flag, T T • noIrRet = FALSE, mwOff = FALSE • Retr.ir.scene.nc • Profile: IR+MW profile with differing values from baseline • Iteration = 4 • rms = values differ from baseline • chisq= values differ from baseline • chisq1 = same as baseline • chisq2 = values differ from baseline • noiseReduct = 0 • QcFlag= same as baseline • ClearFlag = 1 0 0 0 0 0 0 0 0 • CCFlag = same as baseline

  20. Testing LaRC Proposed Changes • Clear Flag and CC flag, T F • noIrRet = FALSE, mwOff = FALSE • Retr.ir.scene.nc • Profile: MW retrieval • Iteration = 1 • rms = 999999 x 4 • chisq= 999999 x 4 • Chisq1 = 999999 x 4 • LaRC = 1.00101 , 1.00101 , 1.00101 , 1.00101 • NGAS = 1.00101 , 999999 , 999999 , 999999 • chisq2 = 999999 x 4 • noiseReduct = 0 • QcFlag = 0010F1F11F1F0000000100000000000 • LaRC = 1011F1F11F1F0000000100000000000 • NGAS = 101111F11F1F0000000100000000000 • ClearFlag= 1 0 0 0 0 0 0 0 0 • CCFlag = 000000000

  21. Testing LaRC Proposed Changes • Using the following run as baseline: Clear Flag and CC flag, F T noIrRet = FALSE, mwOff = FALSE Clear Flag and CC flag, F T • noIrRet = FALSE, mwOff = TRUE • Retr.ir.scene.nc • Profile: IR+MW retrieval • Iteration = 4 • rms = same as baseline • chisq= same as baseline • chisq1 = same as baseline • chisq2 = same as baseline • noiseReduct = same as baseline • QcFlag = 0000F1F11F1F0000000100000000000 • ClearFlag= same as baseline • CCFlag = same as baseline

  22. Testing LaRC Proposed Changes • Using the following run as baseline: Clear Flag and CC flag, F T noIrRet= FALSE, mwOff = FALSE • Clear Flag and CC flag, F T • noIrRet = TRUE, mwOff = FALSE • Retr.ir.scene.nc • Profile: MW retrieval with differences from baseline • Iteration = 0 • rms = 0 x 4 • chisq = 0 x 4 • chisq1 = 0 x 4 • chisq2 = 0 x 4 • QcFlag = FFFFF1F1FF1F0000000110000000000 • indexFov= 0 x 9 • nFovcc = 0 x 9 • nCluster = 0 • clearFlag = 0 x 9 • CCFlag = 0 x 9

  23. Testing LaRC Proposed Changes • Using the following run as baseline: Clear Flag and CC flag, F T noIrRet= FALSE, mwOff = FALSE • Clear Flag and CC flag, F T • noIrRet = TRUE, mwOff = TRUE • Retr.ir.scene.nc • Profile: MW retrieval with differences from baseline • Iteration = 0 • rms = 0 x 4 • chisq = 0 x 4 • chisq1 = 0 x 4 • chisq2 = 0 x 4 • noiseReduct = 0 • QcFlag = FFFFF1F1FF1F0000000110000000000 • indexFov= 0 x 9 • nFovcc = 0 x 9 • nCluster = 0 • clearFlag = 0 x 9 • CCFlag = 0 x 9

More Related