L2OS threshold optimisation 20 June 2014, PM26

1. L2OS threshold optimisation 20 June 2014, PM26 JL Vergely, J. Boutin, P. Spurgeon ACRI-ST,LOCEAN, ARGANS

2. RFI/outlier detection Aim : • To improve the thresholds of the L2OS processor to be applied on TB measurements in order to remove outliers. About thresholds : • should be independent on L1c quality products

3. Thresholds to be tested test for outlier detection (dwell test) nsig test for out of range TB detection (FOV test) Tm_out_of_range_affov Tm_out_of_range_eaffov Tm_out_of_range_stokes3_affov Tm_out_of_range_stokes3_eaffov Tm_out_of_range_stokes4_affov Tm_out_of_range_stokes4_eaffov test for oscillation TB detection (FOV test) Ts_std Ts_std_stokes3 Ts_std_stokes4 Other tests : max of iteration

4. Tests conditions |X_swath| < 400 km Coast : 1000 km PCT_var < 80 -40° < lat < 40° SSS ref: Coriolis global NRTOA (MyOcean) Day : 1,2,3,4,5/5/2013, L1C v550 L2OS proc : v600 (CATDS processing chain)

5. Indicators / SSS quality filter • chi2P : good TB fit if chi2P high. Chi2P > 0.05 in current processor. Warning : Dg_chi2P in L2OS processor = 1-Chi2P • SSS error < 1.4 psu • mean(SSS SMOS – SSS Coriolis) and std(SSS SMOS – SSS Coriolis) • X = (SSS SMOS – SSS Coriolis)/SSS_error. X should be close to a Gaussian law with mean(X)=0 and std(X)=1. Does not depend on SSS accuracy (close to the ratio between empirical error and theoretical error).

6. Chi2P and RFI Percentage of RFI contamination : january 2012, asc Chi2P, 5/5/2013, asc

7. nsig : full ocean Outlier detection. Dwell test. TB removed if : |TBsmos – TBmodel – DA| > nsig.rad_noise DA = mean dwell correction Current value : 5 Tested values : 2, 3, 4, 5

8. nsig full ocean

9. nsig full ocean 4 sigmas test Queue distribution : outliers Expected distribution Centred reduced variable

10. nsig full ocean Mean and std(X) nsig = 2 No specific filter Chi2p > 0.05 & sigSSS < 1.35

11. nsig full ocean Mean and std(X) nsig = 3 No specific filter Chi2p > 0.05 & sigSSS < 1.35

12. nsig full ocean Mean and std(X) nsig = 4 No specific filter Chi2p > 0.05 & sigSSS < 1.35

13. nsig full ocean Mean and std(X) nsig = 5 No specific filter Chi2p > 0.05 & sigSSS < 1.35

14. nsig full ocean nsig=2: Many outliers at 4 sigmas

15. nsig : coast (1000km)

16. nsig coast Expected distribution

17. nsig coast Mean and std(X) nsig = 2 No specific filter Chi2p > 0.05 & sigSSS < 1.35

18. nsig coast Mean and std(X) nsig = 3 No specific filter Chi2p > 0.05 & sigSSS < 1.35

19. nsig coast Mean and std(X) nsig = 4 No specific filter Chi2p > 0.05 & sigSSS < 1.35

20. nsig coast Mean and std(X) nsig = 5 No specific filter Chi2p > 0.05 & sigSSS < 1.35

21. nsig coast nsig=2: Many outliers at 4 sigmas nsig=2 : very biased !!

22. Tm_out_of_range_affov or eaffov (polar X,Y,3,4) Snapshot removed if at least one TB is an outlier : |TB smos – TB model| > Tm_out_of_range Problem because the test is applied directly on the TBs and not on the TBs normalised by the radiometric noise X and Y from short and long integration time Current value : 50 K for affov and 100 K for eaffov Tested value : 10, 20, 30, 40 K

23. Tm_out_of_range_affov full ocean

24. Tm_out_of_range_affov full ocean Tm = 10K No specific filter Chi2p > 0.05 & sigSSS < 1.35

25. Tm_out_of_range_affov full ocean Tm = 40K No specific filter Chi2p > 0.05 & sigSSS < 1.35

26. Tm_out_of_range_affov full ocean

27. Tm_out_of_range_eaffov full ocean Tm=10K : Little bit better but lost of accuracy No significative change (with Tm_out_of_range_affov = 12)

28. Tm_out_of_range_stokes3_affov full ocean Tm=6K : Little bit better -> try to work in dual pol mode ? No significative change (with Tm_out_of_range_affov/eaffov = 12/18)

29. Tm_out_of_range_stokes3_eaffov full ocean No significative change (with Tm_out_of_range_stokes3_affov = 8)

30. Tm_out_of_range_stokes4_affov full ocean No significative change (with Tm_out_of_range_stokes3_affov/eaffov = 8/16)

31. Tm_out_of_range_stokes4_eaffov full ocean No significative change (with Tm_out_of_range_stokes3_affov/eaffov = 8/16 & Tm_out_of_range_stokes4_affov = 10)

32. Ts_std thresholds Snapshot is removed if : rms((TB smos –TB model)/ra) > Ts_std rms((TB smos –TB model)/ra) is expected to be close to 1 (if OTT well applied) Current value = 2.5

33. Ts_std full ocean Ts_std=1.5 : Little bit better

34. Ts_std_stokes3 full ocean Ts_std=0.5 or 1 : Too low Too biased No significative improvement

35. Ts_std_stokes4 full ocean Ts_std=1 : Too low Too inaccurate No significative improvement

36. Comparison current configuration and configuration without thresholds

37. Comp current/without thres. full ocean current conf Chi2p > 0.05 & sigSSS < 1.35

38. Comp current/without thres. full ocean without filter No signicative change Chi2p > 0.05 & sigSSS < 1.35

39. Comp current/without thres. full ocean A little bit better without thresholds

40. Comp current/without thres. coast Without thresholds : better for 4 sigmas SSS

41. 5 day processing 1,2,3,4,5/05/2013; nsig = 2, 3, 4, 5 Dwell test. TB removed if : |TBsmos – TBmodel – DA| > nsig.noise DA = mean dwell correction Current config : nsig = 5

42. Without LS mask. chi2P > 0.05 nsig=2

43. nsig=3

44. nsig=4

45. nsig=5

46. Iteration number 2 modes !!

47. Iteration number Signature TEC ? RFI or island ? Hot spot

48. Global improvement using iterMax Small global effect. What about specific area ?

49. 4 zones with RFI/coast contamination Pacific + coast Pacific Atlantic Indian ocean

50. 4 zones with RFI/coast contamination Pacific + coast Pacific Atlantic Indian ocean