ICRAT – Philadelphia

1. Evaluation of feasible machine learning techniques for predicting the time to fly and aircraft speed profile on final approach ICRAT – Philadelphia Floris Herrema 22 June 2016

2. Overview • Background and problem • Machine Learning steps – example A320 • Techniques • Case studies • Verification • Conclusions and recommendations

3. Background – Solutions Weather Dependant Separation Minimum Separation All solutions define optimised time separations from different perspectives T1 T10 Taxi-Time Enhanced Procedures T2 T9 ORD T8 T3 RECAT-EU T7 T4 Runway Occupancy Time T6 T5 Time Based Separation Pairwise Separation CSPR Operations Mixed Mode Ops Background - ML steps - Feasible ML techniques - Case studies - Verification & Conclusion

4. Background – Solutions Weather Dependant Separation Minimum Separation Since all minima shall be respected, only the most constraining one is considered by the ORD Tool: Time separation = max ( [T1,T2,…,Tn] ) T1 T10 Taxi-Time Enhanced Procedures T2 T9 ORD T8 T3 RECAT-EU T7 T4 Runway Occupancy Time T6 T5 Time Based Separation Pairwise Separation CSPR Operations Mixed Mode Ops

5. Background – Previous steps EZY475A M 015 ↓ 160 A319 IBE652R H 005 ↓ 135 A343

6. Background – Previous steps EZY475A M 025 ↓ 160 B772 IBE652R H 015 ↓ 160 ATR Systematic analysis of years of radar tracks has allowed to better predict the buffers

7. Background – Previous steps EZY475A M 025 ↓ 160 B772 Initial Target Distance Buffer to be applied Final Target Distance IBE652R H 015 ↓ 160 ATR This experience will not be sufficient to safety deploy advanced concepts, like pair-wise separations, that increase variability in the separations to be delivered and therefore in the compression buffer to be considered.

8. Background – The problem • The efficient deployment of such concept needs a reliable prediction of the airport operations, aircraft behaviour, what-if cases, complex interactions between ground and air operations short term evolution of specific meteorological parameters. This is the objective of my paper • How Machine Learning (ML) techniques may be used for predicting the T2F and TAS profile on final approach. The ML techniques have been assessed on their forecast performance, computational time and amount of data needed for delivering a reliable prediction.

9. Background – Overview Segment Prediction responses Prediction variables Feature analysis and Principle Component Analysis (PCA)

10. Background – Overview Segment Prediction responses Prediction variables Feature analysis and Principle Component Analysis (PCA)

11. Background – Prediction variables

12. Machine Learning steps – Pre processing • Compute T2F and TAS for each sample: The T2F and TAS profile are extracted for each segment of 0.5NM, 5kts wind band and 50 aircraft types. • Feature selection: The RreliefF technique is applied before a model is learned.

13. Machine Learning steps – Pre processing RreliefF

14. Machine Learning steps – Pre processing • Compute T2F and TAS for each sample: The T2F and TAS profile are extracted for each segment of 0.5NM, 5kts wind band and aircraft type. • Feature selection: The RreliefF technique is applied before a model is learned. • Principal Component Analysis (PCA): finding out which features are important for best describing the variance in a data set.

15. Machine Learning steps – Pre processing PCA

16. Machine Learning steps – Pre processing PCA

17. Machine Learning steps – PCA & RreliefF

18. Machine Learning steps – PCA & RreliefF

19. Machine Learning steps – PCA & RreliefF

20. Machine Learning steps – PCA & RreliefF

21. Machine Learning – example TAS A320

22. Machine Learning steps • Compute T2F and TAS for each sample: The T2F and TAS profile are extracted for each segment of 0.5NM, 5kts wind band and aircraft type. • Feature selection: The RreliefF technique is applied before a model is learned. • PCA: finding out which features are important for best describing the variance in a data set. • Construct the datasets: based on different aircraft performance and weather sets from airport 1 and 2. • Stability of two different data parts: split the matrices X andY in two subsets Xtrain; Ytrain; used to train the model and Xtest;Ytestused to evaluate the model accuracy.

23. Machine Learning – example TAS A320

24. Machine Learning steps • Compute T2F and TAS for each sample: The T2F and TAS profile are extracted for each segment of 0.5NM, 5kts wind band and aircraft type. • Feature selection: The RreliefF technique is applied before a model is learned. • PCA: finding out which features are important for best describing the variance in a data set. • Construct the datasets: based on different aircraft performance and weather sets from airport 1 and 2. • Stability of three different data parts: split the matrices X andY in two subsets Xtrain; Ytrain; used to train the model and Xtest;Ytestused to evaluate the model accuracy. • Accuracy of data and outliers: in the last pre-processing step the accuracy is measured and the outliers are shown.

25. Machine Learning – example TAS A320

26. Machine Learning – feasible techniques • Remember! • The objective of the paper was to show how Machine Learning (ML) techniques may be used for predicting the T2F and TAS profile on final approach. • Therefore the previous described steps have been applied on different ML techniques and have been assessed on their; 1. Forecast performance, 2. Computational time 3. Amount of data needed for delivering a reliable prediction.

27. Feasible ML techniques

28. Case studies – Airport 1 B738 and Airport 2 A320 • We analysed two T2F case studies using the Lasso, MLP and Ensemble techniques.

29. Case studies – results

30. Case studies – results

31. Benchmark MBA with ML ensemble methods • Machine learning are more robust to behaviour changes.

32. Main Conclusions • Learning multitask regularized regression with RreliefF is promising especially in combination with PCA. • Ground speed and other information at 10NM together with headwind information seem to capture a lot of the variation of the T2F and TAS in the last 10NM. • The multi task techniques Lasso and MLP turned out to be the best feasible and most accurate techniques for predicting the TAS and T2F. • By learning a ML model with HW, the MSE is significantly lower than without HW for both RMSE from 8.5-0.5NM and from 4.5 till 0.5NM. • The ML techniques are more accurate and more robust to changes and they improve in overall over the accuracy of the statistical approach. Also the maximum error of our ensemble model is lower compared to MBA.

33. Recommendations • The most important prediction variable – GS at 10NM – might give some operational issues. • Learn new features and find a subspace that captures the variation of the data using PCA dimension reduction. • Learn one task at a time in order to see if the multi-task approach helps and validate that the multi-task approach lead to better results.

34. With this study a better prediction will be established of the T2F and TAS on final approach using feasible machine learning techniques, this research will stimulate further Dynamic Pair-wise Separation studies….

35. Thank you Any questions?