Exploring Data Patterns: Unraveling Insights Through Data Science and Analytics
This document delves into the intricate processes of data analysis and pattern recognition, focusing on the methodologies employed to identify valid, novel, and useful insights from large datasets. It covers historical developments in data management systems, including advancements in databases like DB2, OLAP systems, and tools like SPSS and SAS. The narrative also outlines the significance of proper data management and the evolution of techniques over the decades, ultimately leading to sophisticated predictive analytics and fraud detection strategies in contemporary data science.
Exploring Data Patterns: Unraveling Insights Through Data Science and Analytics
E N D
Presentation Transcript
1. ????? ????? : ?????? ???????
2. 2 ????? ????? ???
????? ?? ?? ???? ???? (?????? ??????? ? ??????)
?????? ???? ???? ?? ?????? ??? ????
?????? ???? ???? ?? ??????
????? ?? ?????? ???? ????
???? ????
???? ????
??? ????? ???????
????? ? ??? ???
??? ????? ??????
?????? ?????: ?????? ???? ???? ?? CRM ???? ????
3. 3 ????? ????? ????? ??? ?????? ???? ? ???????? ????? ???? ????? ??? ????? ???? ??
????? ????? ????? ?????? ??? ??? ????? ???? ?? ??? ????? ?????? ? ????
???? ?? ??????? ?????? ???? ??? ???? ?? ?????? ????? ?????
4. 4 ??????? ???? ???? ?? 1960
????? ????? ?? ? ??? ???? ? ?????? ???? ?? ???? IBM, CDC
????? ???? ?? ??? ???? ?? ? ??????????
??????? ????? (?????? ?? ??? ?? ??????? ?? 5 ??? ?????)
?? 1980
????? ???? ??? ? ?? ???? ???? ??????? ?? ?????? ????
???? ????? ? ????????? ???? ?? ?? DBMS ??? DB2 ? Oracle ? Sybase
??????? ???? ?? ??? ????? (????? ???? ?? ???? ?? ?? ???? ????? ??????)
?? 1990
????? ?????? ???? ??? ??? ????Data Warehouse ? OLAP
??????? ???? ?? ??? ??? (?? ????? Drill Down )
OLAP ??????? ????? ?? ???????? ????? ?? ???? ??? ??? ????? ????? ??? ????? ?????? ? ?? ??? ???? ???.
?? ??? ????
???????? ??????? ?????SPSS/Clementine, SGI, SAS
??? ??????? ???? ?? ?????? ???? ??
??????? ???? ?? ???? ????? ?? ?????
???? ?? ???? ?? ??? ????? ?? ?? ???? ??? ???? ???? ? ????
5. 5 ????? ???? ???? “…The non-trivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data…”
Fayyad, Piatetsky-Shapiro, Smyth [1996]
6. 6 ???? ??? ????? ?? ???? ????
7. 7 ??? ? ?? ??? ???? ????? ??? ????? ???? (???????)
??? ??? ????? ????? ?? ????? ???? ???.
???? ??? ?? ??? ???? ? ?? ????? ?????
?????? ? ?????
???? ??? ??????
??? ??????? ????? ? ????? ??? ???????? ???? ?? ???? ??? ????? ???? ??
????? ????? ???? ???? ????? ?????? ???? ??? ?????? ?? ?? ???? ???? ? ????? ? ???? ? ???? ??? ? ???? ???.
??? ????? ???? ?? ? ????? ?????? ??
?????? ?? ??????? ????? ?? ???? ???? ?? ???? ??
??? ???? ???? ???? ??
8. 8
9. 9 ?????? ???? ???? ???? ?? ??????
10. 10 ???? ??? ???? ???? ??????? ? ?????? ??? ????? ???? ??
???? ??? ????? ????
???? ?? ?????? ? ????? ??? ????? ?? ???? ????? ???? ???? ?? ????? ?????? ????? ? ????? ??? ???? ?????
?????? ??????? ? ?? ????? ???? ??? ???
??????? ?? ????? ? ???????? ??? ????? ???? ??????? ????
??? ??? ???? ???? ?????
???? ????? ? ???????? ???? ??
11. 11 ????????? ???? ???? ???? ?????
????? ??????? ???? ???????
????
??????? ????? ???? ???? ???? ??? ???? ???? ???????
?????
??????? ????? ?????? ????? ????? ?? ???? ????? ???????
????????
??? ???? ??????? ?????????? ?? ???? ??????? ???????
????? ??????? ????
12. 12 ?? ???? ???? ?? ???? ... ??? ???? ????? ????? ?? ???? ???? ?????
???? ???? ???? ? ????? ???? ??????? ????
???? ???? ???? ? ????? ?? ???? ???? ? ????? ??
??????? ?????? ?? ?????? ???? ?????? ?? ????
??????? ?????? ?? ??? ???? ???? ????? ?? ???
13. 13
14. 14 1. ???? ????
15. 15 ????? ?? ???? ???? ?? ???? ?????
16. 16 ????? ?? ???? ???? ?? ???? ????
17. 17 ????????? ???? ???? ????????? ??????
????? ???? - Fraud Detection
??????? ? ???? ???? ?????
????? ???????
18. 18 2. ???? ????
19. 19 ???? ???? ... ?????
??????? ???? ???? ??? ?? ??????? ??????? ? ????? ???????
??? ???? ?? ?????? ????
??????
????? ???? ????? ?????
??????? ???????
????????? ??????
20. 20 3. ??? ????? ??????? ??? ? ????? ???????? ?? ???? ?? ????? ?? ?????? ????? ???? ??? ???? ???
21. 21 ????????? ??? ????? ???????
?????? ?????? ? ?????
??????? ? ????????? ??? ????
?????? ?????? ???? ??? ???????
22. 22 4.????? ? ??? ??? ??? ??????? ?? ???? ?? ?????? ????? ????? ?????
???? ??????
????? ???? ?? ???? ?? ????
??? ??????? ???? ???????? ???? ???? ??? ???????
23. 5. ??? ????? ?????? ??????? ????? ????? ?????? ??????? ?????
????:40% ??????? ?? ?? ??????? ????? ???? ?? ??? ??? ????? ???? ????? ???.
??????
???? ??? ????? ???? ? ?????
?????
?????? ???? ????? ????
?????? ????? ?????
25. 25 ???? ?????? ?????? ?? ?????
26. 26 ?????? ?????: ???? ???? ????? ????? ???? ??????
???? ??? ????? ??? ???????? ?? ???? ???? ???? ???? ???? ?????? ???? ?????? ? ???? ??? ????
??????? ?????????? ???? ???? ?? ????
17000 ?????
62 ???? ??????
27. 27 ????? ????? ???? ???? ??? ????? ?????????? ????? ??????? ?????
???? ???? ?????? ?? ????? (? ?? ????????) ????? ?? ????.
?? ???? ??? ????: ??? ????????? ????? < ??? ??? ?????
?? ???? ???? ?????: ??? ????????? ????? > ??? ??? ?????
??? ????? ?????? ???? ????
????? ????? ?? ?????? ???? ??????? ???? ????? ????? ???? ???? ??? ????.
??? ??? ??????? ?? ????? ????? > ??????? ???? ????? ????? (?? ???? ???? ??????)
28. 28 ????? ????? ???? ???? ... ????? ??????? ???? ?? ?????? ????? ? ????? ????? ???? ??? ???????? ????? ???????
???????? ??? ??? ?????: ??????? ?? ????? (????? ???? ?? ???? ?????????)
??????? ??? ?????????: ???? ???????
?????
???? ????? ???? ????? ? ?????? ???? ?????? ?? ????? ????? ???? ??? ?? ????? ????? (???? ?????) ????? ?? ???.
29. 29 ???? ??? ??? ???? ????? ????
?? ???? ?? ?? ???? ?? ??????? ????? ???? ? ?????? ??? ???? ?? ????
???? ???? ????
????? ???? ?????? ?? ????? ? ????
?????? ??? ???? ???? ????