Term Extraction from Financial News

1. Term Extraction from Financial News Jian-Shiun 2008/10/31

2. Financial News－鉅亨網

3. Data Collection • Period：2008/10/10 ~ 2008/10/30 • Number of news：1,987

4. Accumulated Grams

5. grams docs

6. Metrics • Frequency • Conditional Probability • Mutual Information

7. Mutual Information • If f(w) ≥ f(c1) f(c2)… f(cn), then Mi(w) ≥ 0

8. uni-gram (first 60 by freq)

9. bi-gram (first 30 by freq, MI>0)

10. bi-gram (first 30 by freq, MI<0)

11. tri-gram (first 30 by freq , MI>0)

12. tri-gram (first 30 by freq , MI<0)

13. 4-gram (first 30 by freq , MI>0)

14. 5-gram (first 30 by freq , MI>0)

15. Extreme Status Using MI • f(w) is very low, and MI is very high＊ • f(w) is very low, and MI is very low • f(w) is very high, and MI is very high＊ • f(w) is very high, and MI is very low

16. 1. f(w) is very low, and MI is very high＊

17. 2. f(w) is very low, and MI is very low

18. 3. f(w) is very high, and MI is very high＊

19. 4. f(w) is very high, and MI is very low

20. Further Work • PAT-Tree • Pattern Filter • Cross Validate with CKIP

21. Reference • 劉開瑛(2000)，中文文本自動分詞和標註，北京：商務印書館。