Download
1 / 3
30 likes | 142 Vues
This chapter discusses the implications of using massive datasets in machine learning, exploring key challenges such as memory constraints and the efficiency of various learning methods. It highlights the naive Bayes method and the differences between incremental and non-incremental learning schemes. Important concepts include the impact of dataset size on model performance, the risk of overfitting, and the benefits of parallelization in learning algorithms. Furthermore, the chapter delves into incorporating domain knowledge through metadata and practical applications like text and web mining and adversarial situations.
E N D
Learning from Massive Datasets • Can it be held in main memory?---Naïve Byaes Method • Some learning schemes are incremental; some are not. • What about time it takes to model?—should be linear or near linear • What to do when data set is too large? • Use a small subset of data for training---law of diminishing returns • Some schemes do better with more data; but there is also a danger of overfitting • Parallelization is another way---develop parallelized versions of learning schemes
Incorporating Domain Knowledge :Metadata---data about data---semantic, causal, and functional • Text and web mining: • Adversarial situations: Junk email filtering, for example
