Multiple Instance Learning with Query Bags
This paper provides a comprehensive review of Multiple Instance Learning (MIL) focusing on the Query Bag model. It begins with an overview of MIL principles and typical applications across various domains like vision, audio, and biology, highlighting the ambiguity in training data. The discussion extends to filtering strategies and the implications of bag sizes on learning outcomes. Experimental comparisons of classical versus query bags on real datasets, such as MNIST, are presented, showcasing how bag configurations affect performance. The work aims to unpack the potential of query bags in capturing underlying data distributions and improving MIL algorithms.
Multiple Instance Learning with Query Bags
E N D
Presentation Transcript
Multiple Instance Learning with Query Bags Boris Babenko, Piotr Dollar, Serge Belongie [In prep. for ICML’09 – feedback appreciated!]
Outline • Multiple Instance Learning (MIL) Review • Typical MIL Applications • Query Bag Model for MIL • Filtering Strategies • Conclusion
Outline • Multiple Instance Learning (MIL) Review • Typical MIL Applications • Query Bag Model for MIL • Filtering Strategies • Conclusion
Multiple Instance Learning (MIL) • Ambiguity in training data • Instead of instance/label pairs, get bagof instances/label pairs • Bag is positive if one or more of it’s members is positive
Multiple Instance Learning (MIL) • Supervised Learning Training Input • MIL Training Input • Goal: learning instance classifier
MIL Assumptions • Bags are predefined/fixed & finite (size ) • Bag label determined by: • Typical assumption: instances all drawn i.i.d. • Refer to this as a classical bag
MIL Theory • Best known PAC bound due to Blum et al. • = dimensionality • = bag size • = the desired error. • Problem harderfor larger bags • Result relies on i.i.d. assumption
Outline • Multiple Instance Learning (MIL) Review • Typical MIL Applications • Query Bag Model for MIL • Filtering Strategies • Conclusion
MIL Applications • Most MIL Apps: bag generated by breaking object into many overlapping pieces. • Let’s see some examples…
Vision • Image known to contain object, but precise location unknown [Andrews et al. 02, Viola et al. 05]
Audio • Audio wave can be broken up spatiallyor in frequency domain [Saul et al. 01]
Biological Sequences • Known to contain short subsequence of interest ACTGTGTGACATGTAGC { ACTG, CTGT, TGTG…} … [Ray et al. 05]
Text • Text document broken down into smaller pieces [Andrews et al. 02]
Observations • Sliding windows: bags are large/infinite. • In practice, bag is sub-sampled • Could violate the assumption ! • Instances of bag not independent – often lie on low dim. manifold (i.e. image patches)
Outline • Multiple Instance Learning (MIL) Review • Typical MIL Applications • Query Bag Model for MIL • Filtering Strategies • Conclusion
Query Bags for MIL • Bag not fixed – can query oracle to get arbitrary number of instances • Each query bag represented by object • To retrieve instances, use query functionwith location parameter
Query Bag for MIL • Instances often lie on low dim. manifold • Can query for nearby instances
Query Bags for MIL • Can express bag as • Define bag label as
Distribution of locations • Assume for each bag there is some distribution (known or unknown) • Could provide some prior information. • Let , how informative is
Query Bag Size • To determine bag label with confidence need • Bigger bag = better. Less chance of missing correct positive instance • Note the difference between query bags and classical bags
Example: Line Bags • Instances of a bag lie on a line.
Example: Hypercube Bags • Instances of a bag lie in a hypercube
Example: Image Translation Bags • Let be large image, be patch centered at location • Could easily extend this to rotations, scale changes, etc.
Experiments • Goal: compare behavior of synthetic classical bags and query bags to real dataset (MNIST). • Use MILBoost (Viola et al. ’05). • Expect qualitatively similar results for other MIL algorithms. • For query bags, subsample instances
Experiment: Variance • How does distribution affect error? • Repeat Line Bag experiment, increase variance of - spreads points out along the line.
Observations • PAC results not applicable to query bags – performance increase as increases. • MNIST results closely resemble synthetic query bag examples. • Need computational strategy for dealing with large bags. • Take advantage of relationships between instances.
Outline • Multiple Instance Learning (MIL) Review • Typical MIL Applications • Query Bag Model for MIL • Filtering Strategies • Conclusion
MILBoost Review • Train a strong classifier (just like AdaBoost) • Optimize log likelihood of bagswhere and • Use Gradient Boosting (Friedman ’01) • In each iteration add close to
MILBoost w/ Query Bags • Bag probability over all instances • In practice, subsample bag: • Could subsample once in the beginning, or do something more clever…
Filtering Strategies • Recently, Bradley & Schapire proposed FilterBoost, which learns from continuous source of data. • Alternates between training weak classifier and querying oracle for more data. • Apply this idea to MILBoost
Filtering Strategies • Want highest probability instances • Parameters: • = number of boosting iterations • = number of instances to evaluate • = frequency of filtering
MEM Filtering Strategies RAND • Random Sampling (RAND) • Query instances, keep best • Memory (MEM) • Query new instances, combine with old ones, keep best
MEM Filtering Strategies SRCH RAND • Search (SRCH) • Assume instances lie on low dimensional manifold • Search for nearby such that • Test nearby locations
MNIST Filtering Experiments • Turn SRCH and MEM on and off. • Sweep through: • R = sampling amount (16) • m = bag size (4) • F = sampling frequency (1) • T = boosting iterations (64)
MNIST Filtering Exp: m • Filtering converges w/ smaller memory usage
MNIST Filtering Exp: R & F • MEM is very effective • SRCH helps when MEM is OFF, not as big of a difference when MEM is ON
MNIST Filtering Exp: T • w/o MEM filtering does not converge • Positive region becomes sparse
Why MEM Works • Let be log likelihood with • Can show (for a fixed classifier H) • Using MEM, we add new instances per bag in each iteration, so • In reality H is not fixed; hard to show convergence.
Outline • Multiple Instance Learning (MIL) Review • Typical MIL Applications • Query Bag Model for MIL • Filtering Strategies • Conclusion
Summary • Current assumptions for MIL are not appropriate for typical MIL applications. • We proposed query bag model, fits real data better • For query bags, sampling more instances is better. • We proposed some simple strategies for dealing with large/infinite query bags.
Future Work • Develop more theory for the query bag model. • Experiments with other domains (audio, bioinformatics). • MCL – learning pedestrian parts automatically.
