Random Regression: Example Target Query: P( gender ( sam ) = F) ?

1. Relational Random Regression for Bayes Nets Oliver Schulte, Hassan Khosravi, TianxiangGaoand Yuke Zhu School of Computing ScienceSimon Fraser University, Vancouver, Canada Project Website: http://www.cs.sfu.ca/~oschulte/jbn/ • Bayes Net Inference: The Cyclicity Problem • In the presence of recursive dependencies (autocorrelations), a ground first-order Bayes net template may contain cycles (Schulte et al. 2012, Domingos and Richardson 2007). • Overview • How to define Bayes net relational inference with ground cycles? • Define Markov blanket probabilities: the probability of a target node value given its Markov blanket. • Random regression: compute the expected probability over a random instantiation of the Markov blanket. • Closed-form result: equivalent to a log-linear regression model that uses Markov blanket feature frequencies rather than counts. • Random regression works well with Bayes net parameters • very fast parameter estimation. • Random Regression: Example • Target Query: P(gender(sam) = F)? • Sam is friends with Bob and Anna. Unnormalized Probability: • Evaluation • Use Learn-and-Join Algorithm for Bayes net structure learning (Khosravi et al. 2010, Schulte and Khosravi2012). • MBN Convert Bayes net to Markov net,use Alchemy to learn weights in log-linear model with Markov Blanket feature counts. • CP+CountUse log-conditional probsas weights in log-linear model with Markov Blanket feature counts. • CP+Frequency. Use log-conditional probs as weights in log-linear model with Markov Blanket feature frequencies (= random regression). • Learning • Times • ConditionalLog-likelihood • Quick Summary Plot • Average performance over databases • Smaller Learning Time is better. • Bigger CLL is better. • Conclusion • Random regression: principled way to define relational Bayes net inferences even with ground cycles. • Closed form evaluation: log-linear model with feature frequencies. • Bayes net conditional probabilities are fast to compute, interpretable and local. • Using feature frequencies rather than counts addresses the balancing problem:in the count model, features with more groundings carry exponentially more weights. • Regression Graph Methods Compared gender(Y) gender(Y) Friend(sam,Y) Friend(X,Y) coffee_dr(sam) coffee_dr(X) gender(X) gender(sam) P(g(X) = F |g(Y) =F, F(X,Y) = T)= .6 P(g(X) = M|g(Y) = M, F(X,Y) = T) = .6 ... • Closed Form • Proposition The random regression value can be obtained by multiplying the probability associated with each Markov blanket state, raised to the frequency of the state. Example: • P(g(sam) = F|mb) =α P(cd(sam) = T|gd(sam) = F) x[P(g(sam) = F|g(anna)=F, Fr(sam,anna) = T) xP(g(sam) = F|g(bob) = M, Fr(sam,bob) = T)] 1/2 =70% x [60% x 40%] 1/2 = 0.34 = e-1.07 P(cd(X) = T|g(X) = F) = .7 P(cd(X) = T|g(X) = M) = .3 • Relational regression in graphical models • Bayes net dependency net, use geometric meanas combining rule = log-linear model with frequencies = random regression. • Bayes net Markov net, use standard Markov network regression = log-linear model with counts. Example: • P(g(sam) = F|mb) = α 70% x 60% x 40% = 0.168. • References • H. Khosravi, O. Schulte, T. Man, X. Xu, B. Bina, Structure learning for Markov logic networks with many descriptive attributes, AAAI, 2010, pp. 487–493. • O. Schulte and H. Khosravi. Learning graphical models for relational data via lattice search. Machine Learning, 88:3, 331-368, 2012. • Schulte, O.; Khosravi, H. & Man, T. Learning Directed Relational Models With Recursive Dependencies Machine Learning, 2012, Forthcoming. • Domingos, P., Richardson, M.: Markov logic: A unifying framework for statistical relational learning. in Statistical Relational Learning, 2007. random regression Log-linear Model with Frequencies Bayes Net geo.mean Dependency Net product Log-linear Model with Counts Markov Net

2. Schulte, O.; Khosravi, H. & Man, T. Learning Directed Relational Models With Recursive Dependencies Machine Learning, 2012, Forthcoming. • Regression Graph: gender(Y) Friend(sam,Y) coffee_dr(sam) gender(sam) Domingos, P., Richardson, M.: Markov logic: A unifying framework for statistical relational learning. Regression Graph: P(g(X) = F |g(Y) =F, F(X,Y) = T)= .6 P(g(X) = M|g(Y) = M, F(X,Y) = T) = .6 ... P(cd(X) = T|g(X) = F) = .7 P(cd(X) = T|g(X) = M) = .3 People