Heuristic learning

1. Heuristic learning Intelligent Information Analysis based on V-Graph knowledge representation

2. Speech Plot • Problem • Redundancy / Insufficiency / Corruption of input data. • Representation Learning • Recode • Intrinsic Confliction • Heuristic Learning • Roaming, interactivity, reuse of learning achievement.

3. Machine learning • Ideas: • Supervised, Unsupervised learning • Clustering • Dimension reduction • Structure learning • Framework • Artificial Neuron Network Series • Statistical Learning • Feature learning

4. Deprecated methods • Dead/Stall Methods: • Expert System • Genetic Programming • Expert System: It’s impractical to establish a perfect knowledge system.(Academic Research stalled doesn’t mean no advantage for industrial use) • Software engineering problem • Un-model-able/Un-logic of Real world. • Genetic Programming: Evolution functions and target evaluation aren’t compatible. • Huge under-fitting/over-fitting problem

5. Under/Over-Fit • Under-fit: Model is too simple. • Over-fit: • Assume input data is noiseless, there won’t be over-fitting problem. • Over-fit confused signal with noise. • Noise? Signal?– Noise could lead to important discovery, e.g. Discovery of Argon • My Theory: Fitting is verification/tool/part of chain/sample/… is not solution. Interactivity could be the solution.

6. Software engineering problem • Industrialize usage and academic researches are very different. • For machine learning: • Data is precious • And worthless • The simpler/straightforward/constant/… the problem is, the better to use machine learning. • Otherwise we have all witnessed.

7. Summary of commonly discussed machine learning problems • http://www.denizyuret.com/2014/02/machine-learning-in-5-pictures.html • Illustrated the common issues and description. • Major problem: • Under/Over-fitting • Software-Engineering problem

8. Representation learning • Aims • Supervised <-> Semi-supervised: a balance between “aesthetics” and reality • Means • Recode of input data • Lessen Variable combination • Detect and remove irrelevant features • De-tangle, Re-tangle. (Very much alike Annealing Algorithms) • Frameworks: • RBMs

9. Encoding • Deep Architecture • Unsupervised pre-training helps. • Aesthetics • I think it’s essentially “roaming”, in a narrow form • Supervised training has few spots. • Pre-training <-> Training • Cost function compatibility • Data required is huge.

10. Major problem of current ml methods • Cost functions works well for SIMPLE CASES, for complex cases, dimension of evaluation explodes. i.e. Cost function cannot express choices. • Training data could be very redundant, while insufficient. The achievement by using statistical analysis highly depends on how well the application data meets the analysis assumption. • Different form of issue lies in Expert System. • “Aesthetics” is vital, and un-definable.

11. Paragraph Summary • Problems to be solved: • Formulate/Automate/Reproduce Aesthetics • Engineering/Noise/Quantity Problem

12. Heuristic Learning • Main Ideas • Roaming • In strong knowledge relations. • Means • Use V-Graph to represent data. • Use Heurons to express “aesthetics” activity.

13. roaming • Roaming refers to extending target and learning data during training. • Critical to strong-typed heuristic learning, where original target and learning data could not reach a good result. • Implementation in V-Graph: Heurons • Logic roaming(The meta-movement: generate a meta-link)

14. V-Graph concept • View • (A->B)@C • Un-logic Quantize Relationships. • Flattened Complex Structure • Database Usage • Search Engine, Linguistic corpus…

15. Heuron • Neutron like information propagation for V-graph • Ruled Meta-link Generation

16. Software framework summary • Extended Graph • Capable to express more. • Extended Neuron • Based on Strong Relationship, Qualitative descriptor • Allow quantitative description.

17. heuristic • Main idea: • Take a reasonable guess, rather than simple parameter fitting.

18. Heuristic learning framework • Extended Strong typed: Type recognition and transformation performed in V-Graph. • A combination and improvement to Cost function and Evaluation function. • Multiple Heuristic approach allows roaming. • Initial roaming generate a basis set of meta-link. • Heuristic learning still apply statistics to input data, however, in more representative form.

19. Heurons for instance • Spot • Recall • Short-cut • Preference-Intention • Curiosity • Contrast • Trial

20. Example: Reverse Pendulum • A solved problem. • Heurons involved: • Short-cut(memory), Stimulus, Trial, (I think it’s essential) • Expectation, Preference, Mimicry, Demand, Interest…

21. Incremental and Interactivity • The framework has room for the feature, by implement specific heurons.

22. Summary • V-graph and heurons are methodology to realize Roaming. • Roaming shouldn’t necessarily require V-Graph or heurons. However, the choice of V-Graph and Heurons is because…

23. Roadmap • V-graph Query Language, V-graph Database. • Architecture Expressway: A methodology and toolkits aims to reduce software engineering difficulties. • Use V-Graph to train an eye tracking program.

24. Summary • Roaming • Engineering