> If gradient descent is limited in its ability to learn representations, better
methods for this purpose are a key research direction. Current nonlinear
alternatives include predicate invention (Muggleton and Buntine, 1988) and latent
variable discovery in graphical models (Elidan et al., 2000).
Hah! Fancy seeing that here! Predicate invention is a main line of my PhD
research.
Briefly, predicate invention is the ability of Inductive Logic Programming (ILP)
systems to learn their own inductive bias. It is in a sense similar to feature
learning or learning-to-learn. ILP systems learn logic programs from examples
usually by searching the space of programs defined by a set of sub-programs,
called the background knowledge (BK), and a language bias that determines the
structure of learned programs. Predicate invention then means learning new BK
and language bias to change the program search space while searching it.
The reference in Domingo's article is the first description of the concept which
was for a long time more theoretical than practical: ILP approaches could only
perform a limited form of predicate invention, e.g. could only invent BK programs
of fixed structure or could not invent recursive programs etc. Things changed in
2013 with a new approach, Meta-Interpretive Learning (MIL). MIL systems are for
the first time capable of unconstrained predicate invention, including the
invention of mutually recursive programs. Full discolosure: my PhD research is
on MIL.
Here are some more recent references on predicate invention in MIL:
Comments
Hah! Fancy seeing that here! Predicate invention is a main line of my PhD research.
Briefly, predicate invention is the ability of Inductive Logic Programming (ILP) systems to learn their own inductive bias. It is in a sense similar to feature learning or learning-to-learn. ILP systems learn logic programs from examples usually by searching the space of programs defined by a set of sub-programs, called the background knowledge (BK), and a language bias that determines the structure of learned programs. Predicate invention then means learning new BK and language bias to change the program search space while searching it.
The reference in Domingo's article is the first description of the concept which was for a long time more theoretical than practical: ILP approaches could only perform a limited form of predicate invention, e.g. could only invent BK programs of fixed structure or could not invent recursive programs etc. Things changed in 2013 with a new approach, Meta-Interpretive Learning (MIL). MIL systems are for the first time capable of unconstrained predicate invention, including the invention of mutually recursive programs. Full discolosure: my PhD research is on MIL.
Here are some more recent references on predicate invention in MIL:
Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited (IJCAI 2013):
https://www.ijcai.org/Proceedings/13/Papers/231.pdf
Bias reformulation for one-shot function induction (ECAI 2014):
http://www.doc.ic.ac.uk/~shm/Papers/metabias.pdf
Logical minimisation of meta-rules within meta-interpretive learning (ICLP 2015):
https://www.doc.ic.ac.uk/~shm/Papers/minmeta.pdf
Like I say this is my field of study and as you can probably tell I'm very excited about it so I'm happy to answer questions- email in my profile.