I generally like the approach, but you can implement sorting of fixed number of items in a Boolean circuit(combinatorial logic, a bunch of ANDs and ORs stringed together). There is no need for recurrence or memory at all to sort a fixed number or items. There are a handful of abstractions possible in Boolean logic than n-gram statistics and there are a lot of Boolean circuits that can be very useful. But I’d like to see neural networks to be able to figure out algorithm that require at least a finite state machine(FSM), but preferably something harder. The challenge is to find a size difference for a problem between FSM and combinatorial logic.
Comments
I generally like the approach, but you can implement sorting of fixed number of items in a Boolean circuit(combinatorial logic, a bunch of ANDs and ORs stringed together). There is no need for recurrence or memory at all to sort a fixed number or items. There are a handful of abstractions possible in Boolean logic than n-gram statistics and there are a lot of Boolean circuits that can be very useful. But I’d like to see neural networks to be able to figure out algorithm that require at least a finite state machine(FSM), but preferably something harder. The challenge is to find a size difference for a problem between FSM and combinatorial logic.