Try fewer neurons and solve it to global optimality with gurobi. This way you will know if the optimization step was your bottleneck.