Your analogy is particularly apt if you consider that two of the means of achieving AI are emergent and thus not engineering and not at all friendly to reverse engineering. If we can simulate the brain's processes in bulk, we still don't necessarily understand how it does what it does. (Like making a photocopy of ciphertext.) Likewise, if we create some sort of architecture that lets us evolve AIs, there is no guarantee that we will be able to understand how the results work. There's no guarantee that the resulting AIs will be able to either.