One of the biggest differences is that if you do:

  foreach (var byte in byteArray)

The compiler transforms that into a normal for-loop that accesses the array directly. So that's already an advantage for the array over your enumerator.

The other difference that will have a major performance impact is the way the IEnumerator pattern works, even with generics. For:

  foreach (var b in f())

The generated code looks roughly like this:

  using (var enumerator = f())
  while (enumerator.MoveNext()) {
    var byte = enumerator.get_Current();
  }

As a result, you've gone from 0 method calls per iteration (direct array access) to 2 virtual method calls per iteration (MoveNext and get_Current). An incredibly smart JIT might be able to figure out that the virtual methods are always the same and turn them into static invocations, or even inline them, but I don't think the CLR can do this for you reliably.