Isn't that the same?

I suppose that technically, the 'arrows are matrices' definition rules out infinite dimensional vector spaces, but I'd guess that OP meant to include them.

An argument against would be to keep to a small category.