Relativity just demands that changes in time are a measurement you could make. Then it talks about the geometry of those measurements. What are you arguing against?