But languages like C/C++/Rust/Go work differently. As these languages are commonly used, the string -> UTF-8 step is actually a no-op, because the assumption is that strings are already UTF-8 in memory.
No.
Rust's string types are explicitly UTF-8 text. If what you've got isn't UTF-8 text, it's not a Rust string type. Here's the signature of the conversion you say is "actually a no-op".
That says if you've got a growable array of bytes and you claim it's UTF-8 text, you can have a String back if you're correct about that. If you were wrong you get a FromUtf8Error, which is a wrapper around that growable array and some diagnostic information.
Edited to add:
The reason I was looking at this thread is because of course Rust for its own purposes does exactly what layer8 describes - it emits a single 0xFF byte to separate strings because Rust's strings are guaranteed UTF-8.
Comments
No.
Rust's string types are explicitly UTF-8 text. If what you've got isn't UTF-8 text, it's not a Rust string type. Here's the signature of the conversion you say is "actually a no-op".
pub fn from_utf8(vec: Vec<u8>) -> Result<String, FromUtf8Error>
That says if you've got a growable array of bytes and you claim it's UTF-8 text, you can have a String back if you're correct about that. If you were wrong you get a FromUtf8Error, which is a wrapper around that growable array and some diagnostic information.
Edited to add:
The reason I was looking at this thread is because of course Rust for its own purposes does exactly what layer8 describes - it emits a single 0xFF byte to separate strings because Rust's strings are guaranteed UTF-8.