You appear to be saying that the values of a type cannot be a subset of another type; that is, there is no '1', only a '1' that is an integer, a '1' that is a short, and so forth, and every '1' is distinct.
That's true, and you can prove it simply by observing that they react differently to ‘the same’ operations, e.g. (2¹⁶ - 1) + 1 has a different value (that responds differently to tests like `≥ 0`) depending on which type we're talking about. There's an injection into the larger type (a type coercion) that is very well-behaved, but it's not an identity map — it changes the operations on the value.
Further, what's the practical difference between a type defined as consisting of the numbers [1 2 3], and an integer that's restricted to those values?
Nothing (ish: you have to be careful to describe what happens to all the integer operations when restricted to your type) — that's a totally valid way to define a type. But it's not the only way to define a type, because the world of types is much bigger than the world of restricted sets of integers.
I need to ensure that a patient has some birthdate that's an anticipated type
No, you just have to be explicit about the possibility of a birthdate (or other fields) being of a wide type, e.g.
Nothing (ish: you have to be careful to describe what happens to all the integer operations when restricted to your type) — that's a totally valid way to define a type.
Then at which point does a type become equivalent to a restriction that could be decoupled from the underlying type by choosing a broader type?
No, you just have to be explicit about the possibility of a birthdate (or other fields) being of a wide type
Yes, you could recreate Clojure's semantics in Rust. More accurately, I'd say you'd be looking at defining it like so:
Since we want to be able to reason about these keypairs individually. You might have a structure that has a WbcCount but not a BirthDate, or one with a BirthDate but not a WbcCount, or one with neither.
We'd then be faced with the challenge of creating a type that could contain an arbitrary number of unique structs, and to be able to pull a struct out of that set by its type. Realistically we'd probably just use a HashMap at that point and discard all static typing.
Alternatively, we could create a mega struct that contains every possible field we could want to use, whether or not we know they are related, and use this data structure to represent all structured data in the application. The "if everything is coupled, nothing is" approach.
But both of those options are difficult or unidiomatic to write.
Languages like Rust, Java, etc. encourage coupling of data because it's more convenient and space efficient to group data together in records/structs. In Clojure, there's no need to do so; we can couple only when necessary. If we don't need to know the birthdate to calculate the white blood cell count, then we can exclude that field from the input type checking. Most statically typed languages find this difficult, with TypeScript being one of the few exceptions in this regard.
Then at which point does a type become equivalent to a restriction that could be decoupled from the underlying type by choosing a broader type?
I'm not totally sure how to interpret this question. If I already have a type that includes all the values I want as a subset, I can restrict that type by limiting the values it can take and restricting or removing the operations on it to guarantee they never produce any of the forbidden values. That's the basis of refinement type systems like Liquid Haskell etc. For any type I can describe this way I can also build it ‘from the ground up’ by starting with the empty type and adding operations, though it might not be as convenient. But the converse isn't true: not every type I can build additively can be refined subtractively from another type. For a start, you need to have a broader type to begin with, so that has to already be built somehow: you can't refine an 8-bit integer into an HTTP server.
You might have a structure that has a WbcCount but not a BirthDate, or one with a BirthDate but not a WbcCount, or one with neither.
Remember that a `Box<dyn Any>` could also be a ‘no value’ type like `()` or a ‘maybe no value’ type like `Option<Date>`. But yes, there are many equivalent ways to write it depending on how likely the values are to exist; the precise formulation is just a question of ergonomics.
We'd then be faced with the challenge of creating a type that could contain an arbitrary number of unique structs, and to be able to pull a struct out of that set by its type.
By its name, but yes, that's exactly what you get with the `other_fields` field in my example above.
and discard all static typing
Well, that's not quite true: we discard (syntactically, but not conceptually!) the type information about the fields of this struct, but we can regain it later (from the conceptual information we retain) by trying to downcast the `Any`. Then you get your static type information back and the compiler can help you again.
But both of those options are difficult or unidiomatic to write.
Sorry, I hope I haven't come across too harsh to Clojure here. As I thought I was clear about above, Clojure's ergonomics for writing dynamically-typed code are vastly superior to Rust's. By coupling runtime type information to values and restricting itself to only being able to deal with values that are data and have runtime type data attached, Clojure gets to make a whole bunch of simplifying assumptions that reduce the work the programmer has to do when dealing with such values. That's the trade-off: Rust chooses to be able to express a much wider range of types by not requiring that coupling, but in exchange you have to be much more explicit when you do want to couple runtime type information to your values. Some languages like C# aim to be able to do both, i.e. start strongly typed and drop down ergonomically into dynamic typing as desired, but Rust is not one of those languages.
If we don't need to know the birthdate to calculate the white blood cell count, then we can exclude that field from the input type checking.
But that very exclusion requires that you know enough about the value to know that it can be safely ‘excluded’ without having any additional information about it. Specifically you need to know:
- how to access a subfield of the type regardless of the other fields on it — this requires that it contain RTTI
- how to discard the value — this requires that it have a destructor with a known calling convention
- how to move/copy the value into the function (and maybe even move out of the function if you return it or imperatively add it to some global state) — this requires that you know a ‘moving constructor’ for the value that can be used to safely move it to another location
In the Rust case you need to write down these things explicitly so the compiler can make sure you don't pass a value that doesn't satisfy them. In Clojure, the reason you don't need to write them down is that every value in the language is restricted to only be able to represent such values, so those constraints are implicitly on every function that you write.
I'm not totally sure how to interpret this question. If I already have a type that includes all the values I want as a subset, I can restrict that type by limiting the values it can take and restricting or removing the operations on it to guarantee they never produce any of the forbidden values.
Sorry, perhaps I'm being a little obtuse. What I'm ultimately trying to get at is that sometimes its useful to have different types for the same data under different circumstances. A subset of data might benefit from a narrower type.
For example, suppose we have some CSV file that contains a bunch of patient information, including average white blood cell count and birthdate. This CSV file might contain bad data! So perhaps we type it as:
This covers all our bases, but it's also somewhat annoying to work with. Perhaps we only want to find patients with WBC counts outside a certain range, and discard those lines where the data is invalid. In which case, we could write a narrower type, and simply not parse the CSV rows that are invalid:
So in this case we have a couple of options for types. The latter type decouples the birthdate (it's never even parsed), but adds the restriction that the WBC count needs to be numerical; while the former type is a more accurate representation of the CSV file overall, but has greater coupling.
Obviously which we use depends on the nature of our program, but what I'm trying to get at is that a type isn't set in stone, but something we choose. Ideally we choose the most restrictive and decoupled type for the circumstances, but that might result in having many hundreds of different variations of the same type, so there's a tension between the number of distinct types and how specific or narrow they are.
In the Rust case you need to write down these things explicitly so the compiler can make sure you don't pass a value that doesn't satisfy them. In Clojure, the reason you don't need to write them down is that every value in the language is restricted to only be able to represent such values, so those constraints are implicitly on every function that you write.
Granted, but Clojure's approach can result in greater decoupling with less effort, particularly when dealing with imperfect data.
In the previous examples I presented a scenario where we might want to think carefully about how exactly we store data that might contain invalid fields. But in Clojure we don't care - we can quite easily have a data structure that's partially invalid or unparsed - the entire problem of how we represent this data in memory is sidestepped.
We can still define what constitutes valid fields:
But these schema definitions are independent of the type system (i.e. decoupled), so we can apply them selectively or not at all. We can say "this function requires patient data with valid birthdates and WBC counts, but we don't care about any other field".
Could you do the same in Rust or other similar languages? Sure, it's ultimately just maps and predicates. But many languages aren't geared up to make that pleasant to use.
Perhaps we only want to find patients with WBC counts outside a certain range, and discard those lines where the data is invalid. In which case, we could write a narrower type, and simply not parse the CSV rows that are invalid:
These two types aren't typing the same data, though: in the latter case (in Rust) you've actually thrown away some of the data, and the type reflects that. In some languages like Clojure or TypeScript you can't distinguish that from the case where you haven't actually thrown away the data and are secretly carrying it around as well as whatever data is explicitly typed there, but if you want to get it back you have to know that it was once there, i.e. have additional (conceptual) type information that you chose not to write down.
Granted, but Clojure's approach can result in greater decoupling with less effort, particularly when dealing with imperfect data.
I think this isn't a great comparison: the usual vehicle for this thing in Rust is the trait, which manages to abstract over the representation statically without introducing a bunch of runtime machinery for it (though you can opt in to the machinery by using a trait object!). That's very idiomatic in Rust, although the trait syntax is a bit noisier (because it's more general).
the entire problem of how we represent this data in memory is sidestepped.
It's not sidestepped — the language just makes an opinionated choice for you, which you can't avoid.
these schema definitions are independent of the type system (i.e. decoupled), so we can apply them selectively or not at all
These are just predicates; I don't think they have much to do with this discussion? They don't help you to type your data: you (the programmer) still have to carry that type information around in your head in order to work with the values even after they are validated.
Could you do the same in Rust or other similar languages? Sure, it's ultimately just maps and predicates. But many languages aren't geared up to make that pleasant to use.
There are good reasons for that, though. Nominal typing allows you to express and enforce constraints that aren't necessarily enforced by the structure of the type, at the cost of some extra ceremony. And systems languages often want to be able to express types that cannot be treated this way: types in which there is no sensible way to ‘ignore’ a value. Clojure makes some things easier to write by forcing you to carry around a bunch of extra stuff (both runtime data and static semantics) with all your values; that's a perfectly valid choice that makes an important subset of programs nicer to write, but it also excludes values that don't fit into that category or can't reasonably be coupled to their RTTI.
As we get better at writing compilers we're increasingly seeing a move towards mechanisms like Rust's traits or C++'s concepts that allow the programmer to actually abstract over the representation of a type (as opposed to forcing a uniform representation as is common in dynamically typed languages). If you do that well you can get the best of both worlds semantically, but these kinds of systems languages will always be a bit heavier syntactically because the syntax needs to support a wider range of types.
These two types aren't typing the same data, though: in the latter case (in Rust) you've actually thrown away some of the data, and the type reflects that.
In Clojure the data is thrown away too, just at a slightly later point. The data is parsed then destructured then processed, and it is at the destructuring stage that irrelevant data is discarded and marked for GC.
Just as the Rust function avoids coupling by using a more narrow type, the Clojure function avoids coupling by using a more narrow binding.
I think this isn't a great comparison: the usual vehicle for this thing in Rust is the trait, which manages to abstract over the representation statically without introducing a bunch of runtime machinery for it
But you have to create and implement those traits. I'm not saying this is impossible in Rust; just a lot more onerous because you don't get all the machinery Clojure has that makes it trivial.
These are just predicates; I don't think they have much to do with this discussion?
This might be why we're talking past each other. When I say "type", I mean a set of possible values. I don't think this is an uncommon definition; the first sentence of the "data type" Wikipedia page pretty much defines it the same way.
Given this definition, you perhaps see why we can narrow a runtime type by composing it with a predicate.
My understanding is that you view types differently, as more of an interpretation of some sequence of bits, rather than a set of data values. Is that correct?
As we get better at writing compilers we're increasingly seeing a move towards mechanisms like Rust's traits or C++'s concepts that allow the programmer to actually abstract over the representation of a type (as opposed to forcing a uniform representation as is common in dynamically typed languages).
I agree that's there's no reason in principle that you couldn't statically type it with a sufficiently advanced compiler.
Comments
That's true, and you can prove it simply by observing that they react differently to ‘the same’ operations, e.g. (2¹⁶ - 1) + 1 has a different value (that responds differently to tests like `≥ 0`) depending on which type we're talking about. There's an injection into the larger type (a type coercion) that is very well-behaved, but it's not an identity map — it changes the operations on the value.
Nothing (ish: you have to be careful to describe what happens to all the integer operations when restricted to your type) — that's a totally valid way to define a type. But it's not the only way to define a type, because the world of types is much bigger than the world of restricted sets of integers.
No, you just have to be explicit about the possibility of a birthdate (or other fields) being of a wide type, e.g.
Clojure just attaches that by default to every value.Then at which point does a type become equivalent to a restriction that could be decoupled from the underlying type by choosing a broader type?
Yes, you could recreate Clojure's semantics in Rust. More accurately, I'd say you'd be looking at defining it like so:
Since we want to be able to reason about these keypairs individually. You might have a structure that has a WbcCount but not a BirthDate, or one with a BirthDate but not a WbcCount, or one with neither.We'd then be faced with the challenge of creating a type that could contain an arbitrary number of unique structs, and to be able to pull a struct out of that set by its type. Realistically we'd probably just use a HashMap at that point and discard all static typing.
Alternatively, we could create a mega struct that contains every possible field we could want to use, whether or not we know they are related, and use this data structure to represent all structured data in the application. The "if everything is coupled, nothing is" approach.
But both of those options are difficult or unidiomatic to write.
Languages like Rust, Java, etc. encourage coupling of data because it's more convenient and space efficient to group data together in records/structs. In Clojure, there's no need to do so; we can couple only when necessary. If we don't need to know the birthdate to calculate the white blood cell count, then we can exclude that field from the input type checking. Most statically typed languages find this difficult, with TypeScript being one of the few exceptions in this regard.
I'm not totally sure how to interpret this question. If I already have a type that includes all the values I want as a subset, I can restrict that type by limiting the values it can take and restricting or removing the operations on it to guarantee they never produce any of the forbidden values. That's the basis of refinement type systems like Liquid Haskell etc. For any type I can describe this way I can also build it ‘from the ground up’ by starting with the empty type and adding operations, though it might not be as convenient. But the converse isn't true: not every type I can build additively can be refined subtractively from another type. For a start, you need to have a broader type to begin with, so that has to already be built somehow: you can't refine an 8-bit integer into an HTTP server.
Remember that a `Box<dyn Any>` could also be a ‘no value’ type like `()` or a ‘maybe no value’ type like `Option<Date>`. But yes, there are many equivalent ways to write it depending on how likely the values are to exist; the precise formulation is just a question of ergonomics.
By its name, but yes, that's exactly what you get with the `other_fields` field in my example above.
Well, that's not quite true: we discard (syntactically, but not conceptually!) the type information about the fields of this struct, but we can regain it later (from the conceptual information we retain) by trying to downcast the `Any`. Then you get your static type information back and the compiler can help you again.
Sorry, I hope I haven't come across too harsh to Clojure here. As I thought I was clear about above, Clojure's ergonomics for writing dynamically-typed code are vastly superior to Rust's. By coupling runtime type information to values and restricting itself to only being able to deal with values that are data and have runtime type data attached, Clojure gets to make a whole bunch of simplifying assumptions that reduce the work the programmer has to do when dealing with such values. That's the trade-off: Rust chooses to be able to express a much wider range of types by not requiring that coupling, but in exchange you have to be much more explicit when you do want to couple runtime type information to your values. Some languages like C# aim to be able to do both, i.e. start strongly typed and drop down ergonomically into dynamic typing as desired, but Rust is not one of those languages.
But that very exclusion requires that you know enough about the value to know that it can be safely ‘excluded’ without having any additional information about it. Specifically you need to know:
- how to access a subfield of the type regardless of the other fields on it — this requires that it contain RTTI
- how to discard the value — this requires that it have a destructor with a known calling convention
- how to move/copy the value into the function (and maybe even move out of the function if you return it or imperatively add it to some global state) — this requires that you know a ‘moving constructor’ for the value that can be used to safely move it to another location
In the Rust case you need to write down these things explicitly so the compiler can make sure you don't pass a value that doesn't satisfy them. In Clojure, the reason you don't need to write them down is that every value in the language is restricted to only be able to represent such values, so those constraints are implicitly on every function that you write.
Sorry, perhaps I'm being a little obtuse. What I'm ultimately trying to get at is that sometimes its useful to have different types for the same data under different circumstances. A subset of data might benefit from a narrower type.
For example, suppose we have some CSV file that contains a bunch of patient information, including average white blood cell count and birthdate. This CSV file might contain bad data! So perhaps we type it as:
This covers all our bases, but it's also somewhat annoying to work with. Perhaps we only want to find patients with WBC counts outside a certain range, and discard those lines where the data is invalid. In which case, we could write a narrower type, and simply not parse the CSV rows that are invalid: So in this case we have a couple of options for types. The latter type decouples the birthdate (it's never even parsed), but adds the restriction that the WBC count needs to be numerical; while the former type is a more accurate representation of the CSV file overall, but has greater coupling.Obviously which we use depends on the nature of our program, but what I'm trying to get at is that a type isn't set in stone, but something we choose. Ideally we choose the most restrictive and decoupled type for the circumstances, but that might result in having many hundreds of different variations of the same type, so there's a tension between the number of distinct types and how specific or narrow they are.
Granted, but Clojure's approach can result in greater decoupling with less effort, particularly when dealing with imperfect data.
In the previous examples I presented a scenario where we might want to think carefully about how exactly we store data that might contain invalid fields. But in Clojure we don't care - we can quite easily have a data structure that's partially invalid or unparsed - the entire problem of how we represent this data in memory is sidestepped.
We can still define what constitutes valid fields:
But these schema definitions are independent of the type system (i.e. decoupled), so we can apply them selectively or not at all. We can say "this function requires patient data with valid birthdates and WBC counts, but we don't care about any other field".Could you do the same in Rust or other similar languages? Sure, it's ultimately just maps and predicates. But many languages aren't geared up to make that pleasant to use.
These two types aren't typing the same data, though: in the latter case (in Rust) you've actually thrown away some of the data, and the type reflects that. In some languages like Clojure or TypeScript you can't distinguish that from the case where you haven't actually thrown away the data and are secretly carrying it around as well as whatever data is explicitly typed there, but if you want to get it back you have to know that it was once there, i.e. have additional (conceptual) type information that you chose not to write down.
I think this isn't a great comparison: the usual vehicle for this thing in Rust is the trait, which manages to abstract over the representation statically without introducing a bunch of runtime machinery for it (though you can opt in to the machinery by using a trait object!). That's very idiomatic in Rust, although the trait syntax is a bit noisier (because it's more general).
It's not sidestepped — the language just makes an opinionated choice for you, which you can't avoid.
These are just predicates; I don't think they have much to do with this discussion? They don't help you to type your data: you (the programmer) still have to carry that type information around in your head in order to work with the values even after they are validated.
There are good reasons for that, though. Nominal typing allows you to express and enforce constraints that aren't necessarily enforced by the structure of the type, at the cost of some extra ceremony. And systems languages often want to be able to express types that cannot be treated this way: types in which there is no sensible way to ‘ignore’ a value. Clojure makes some things easier to write by forcing you to carry around a bunch of extra stuff (both runtime data and static semantics) with all your values; that's a perfectly valid choice that makes an important subset of programs nicer to write, but it also excludes values that don't fit into that category or can't reasonably be coupled to their RTTI.
As we get better at writing compilers we're increasingly seeing a move towards mechanisms like Rust's traits or C++'s concepts that allow the programmer to actually abstract over the representation of a type (as opposed to forcing a uniform representation as is common in dynamically typed languages). If you do that well you can get the best of both worlds semantically, but these kinds of systems languages will always be a bit heavier syntactically because the syntax needs to support a wider range of types.
In Clojure the data is thrown away too, just at a slightly later point. The data is parsed then destructured then processed, and it is at the destructuring stage that irrelevant data is discarded and marked for GC.
Just as the Rust function avoids coupling by using a more narrow type, the Clojure function avoids coupling by using a more narrow binding.
But you have to create and implement those traits. I'm not saying this is impossible in Rust; just a lot more onerous because you don't get all the machinery Clojure has that makes it trivial.
This might be why we're talking past each other. When I say "type", I mean a set of possible values. I don't think this is an uncommon definition; the first sentence of the "data type" Wikipedia page pretty much defines it the same way.
Given this definition, you perhaps see why we can narrow a runtime type by composing it with a predicate.
My understanding is that you view types differently, as more of an interpretation of some sequence of bits, rather than a set of data values. Is that correct?
I agree that's there's no reason in principle that you couldn't statically type it with a sufficiently advanced compiler.