Sure, I agree that if you're making a CPUID inline assembly template that isn't setting the register value, and it accepts any values for EAX that would also cause it to read ECX then the behavior is going to be undefined and likely unexpected, but that doesn't seem to be a reason for this not to compile.
For one, requiring an input parameter when it isn't mandatory would mean that you're spending cycles setting a register with a value that you don't need and is just going to be overwritten anyway. A good number of the CPUID calls never read from ECX, and if you're going to call any of them, then the value in ECX is irrelevant. And sure, it's only an extra instruction or two, but presumably if you're dropping down to inline assembly, you kind of care about every wasted instruction.
// Returns the maximum input value for basic CPUID information
cpu_extended_feature_flags :: asm() -> (a, b, c, d: u32) [
a = %eax,
b = %ebx,
c = %ecx,
d = %edx,
] {
mov %eax 0x0
cpuid
}
I admit I never touch inline assembly or really assembly at all save for the occasional microcontroller project, so maybe I'm missing something that's obvious to people more familiar with this. But to me the article seems to be sayin that the Odin templates will type check and validate that IF you have inputs from Odin types that are being put into registers or used as operands to your assembly, or are mapping registers and outputs from your assembly back to Odin types, that those mappings will be type compatible, and when you get those mappings wrong, you'll get a more useful error. But I didn't read it as saying that it will prevent you from writing assembly that does something completely unrelated to those inputs or outputs.
For CPUID specifically this isn't going to be on a fast path. As its name might suggest it's a way to have the runtime CPU tell us what features are available so that we can adopt different implementations of perf-sensitive routines for that CPU.
So e.g. maybe the CAD software executes a dozen CPUID instructions during startup, and then based on those it uses AVX512 vectorized operations later on some hardware but uses SSE instead on other hardware.
I haven't read Intel's manual for a modern x86-64 CPU, and I certainly haven't read the community notes about this stuff which would tell you if, despite the documentation you need to behave differently, but my assumption would be that everybody clears ECX if they don't want a non-zero subleaf and that this is either known to be necessary or an obvious way to avoid nasty surprises in code that is never perf critical.
I think we might be talking past each other here. Whether the assembly op is `cpuid` or something else, isn’t really my driving concern. The comment that started this thread (which I’m pretty sure wasn’t yours, so I don’t necessarily expect you to have an answer) was that they didn’t think this example should have compiled because `cpuid` may read from a second register as part of its operations, depending on the value passed to the first register.
But that seems to be saying that the inline assembly template should have the same shape as the underlying assembly calls that are made. If the underlying call might read 3 registers, then the complaint appears to be that the templating system MUST require a template that calls that assembly to also have 3 inputs, even if it won’t use all 3.
My thinking on this is the templating is a syntax for function declarations where the function is assembly code and not more higher level language code. We don’t require that functions have the same number of inputs and outputs that the code called within the function has in order for it to compile, I don’t see why it would be necessary for the assembly templates to work any differently. I get that `cpuid` specifically isn’t likely to run in the hot path of any code, but the underlying principle is the same. If you’re dropping to assembly, you likely have some performance tuning you’re trying to do. A templating system that requires your template inputs to 1:1 map to all the possible inputs of all the assembly calls you use in the template, regardless of wether you need or use them is adding extra instructions and waste for (to me) no obvious benefits.
A templating system that requires your template inputs to 1:1 map to all the possible inputs of all the assembly calls you use in the template, regardless of wether you need or use them is adding extra instructions and waste for (to me) no obvious benefits.
But we're not suggesting this ridiculous and arbitrary restriction. What we're suggesting, and in fact I think what Bill actually intended in Odin, is something much more useful.
A templating system that requires all possible inputs of all the assembly calls you use in the template are well defined.
What you seem to want is exactly the thing Bill doesn't like about existing functionality. You can cheerfully ADD two registers together and then use the result but without ever determining what's in those registers. What is the result? -shrug- might be anything.
What Odin seems to intend (but this CPUID example seems to suggest is buggy) is that it will check you've actually written code which means something. You do not need to make those registers inputs of your template, you just need to make sure they're well defined. For example you could set them directly in the template, as with your leaf 7 sub-leaf 1 example.
That (modulo bugs) is a big improvement for this particular corner of the language.
I'm starting to get a better handle on what you're getting at, but I still think I disagree that the code shouldn't compile. To really settle this I guess we would need Bill to weigh in on his intent, because I read this article about being all about types and type checking. He says:
However every instruction has a set of valid forms. Each form dictates the
kind of each operand (register, memory, immediate, label), the class of
each register (general-purpose, vector, mask), the width of each operand,
the range each immediate may take, and what the instruction clobbers
(flags, memory, particular registers).
...
That is not the absence of a type system: that is a type system; a rather
rich, dependent, per-instruction one.
...
The params are your inputs, as plain names with Odin types. The results are
your outputs, again plain names with types, sharing the same signature
syntax as an Odin procedure.
...
The params and results are optional, like with a normal procedure type, and
the bindings are completely optional if they are not necessary.
That last part in particular is important here, bindings are optional if they're not necessary. Well if the "types" for any given ASM instruction come from their valid definitions, then we'd have to look at the intel instruction definitions[1].
Some instructions like ADD[2] (Vol 2A 3-14) are defined to have multiple forms and those forms each take 2 operands. So the type of the instruction would be `ADD T-OP1, T-OP2` for each of the possible combinations of operand types for the ADD forms. And the templating system and compiler would validate that if you pass a given parameter of a given type to one of those operands, that your types match up.
Some like ANDPD (Vol 2A 3-63) have two forms, one that takes 2 operands and one that takes 3. The 2 operand one says that the first operand is a read/write register, where as the 3 operand forms say the first operand is write only. Presumably if you took an input and bound it to the first operand in a 3 operand form, the compiler would at least emit a warning about this if not an error.
But now if we look at the definition of CPUID (Vol 2A 3-203) the only valid form it has is a form with no operands. So if we were defining a type for the type system to compare against, the only possible correct answer (to me) is `cpuid` with no operands. That is the type system should happily allow inline assembly that calls `cpuid` but not inline assembly that calls `cpuid %eax %ecx` because that's not a valid form of the instruction.
Further the definition explicitly says that for some values of EAX, the value in ECX would be ignored entirely, and for invalid values in either field, the output values are all "Reserved". That tells me that as far as a type checker is concerned, any invocation of `cpuid` as long as there are no operands is a valid invocation. A type checker doesn't check the values of the fields being used, and IMO any argument that since it could read ECX, then the type system should enforce you set a value it to it can be equally countered by an argument that since cpuid will explicitly ignore ECX with certain values, it should never require you to set ECX because that might cause you to set it with a value and get an unexpected result because the value you provided to EAX was one of the values that causes it to be ignored. In either case we're asking the type checker to help us prevent a logic bug, not a type bug.
You might say that the "always require both fields to be set" is at least an easy check that could be applied universally, then the question would become how would it intersect with multiple invocations of CPUID? If you invoke:
mov eax, 0x0
mov ecx, 0x0
cpuid
cpuid
that is valid and the outcome of that should be (assuming I'm reading the documentation correctly) that after the first invocation, EAX would contain the maximum valid value for EAX when invoking cpuid, and after the second invocation it should contain the results of invoking cpuid with whatever that value was (and obviously whatever was in ECX after the first invocation). If the purpose of the check is to prevent invocations where ECX might contain an arbitrary value that makes no sense, we'd have to mandate that there's some additional steps in between the two cpuid invocations to reset EAX and ECX. Otherwise all we've done is make it possibly even more confusing when ECX changes after the first invocation, but the code strongly implies it should be 0x0.
Given the stated goals in the article seem to include not requiring the explicit statement of implicit behavior, requiring setting ECX for instruction that will ignore it would seem counter to the goals.
As a side note, I'm curious why (based on your comment about ADD), it would be desirable for a type checker to want to prevent you from doing something like this:
It's perfectly valid to not always care about the starting value in a register even if you're going to use it. Plenty of pseudo-rng type code has read arbitrary registers or addresses as a source for some of their calculation without ever caring what the starting value was. I feel like in some way this gets to the heart of what we're disagreeing about. I read the article as saying "within the bounds of the shape of assembly code as defined by the ISA, odin templates can help enforce types for those shapes and help make wiring normal types to registers for input and output easy. I feel like what you're saying that in addition to that, it's also supposed to help stop you from doing things that are likely to give you non-sensical results.
It's perfectly valid to not always care about the starting value in a register even if you're going to use it. Plenty of pseudo-rng type code has read arbitrary registers or addresses as a source for some of their calculation without ever caring what the starting value was.
No. Never do this. If you don't care and just want random numbers use for example RDRAND, if you do care, design a proper PRNG with your preferred characteristics and seed management. YOLO programming is a bad idea always but it's especially bad when applied to assembly.
To underscore what this software is for let me quote you a diagnostic it emits:
{instruction} implicitly reads {register}, but nothing in this template produces a value for it; pin an input parameter to {register}, or write {register} before this instruction
I agree you shouldn't do it, that doesn't make it invalid to do.
To underscore what this software is for let me quote you a diagnostic it emits
Well, then I stand corrected on the intent. That pretty clearly spells out that the template checker will validate that (at least for non-branching assembly) all implicitly read registers are given an explicit value and error if not. So that should mean the example in the article actually would not compile, since the rexcode definition lists both RAX and RCX as `implicit_rd`.[1] I appreciate you finding that bit of validation code and persisting in helping me learn something new, this isn't an area of coding I spend a lot of time in so it was instructive.
[1]: Does it compile? I don't have any odin dev stuff set up, but I might just try it in the next little bit just to see.
I don't run nightly Odin. I do happen to a have a non-nightly Odin installed because I was wondering if some of the trash in their kitchen sink library is in fact trash (it is) or whether it's clever in a way I didn't understand ‡
However my reading of the implementation is that:
1. Bill over-sells the value of rexcode. rexcode is Odin code, so the actual "value" derived in Odin's own compiler is just that it scrapes the data out of rexcode. That's not nothing but it's not much.
2. Bill's new template feature doesn't remember that we can pin an output, so it notices that ECX is pinned and concluded it is safe for CPUID to read it. That's a very small bug, and it's even possible I've misunderstood it, but that's my reading.
‡ Odin provides a lot of sorting algorithms. I wanted to measure how fast they are, which I did by comparing against a Rust install on the same toy machine, all of them are much slower than Rust's built-in sorts, but interestingly the provided "slice sort" which is most analogous to Rust's [T]::sort and [T]::sort_unstable is a "Smooth sort" which is relatively obscure and actually is only as slow for pre-sorted input as my Rust sorts are for unsorted input whereas most of what's provided is way slower even for pre-sorted input.
1. I don't think I am overselling rexcode, rather there isn't anything like it in the first place as a single library with that many ISAs and IRs. That's what I love about it, since I can now trivially make (non-optimizing) compilers without needing to use LLVM or use another external tool (including outputting C or assembly). Also all of those tables become binary blobs, so you could just use them in your favourite language any way.
2. That was a bug and it has now been fixed, along with many other analysis bugs.
3. The sorting algorithms are getting an overhauls soon to be a lot faster.
Comments
Sure, I agree that if you're making a CPUID inline assembly template that isn't setting the register value, and it accepts any values for EAX that would also cause it to read ECX then the behavior is going to be undefined and likely unexpected, but that doesn't seem to be a reason for this not to compile.
For one, requiring an input parameter when it isn't mandatory would mean that you're spending cycles setting a register with a value that you don't need and is just going to be overwritten anyway. A good number of the CPUID calls never read from ECX, and if you're going to call any of them, then the value in ECX is irrelevant. And sure, it's only an extra instruction or two, but presumably if you're dropping down to inline assembly, you kind of care about every wasted instruction.
Again, this seems to me like it should be a perfectly valid assembly template (based on https://www.felixcloutier.com/x86/cpuid):
I admit I never touch inline assembly or really assembly at all save for the occasional microcontroller project, so maybe I'm missing something that's obvious to people more familiar with this. But to me the article seems to be sayin that the Odin templates will type check and validate that IF you have inputs from Odin types that are being put into registers or used as operands to your assembly, or are mapping registers and outputs from your assembly back to Odin types, that those mappings will be type compatible, and when you get those mappings wrong, you'll get a more useful error. But I didn't read it as saying that it will prevent you from writing assembly that does something completely unrelated to those inputs or outputs.For CPUID specifically this isn't going to be on a fast path. As its name might suggest it's a way to have the runtime CPU tell us what features are available so that we can adopt different implementations of perf-sensitive routines for that CPU.
So e.g. maybe the CAD software executes a dozen CPUID instructions during startup, and then based on those it uses AVX512 vectorized operations later on some hardware but uses SSE instead on other hardware.
I haven't read Intel's manual for a modern x86-64 CPU, and I certainly haven't read the community notes about this stuff which would tell you if, despite the documentation you need to behave differently, but my assumption would be that everybody clears ECX if they don't want a non-zero subleaf and that this is either known to be necessary or an obvious way to avoid nasty surprises in code that is never perf critical.
I think we might be talking past each other here. Whether the assembly op is `cpuid` or something else, isn’t really my driving concern. The comment that started this thread (which I’m pretty sure wasn’t yours, so I don’t necessarily expect you to have an answer) was that they didn’t think this example should have compiled because `cpuid` may read from a second register as part of its operations, depending on the value passed to the first register.
But that seems to be saying that the inline assembly template should have the same shape as the underlying assembly calls that are made. If the underlying call might read 3 registers, then the complaint appears to be that the templating system MUST require a template that calls that assembly to also have 3 inputs, even if it won’t use all 3.
My thinking on this is the templating is a syntax for function declarations where the function is assembly code and not more higher level language code. We don’t require that functions have the same number of inputs and outputs that the code called within the function has in order for it to compile, I don’t see why it would be necessary for the assembly templates to work any differently. I get that `cpuid` specifically isn’t likely to run in the hot path of any code, but the underlying principle is the same. If you’re dropping to assembly, you likely have some performance tuning you’re trying to do. A templating system that requires your template inputs to 1:1 map to all the possible inputs of all the assembly calls you use in the template, regardless of wether you need or use them is adding extra instructions and waste for (to me) no obvious benefits.
But we're not suggesting this ridiculous and arbitrary restriction. What we're suggesting, and in fact I think what Bill actually intended in Odin, is something much more useful.
A templating system that requires all possible inputs of all the assembly calls you use in the template are well defined.
What you seem to want is exactly the thing Bill doesn't like about existing functionality. You can cheerfully ADD two registers together and then use the result but without ever determining what's in those registers. What is the result? -shrug- might be anything.
What Odin seems to intend (but this CPUID example seems to suggest is buggy) is that it will check you've actually written code which means something. You do not need to make those registers inputs of your template, you just need to make sure they're well defined. For example you could set them directly in the template, as with your leaf 7 sub-leaf 1 example.
That (modulo bugs) is a big improvement for this particular corner of the language.
I'm starting to get a better handle on what you're getting at, but I still think I disagree that the code shouldn't compile. To really settle this I guess we would need Bill to weigh in on his intent, because I read this article about being all about types and type checking. He says:
That last part in particular is important here, bindings are optional if they're not necessary. Well if the "types" for any given ASM instruction come from their valid definitions, then we'd have to look at the intel instruction definitions[1].Some instructions like ADD[2] (Vol 2A 3-14) are defined to have multiple forms and those forms each take 2 operands. So the type of the instruction would be `ADD T-OP1, T-OP2` for each of the possible combinations of operand types for the ADD forms. And the templating system and compiler would validate that if you pass a given parameter of a given type to one of those operands, that your types match up.
Some like ANDPD (Vol 2A 3-63) have two forms, one that takes 2 operands and one that takes 3. The 2 operand one says that the first operand is a read/write register, where as the 3 operand forms say the first operand is write only. Presumably if you took an input and bound it to the first operand in a 3 operand form, the compiler would at least emit a warning about this if not an error.
But now if we look at the definition of CPUID (Vol 2A 3-203) the only valid form it has is a form with no operands. So if we were defining a type for the type system to compare against, the only possible correct answer (to me) is `cpuid` with no operands. That is the type system should happily allow inline assembly that calls `cpuid` but not inline assembly that calls `cpuid %eax %ecx` because that's not a valid form of the instruction.
Further the definition explicitly says that for some values of EAX, the value in ECX would be ignored entirely, and for invalid values in either field, the output values are all "Reserved". That tells me that as far as a type checker is concerned, any invocation of `cpuid` as long as there are no operands is a valid invocation. A type checker doesn't check the values of the fields being used, and IMO any argument that since it could read ECX, then the type system should enforce you set a value it to it can be equally countered by an argument that since cpuid will explicitly ignore ECX with certain values, it should never require you to set ECX because that might cause you to set it with a value and get an unexpected result because the value you provided to EAX was one of the values that causes it to be ignored. In either case we're asking the type checker to help us prevent a logic bug, not a type bug.
You might say that the "always require both fields to be set" is at least an easy check that could be applied universally, then the question would become how would it intersect with multiple invocations of CPUID? If you invoke:
that is valid and the outcome of that should be (assuming I'm reading the documentation correctly) that after the first invocation, EAX would contain the maximum valid value for EAX when invoking cpuid, and after the second invocation it should contain the results of invoking cpuid with whatever that value was (and obviously whatever was in ECX after the first invocation). If the purpose of the check is to prevent invocations where ECX might contain an arbitrary value that makes no sense, we'd have to mandate that there's some additional steps in between the two cpuid invocations to reset EAX and ECX. Otherwise all we've done is make it possibly even more confusing when ECX changes after the first invocation, but the code strongly implies it should be 0x0.Given the stated goals in the article seem to include not requiring the explicit statement of implicit behavior, requiring setting ECX for instruction that will ignore it would seem counter to the goals.
[1]: https://www.intel.com/content/www/us/en/developer/articles/t...
[2]:
As a side note, I'm curious why (based on your comment about ADD), it would be desirable for a type checker to want to prevent you from doing something like this:
It's perfectly valid to not always care about the starting value in a register even if you're going to use it. Plenty of pseudo-rng type code has read arbitrary registers or addresses as a source for some of their calculation without ever caring what the starting value was. I feel like in some way this gets to the heart of what we're disagreeing about. I read the article as saying "within the bounds of the shape of assembly code as defined by the ISA, odin templates can help enforce types for those shapes and help make wiring normal types to registers for input and output easy. I feel like what you're saying that in addition to that, it's also supposed to help stop you from doing things that are likely to give you non-sensical results.No. Never do this. If you don't care and just want random numbers use for example RDRAND, if you do care, design a proper PRNG with your preferred characteristics and seed management. YOLO programming is a bad idea always but it's especially bad when applied to assembly.
To underscore what this software is for let me quote you a diagnostic it emits:
{instruction} implicitly reads {register}, but nothing in this template produces a value for it; pin an input parameter to {register}, or write {register} before this instruction
I agree you shouldn't do it, that doesn't make it invalid to do.
Well, then I stand corrected on the intent. That pretty clearly spells out that the template checker will validate that (at least for non-branching assembly) all implicitly read registers are given an explicit value and error if not. So that should mean the example in the article actually would not compile, since the rexcode definition lists both RAX and RCX as `implicit_rd`.[1] I appreciate you finding that bit of validation code and persisting in helping me learn something new, this isn't an area of coding I spend a lot of time in so it was instructive.
[1]: Does it compile? I don't have any odin dev stuff set up, but I might just try it in the next little bit just to see.
I don't run nightly Odin. I do happen to a have a non-nightly Odin installed because I was wondering if some of the trash in their kitchen sink library is in fact trash (it is) or whether it's clever in a way I didn't understand ‡
However my reading of the implementation is that:
1. Bill over-sells the value of rexcode. rexcode is Odin code, so the actual "value" derived in Odin's own compiler is just that it scrapes the data out of rexcode. That's not nothing but it's not much.
2. Bill's new template feature doesn't remember that we can pin an output, so it notices that ECX is pinned and concluded it is safe for CPUID to read it. That's a very small bug, and it's even possible I've misunderstood it, but that's my reading.
‡ Odin provides a lot of sorting algorithms. I wanted to measure how fast they are, which I did by comparing against a Rust install on the same toy machine, all of them are much slower than Rust's built-in sorts, but interestingly the provided "slice sort" which is most analogous to Rust's [T]::sort and [T]::sort_unstable is a "Smooth sort" which is relatively obscure and actually is only as slow for pre-sorted input as my Rust sorts are for unsorted input whereas most of what's provided is way slower even for pre-sorted input.
1. I don't think I am overselling rexcode, rather there isn't anything like it in the first place as a single library with that many ISAs and IRs. That's what I love about it, since I can now trivially make (non-optimizing) compilers without needing to use LLVM or use another external tool (including outputting C or assembly). Also all of those tables become binary blobs, so you could just use them in your favourite language any way.
2. That was a bug and it has now been fixed, along with many other analysis bugs.
3. The sorting algorithms are getting an overhauls soon to be a lot faster.