@casandro @whitequark
I thought we'd have stopped arguing about CPU instruction sets when microarchitectures kinda made them unimportant and even x86 became as fast as its competitors.
Article-length version of this post.
We absolutely haven't. There are a lot of things in an architecture that bias it towards easy or difficult implementations with different microarchitectures.
The memory model on x86 was never really designed. The original 8086 had no caches and could overwrite the next instruction that followed it. That's simple to implement on an in-order microarchitecture with no caches. And software depended on this behaviour. TSO with instruction caches being coherent with data caches imposes a whole bunch of issues. The clearest example we have of the performance impact of TSO is from Apple's chips, because they were designed both to run native code (Arm's weak memory model) and emulated x86 code. They have an option that automatically makes every load into a load-acquire and every store into a store-release, which gives the same effect as the TSO bits. Turning this on gives about a 7% performance hit. They don't bother to try to emulate the instruction-cache consistency in hardware, Rosetta just makes every page containing code read-only and hits (very) slow paths for stores to them. Modern x86 does something similar: stores anywhere in the speculation window flush the pipeline and make everything slow (on a recent Xeon, you can still do a store over the next instruction to do a conditional jump but don't if you care at all about performance).
There's a lot more in x86 that makes it hard to scale. The decoder complexity is legendary. You can avoid much performance cost here by caching decoded micro-ops, but that then costs power for the extra caches.
Given enough money, you can make any ISA fast, but you can't make it both fast and power efficient. Hardware people talk about PPA: power, performance and area are all axes in a trade-off space. But the reachable points in that space depend on the ISA.
The observation that led to RISC was that (on fairly simple in-order pipelined processors in the '80s), the decoder was an enormous part of the total area. CISC chips were spending over 20% of their total area on a decoder. This happened at about the same time as the transition from mostly running hand-written assembly code to mostly running compiled code. Compilers had smallish instruction-selection windows (tree matching engines) and so largely ignored the complex instructions. Most of these were microcoded anyway, so these chips had big microcode engines and RAM (or even RAM!). Removing all of this meant more area to dedicate to functional units.
Microcode got more annoying with superscalar architectures because you want to dispatch a bunch of micro-ops, but now the places where you can take interrupts are more complex. A lot of non-x86 systems that have microcode for infrequent operations basically just stop the pipeline and issue microcode in-order. x86 used to do that too, but it turns out there are too many useful microcoded x86 instructions.
On a high-end microarchitecture, register rename is one of the hottest parts of the chip. At each point in speculation, you're tracking which of a large set of physical registers corresponds to each architectural register. This is closely related to instruction scheduling. Between the two, there's a huge (power / area) cost for executing any instruction, independent of how much work the instruction does.
This means some of x86's early disadvantage in terms of decoder complexity is often offset by doing more work per instruction. Not fully, but it's less of a disadvantage than it was. Doing more work per instruction is the goal if you want a good ISA for superscalar out-of-order pipelines.
To give a simple example, both AArch64 and x86-64 have fairly rich addressing modes. A load or store instruction in each may be equivalent to a short arithmetic sequence followed by a load or store in a purist RISC architecture. And those arithmetic operations each need scheduling and need rename registers allocating for the temporaries.
Krste's answer to all of this is 'micro-op fusion!'. If the compiler generates a sequence of a compressed add / multiple / shift / whatever followed by a load that clobbers the architectural register into a single micro-op that does some arithmetic. Only now you've got a per-implementation ad-hoc variable-length instruction encoding. The compiler needs to know to emit the instruction sequences that get optimised but your decode stage is almost more complex because it has to handle spotting pairs of instructions that can be folded. x86 does this, but no one notices the cost because the x86 front end is already ludicrously complex. For a fixed-length instruction-set front end, it's a big overhead.
Oh, and it doesn't actually solve the problem. Loads can fault. If they do, you are required by the architecture to have the values in the registers from before the load. And that means that you either need to allocate a rename register for the intermediate (you know, the thing you were trying to avoid), or you need to track the corresponding architectural register so that the load/store unit can write back the temporary value if it faults. All of that consumes power and area to achieve the same performance that a richer addressing mode in an instruction could do.
RISC-V makes it hard to add additional instructions because they've burned so much encoding space on things that aren't that helpful for complex microarchitectures. The C extension uses a huge amount of the 16-bit encoding space and is 100% duplication: every C instruction was designed to be expanded to an equivalent I instruction, so this is pure redundancy. If you design this in from the start, you can remove those cases from the 32-bit encodings and free up more space there.
The JAL instruction in RISC-V is enormous for the sole reason of avoiding marking the link register architectural for purity reasons. Except that, if you want a branch predictor that isn't terrible, you need to treat it as architectural. So it has a full 5-bit link-register field. This gives it 26 bits of operand space where any other RISC encoding (which just uses one bit to discriminate between jump and jump-and-link cases) would need 22 for the same instruction. With 26 bits of operand space, it's an entire major opcode: 1/128 of the total encoding space for 32-bit instructions. Actually more, because the RISC-V encoding requires two bits to have fixed values for 32-bit instructions, so it's 1/32 of the entire available 32-bit encoding space. For a single instruction.
To put this in perspective, AArch64 uses about as big a proportion of its encoding space for all memory instructions with all of its rich addressing modes as RISC-V uses for JAL.
I like to pick on x86's encoding because the long encodings are ludicrous but the shorter ones are actually not terrible. Most instructions are two-operand destructive operations. Compilers are really good at generating those now (in part because x86 performance is really bad if they aren't). In some measurement we did when we were designing a custom ISA at Microsoft, we found that around 2/3 of instructions had exactly one use of the result (around 2/3 of instructions were used only in the same basic block as well, but annoyingly not the same 2/3). Almost all of these can use two-operand encodings, which can be a huge space saving. If you disassemble an x86 binary, you might be surprised at the number of 16-bit instructions.
