The nearest-even rounding mode, aka Banker's Rounding, is the default for IEEE-754 math, so normal programs (which haven't changed the FP environment) on most ISAs including x86 can just use instructions that round a float to an integer-valued float, or convert to integer with the current rounding mode. (Most programs don't change the rounding mode away from the default.)
This is basically just using built-ins, but I didn't want to clutter up the existing answer with a bunch of machine-code stuff and associated interesting stuff I wanted to write about x86. If people want to add sections for other ISAs, feel free, although most other mainstream ISAs have fixed-width instructions so it's always either a 2 or 4-byte instruction.
For the convert-to-integer instructions, out-of-range inputs produce the most-negative 2's complement integer of whatever destination width, which Intel calls the "integer indefinite" value. (e.g. 0x80000000
for 32-bit.) For AVX-512 unsigned conversions, the "integer indefinite" value is all-ones, 0xffffffff
(e.g. from vcvtss2usi
.) Rounding instructions that produce a number in the same format as the input don't have this problem.
x86(-64) machine code, x87 scalar math, 3 bytes
Uses the rounding mode in the x87 control word, which defaults to round-to-nearest w. even as a tie-break. (at reset and finit
, and in fresh processes under normal OSes).
Input arg in st0
, top of the x87 stack, result stored to an int *
pointed-to by RDI (or EDI if the same bytes execute in 32-bit mode). None of the standard calling conventions even for 32-bit mode pass FP args in x87 registers; unlike the SSE part of the answer, you couldn't write a C prototype to describe it to a C compiler.
db 1f fistp DWORD PTR [rdi]
c3 ret
Or to round to an integer-valued long double
(modifying st0
in place), 2-byte frndint
is D9 FC
.
x86(-64) SSE2 or AVX scalar math, 5 bytes
These use the current rounding mode in MXCSR, the SSE math control/status register, which is fully independent of the x87 FP environment. Both use the same 2-bit codes for four different rounding modes, ceil/floor/trunc and nearest-even. (The Python 2 / C round()
rounding mode is not directly available in hardware on x86, so should be avoided for performance reasons unless you specifically want it. IIRC, ARM does have it in hardware, as well as IEEE standard rounding.)
All the relevant SSE / SSE2 / AVX conversion instructions are 4 bytes long, whether encoded as legacy-SSE or AVX, except for conversion into an MMX register. SSE1 cvtps2pi mm0, xmm0
is 0f 2d c0
, allowing for a total 4-byte function if you're willing to return a 32-bit signed integer in the bottom of mm0
, which is a pretty inconvenient place for it and would need the caller to use longer instructions to store or do math with it.
SSE1 ps
instructions are only 3 bytes, but cvtps2dq
involves packed integers so was new in SSE2 along with cvtpd2dq
(packed-double vs. packed-single, converting to a double-quad-word of packed integers). SSE1 scalar instructions have an extra prefix, so SSE1 cvtss2si eax, xmm0
is also 4 bytes, same as SSE2 cvtsd2si eax, xmm0
. An AVX encoding of all of these is available with 2-byte VEX prefixes (plus opcode + modrm byte), again for 4-byte instructions, except there's no cvtps2pi
(MMX destination.)
sse: # first arg in XMM0, return in EAX
f3 0f 2d c0 cvtss2si eax,xmm0 # SSE1
c3 ret
Alternatives: # float vs. double, and scalar vs. packed.
f2 0f 2d c0 cvtsd2si eax,xmm0 # SSE2 scalar double to i32, note same 0f 2d opcode but different prefix from SSE1 scalar.
0f 2d c0 cvtps2pi mm0,xmm0 # SSE1 2 floats to 2 i32 in an MMX reg
66 0f 5b c0 cvtps2dq xmm0,xmm0 # SSE2 packed conversion 4 floats to 4x i32. cvtpd2dq (2 doubles to 2 i32) is the same size
c5 f9 5b c0 vcvtps2dq xmm0,xmm0 # AVX encodings; different bits in the C5 ... prefix represent different combinations of prefixes.
c5 fa 2d c0 vcvtss2si eax,xmm0
c5 fb 2d c0 vcvtsd2si eax,xmm0 ... etc.
(There are also truncating versions of these, like cvttsd2si
, to implement C (int)double
rounding semantics of truncation toward 0 without having to change the rounding mode like in the bad old days of x87 before SSE3 fisttp
.)
Implementing C rint[f]
/ nearbyint[f]
(rounding to an integer-valued double
[or float
]) needs multiple instructions until SSE4.1 roundsd
/ roundps
etc, which are 6-byte instructions (with register args and no REX). e.g. 66 0f 3a 0a c0 00 roundss xmm0,xmm0,0x0
. (roundss and roundsd have separate opcodes, instead of being distinguished by prefixes like SSE1 addss
vs. SSE2 addsd
). The instruction needs an immediate to specify the rounding mode: whether to override or not, and if so, which of the 4 modes to use, and whether to suppress FP inexact (precision) exceptions. See roundsd
documentation. An immediate of 0
selects rounding-control = from the immediate, with the 00
field being the nearest-even mode that's the default.
(Like the "alternatives" above, I'm showing machine code + assembly of multiple instructions. You'd only ever use one, depending on the input format (float or double, scalar vs. packed).)
66 0f 3a 08 c0 00 roundps xmm0,xmm0,0x0 # other than SSE1, the ps form isn't shorter.
66 0f 3a 0a c0 00 roundss xmm0,xmm0,0x0
66 0f 3a 0b c0 00 roundsd xmm0,xmm0,0x0
c4 e3 79 0b c0 00 vroundsd xmm0,xmm0,xmm0,0x0 # 3-byte VEX prefix needed for instructions in opcode "maps" used by SSE4 and later, i.e. with 3 prefixes such as 66 0f 3a
c4 e3 79 08 c0 00 vroundps xmm0,xmm0,0x0 ... etc.
x86 + AVX-512 not relying on the current rounding mode, 7 bytes
AVX-512 EVEX prefixes can override the rounding mode on a per-instruction basis, for any instruction with scalar or 512-bit vector width, including when vaddps
or whatever have to round their result to a representable float/double. (Not 128 or 256-bit vector width until APX and AVX10.2 which finally stops caring about obsolete 32-bit mode so it can use more of the bits of the EVEX prefix, instead of having to slot in to only invalid-in-32-bit-mode encodings of some instruction.) Fun fact: out-of-order exec CPUs that support SMT (e.g. hyperthreading) already have to track the rounding mode on a per-instruction basis internally, because different logical cores could be using different rounding mode settings. So it makes some sense to expose this functionality to programmers.
AVX-512 also allows for conversion directly to unsigned integer, and to packed 64-bit integers, not just scalar-only for conversions involving 64-bit signed integers in SSE/AVX (in 64-bit mode only).
An Intel manual from 2016 when AVX-512 was newish has some details in section 2.5.4 Static Rounding and Suppress All Exceptions, where it compares to instructions like vroundps
and vrndscaleps
which allow this via an immediate, vs. setting MXCSR.
avx512:
62 f1 ff 18 2d c0 vcvtsd2si rax,xmm0{rn-sae}
c3 ret
The NASM syntax I used is vcvtsd2si rax, xmm0, {rn-sae}
. The disassembly is from objdump -drwC -Mintel
, so is using GAS's .intel_syntax noprefix
style for the rounding override.
Using an eax
destination wouldn't save any machine code bytes, unlike with legacy SSE (separate REX prefix), or with AVX where vcvtsd2si rax,xmm0
needs a 3-byte VEX prefix for a total length of 5 bytes instead of 4.
*
and+
, as well as for rounding to integer-valued float with C functions likenearbyint
, or as part of conversion to integer with C functions likelrint
. If you're talking about C'sround()
function which rounds away from 0 as a a tie-break, that's not "standard".) \$\endgroup\$round
method in C/Java/JavaScript) is to always round up (for positive numbers) if the last digit is5
. The first time I heard about nearest-even rounding being commonly used was when I searched why Pythons round was not doing what I expected it to do (round(0.5)
not evaluating to1
). \$\endgroup\$round
is the simpler strategy of rounding away from zero as a tie-break; I guess this strategy was at least formerly considered "usual" by high-level language designers. I'd previously assumed C'sround
function was just a bad ancient design choice, and I've always thought oflrint
andnearbyint
as the normal rounding functions after investigating to find out they were the ones supported by hardware features on CPUs like x86, but I guess I'm weird :P \$\endgroup\$