x86 machine code (MMX/SSE1), 26 bytes (4x int16_t)
x86 machine code (SSE4.1), 28 bytes (4x int32_t or uint32_t)
x86 machine code (SSE2), 24 bytes (4x float32) or 27B to cvt int32
(The last version that converts int32 to float isn't perfectly accurate for large integers that round to the same float. With float input, rounding is the caller's problem and this function works correctly if there are no NaNs, identifying floats that compare == to the max. The integer versions work for all inputs, treating them as signed 2's complement.)
All of these work in 16/32/64-bit mode with the same machine code.
A stack-args calling convention would make it possible to loop over the args twice (finding max and then comparing), possibly giving us a smaller implementation, but I haven't tried that approach.
x86 SIMD has vector->integer bitmap as a single instruction (
movmskps or pd), so it was natural for this even though MMX/SSE instructions are at least 3 bytes long. SSSE3 and later instructions are longer than SSE2, and MMX/SSE1 instructions are the shortest. Different versions of
pmax* (packed-integer vertical max) were introduced at different times, with SSE1 (for mmx regs) and SSE2 (for xmm regs) only having signed word (16-bit) and unsigned byte.
pmaxsw on MMX registers are new with Katmai Pentium III, so really they require SSE1, not just the MMX CPU feature bit.)
This is callable from C as
unsigned max4_mmx(__m64) with the i386 System V ABI, which passes an
__m64 arg in
mm0. (Not x86-64 System V, which passes
line code bytes
1 global max4_mmx
2 ;; Input 4x int16_t in mm0
3 ;; output: bitmap in EAX
4 ;; clobbers: mm1, mm2
6 00000000 0F70C8B1 pshufw mm1, mm0, 0b10110001 ; swap adjacent pairs
7 00000004 0FEEC8 pmaxsw mm1, mm0
9 00000007 0F70D14E pshufw mm2, mm1, 0b01001110 ; swap high/low halves
10 0000000B 0FEECA pmaxsw mm1, mm2
12 0000000E 0F75C8 pcmpeqw mm1, mm0 ; 0 / -1
13 00000011 0F63C9 packsswb mm1, mm1 ; squish word elements to bytes, preserving sign bit
15 00000014 0FD7C1 pmovmskb eax, mm1 ; extract the high bit of each byte
16 00000017 240F and al, 0x0F ; zero out the 2nd copy of the bitmap in the high nibble
17 00000019 C3 ret
size = 0x1A = 26 bytes
If there was a
pmovmskw, what would have saved the
packsswb and the
and (3+2 bytes). We don't need
and eax, 0x0f because
pmovmskb on an MMX register already zeros the upper bytes. MMX registers are only 8 bytes wide, so 8-bit AL covers all the possible non-zero bits.
If we knew our inputs were non-negative, we could
packsswb mm1, mm0 to produce non-negative signed bytes in the upper 4 bytes of
mm1, avoiding the need for
pmovmskb. Thus 24 bytes.
x86 pack with signed saturation treats the input and output as signed, so it always preserves the sign bit. (https://www.felixcloutier.com/x86/packsswb:packssdw). Fun fact: x86 pack with unsigned saturation still treats the input as signed. This might be why
PACKUSDW wasn't introduced until SSE4.1, while the other 3 combinations of size and signedness existed since MMX/SSE2.
Or with 32-bit integers in an XMM register (and
pshufd instead of
pshufw), every instruction would need one more prefix byte, except for
movmskps replacing the pack/and. But
pmaxud need an extra extra byte...
callable from C as
unsigned max4_sse4(__m128i); with x86-64 System V, or MSVC vectorcall (
-Gv), both of which pass
__m128 args in XMM regs starting with
20 global max4_sse4
21 ;; Input 4x int32_t in xmm0
22 ;; output: bitmap in EAX
23 ;; clobbers: xmm1, xmm2
25 00000020 660F70C8B1 pshufd xmm1, xmm0, 0b10110001 ; swap adjacent pairs
26 00000025 660F383DC8 pmaxsd xmm1, xmm0
28 0000002A 660F70D14E pshufd xmm2, xmm1, 0b01001110 ; swap high/low halves
29 0000002F 660F383DCA pmaxsd xmm1, xmm2
31 00000034 660F76C8 pcmpeqd xmm1, xmm0 ; 0 / -1
33 00000038 0F50C1 movmskps eax, xmm1 ; extract the high bit of each dword
34 0000003B C3 ret
size = 0x3C - 0x20 = 28 bytes
Or if we accept input as
float, we can use SSE1 instructions. The
float format can represent a wide range of integer values...
Or if you think that's bending the rules too far, start with a 3-byte
0F 5B C0 cvtdq2ps xmm0, xmm0 to convert, making a 27-byte function that works for all integers that are exactly representable as IEEE binary32
float, and many combinations of inputs where some of the inputs get rounded to a multiple of 2, 4, 8, or whatever during conversion. (So it's 1 byte smaller than the SSE4.1 version, and works on any x86-64 with just SSE2.)
If any of the float inputs are NaN, note that
maxps a,b exactly implements
(a<b) ? a : b, keeping the element from the 2nd operand on unordered. So it might be possible for this to return with a non-zero bitmap even if the input contains some NaN, depending on where they are.
37 global max4_sse2
38 ;; Input 4x float32 in xmm0
39 ;; output: bitmap in EAX
40 ;; clobbers: xmm1, xmm2
42 ; cvtdq2ps xmm0, xmm0
43 00000040 660F70C8B1 pshufd xmm1, xmm0, 0b10110001 ; swap adjacent pairs
44 00000045 0F5FC8 maxps xmm1, xmm0
46 00000048 660F70D14E pshufd xmm2, xmm1, 0b01001110 ; swap high/low halves
47 0000004D 0F5FCA maxps xmm1, xmm2
49 00000050 0FC2C800 cmpeqps xmm1, xmm0 ; 0 / -1
51 00000054 0F50C1 movmskps eax, xmm1 ; extract the high bit of each dword
52 00000057 C3 ret
size = 0x58 - 0x40 = 24 bytes
pshufd is still our best bet:
shufps dst,src,imm8 reads the input for the low half of
dst. And we need a non-destructive copy-and-shuffle both times, so 3-byte
unpckhps/pd are both out. If we were narrowing to a scalar max, we could use those, but it costs another instruction to broadcast before compare if we don't have the max in all elements already.
phminposuw can find the position and value of the minimum
uint16_t in an XMM register. I don't think it's a win to subtract from 65535 to use it for max, but see an SO answer about using it for max of bytes or signed integers.