[FFmpeg-devel] [PATCH 06/11] avutil/half2float: adjust conversion of NaN

[Mark Reid]

On Wed, Aug 10, 2022 at 3:56 PM Timo Rothenpieler <timo at rothenpieler.org>
wrote:

> On 11.08.2022 00:37, Mark Reid wrote:
> > On Wed, Aug 10, 2022 at 3:28 PM Timo Rothenpieler <timo at rothenpieler.org
> >
> > wrote:
> >
> >> On 11.08.2022 00:18, James Almer wrote:
> >>> Then maybe the current implementation should be moved back to exr (it
> >>> used to be internal to exr until Paul made it standalone), so this lavu
> >>> module can match the existing hardware implementations of IEEE-734 half
> >>> floats for the purpose of relevant pixel format support.
> >>
> >> That doesn't seem necessary to me.
> >> The values produced before and now are both correct, just different.
> >> But there is no functional difference in the values it produces.
> >>
> >> Duplicating the entirety of that code just for that seems extremely
> >> unnecessary.
> >>
> >
> > openexr does note the intel implementations difference here
> >
> https://github.com/AcademySoftwareFoundation/Imath/blob/main/src/Imath/half.h#L288
>
> It's actually quite curious how that came to be.
> My natural idea would be that our current and EXRs code does it right.
>
> But all hardware as well as gccs software emulation agrees. Makes me
> wonder if it's fully intentional and according to some spec. But I
> couldn't find anything on the matter.
>

Ya I'm curious too now. I might ask the exr folks.
I noticed the difference when I fixed the subnormal bug a couple years ago.
That is why I changed it to match the openexr's halfToFloat() version that
preserves the Nan values instead of changing them.
This new behavior might have been what it was before I changed it
in 8d19b3c4a5.

I looked at the intel architecture developer's manual and sadly it only
describes the float32 to float16 algorithm.
The change back seems pretty benign to me too. The openexr implementation
is relying on the hardware instruction too if it can.
I don't know what one would do with the exact NaN value anyway.


_______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>