The setup macro should be in init function, not in configure
output function. Current placment would break scale_vt based
tone mapping as the output color parameters are never initialized
and cannot be mapped correctly.
We previously used the spec convention of Lw=1000 and gamma = 1.2
for HLG EOTF to linear space, but that mapping works terribly with
common tonemap operators as that would map too many of the area
above diffusion white and make the result overly bright.
In ITU report BT.2446-1 page 15, it recommends to use Lw=291 with
a gamma of 1.03 so that the mapped HLG diffusion white is at around
90% of SDR peak of BT.1886, or around 78nit. This compress most
of the signal into near SDR range before we apply any tonemap
operators, and it shows promising results.
This approach also requires least code change as we only need to
modify EOTF of HLG for each implementations.
Signed-off-by: gnattu <gnattuoc@me.com>
Signed-off-by: nyanmisaka <nst799610810@gmail.com>
only_first_vframe originally waited for the first decoded frame from every
video stream. That breaks on files with multiple video streams where one
target stream never reaches "first frame seen" in this path.
For example, a Matroska file with one real HEVC video stream and two
attached-picture streams (PNG and JPEG). With -threads 1, all three streams
were marked finished quickly. With -threads 0, the attached PNG stream was
opened with auto threading, its only packet was consumed, but
avcodec_receive_frame() returned EAGAIN and no frame was emitted. Since that
stream had no second packet, it never became finished. ffprobe then kept
demuxing the rest of the file while waiting for all target video streams to
finish, which could read essentially to EOF and appear to consume unbounded
memory.
Fix this in two ways:
- exclude attached-picture streams from the only_first_vframe target set, so
the mode only waits for real video streams
- change the safety fallback from counting packets for unresolved target
streams to counting overall scanned packets, so even unknown future cases
still terminate instead of scanning forever
This preserves the intended "first real video frame" behavior while making the
mode safe for problematic multi-stream inputs.
The only_firsrt_vframe option was broken when a video stream is sparse,
starts late, is corrupt, or keeps feeding non-decodable packets.
My test videos are too good to not revealing this, but our users would
have files that is in an unideal state which make this option to read
forever untill EOF, and when the file is huge enough or a livestream,
This option would hang the probing for a long time.
This adds a few safeguard to bail out after a reasonable amount of
packets have being decoded.
Some video sources are not reliable and will contain frames that
is not decodable with hardware accelerators. Current ffmpeg's
scheduling has low tolarance for decoder errors it does not understand.
Just return 0 after unref the frame instead so that ffmpeg will no
longer abort early due to videotoolbox is not able to decode a frame.
avfilter/tonemap_opencl: vectorize Dolby Vision reshaping
Vectorize the Dolby Vision reshaping process to handle 4 pixels at a time, similar
to what is used in the tone mapping stage. This change allows us to read the dovi
buffer once and reuse parameters for 4 pixels which reduces memory load.
Most of the pipeline, including the common polynomial reshaping, benefits from
vectorization. The only exceptions are the coefficients lookup for intensity channel
and MMR reshaping: intensity lookup remains scalar because it processes 4 vec4
values that must be handled sequentially, and MMR reshaping cannot be vectorized
due to its inter-channel dot products.
With this optimization, 4K60 Dolby Vision tone mapping is now reliably achievable
on the RK3588.
This is requried for GPU platforms with unideal compilers like
Qualcomm GPUs on Windows. When the compiler unrolls the IPT
reshaping loop badly the performance suffers a lot. Unroll it by
hand mitigates this, and I noticed no performance difference on
arm mali.
The global static nb_streams is dynamic and can be extended to a
number that would cause out of bounds access. Always use the
nb_streams from the input file structure and stop rescaling our
checking bitmask with the dynamic nb_streams. This should still
work for all video files.
The orignial logic ported from the GPU shader uses (fused) multiply
and add to compute the dovi coefficients. But we are not actually
blending the values, and we are selecting all lanes based on one
boolean condition, a simple conditional assignment is much faster
on CPU.
This commit also uses casting instead of extraction to get the low 128b
for the avx code path. Casting is free but extraction is an instruction
with 3-cycle latency.
The old capping will use the second brightest cube instead of the
brightest one. The new capping will always use the brightest cube
and the values will be cancelled out during calculation, and only
the raw value of the brightest corner will left.
This is slightly slower as it introdced two more scalar ops, but
should still be fast enough.
For certain video inputs, the dolby vision metadata could be
missing on the first few frames which would cause the filter to
re-initialize itself for the first frame containing the dovi metadata.
This re-init behavior would mistakenly re-interpret the current
set param as user input which will create unwanted results due to
wrong tone mapping parameters. Fix this by using a separate value
for calculated param and never change the user input.
This reworks the current tradeoff mode, which used to be a single
1D LUT for linearization, which is not fast enough on slow GPUs.
Instead, this makes the entire tonemap process a 3D LUT lookup.
This implementation first generates a 65x65x65 LUT on the GPU,
which has a similar compute cost to a single 400p frame. Then, it
uses tetrahedral interpolation to apply the LUT to the actual frame.
The interpolation quality is quite decent, and the interpolation
errors are very hard to notice unless in extreme conditions.
Signed-off-by: gnattu <gnattuoc@me.com>
Currently, ffprobe can only read frames at a specific interval,
and users have no options to select only video frames without
explicitly selecting a stream. This option instructs show_frames
to pick the first video frame and prints its information.
This will be useful for extracting tricky metadata, such as the
HDR10plus ST2094 metadata.
In rare edge cases the looping removal might leave some frames
with dovi metadata not removed for hevc. Always remove at the
end of au can also improve performance, and this is what the
upstream dovi_rpu is doing as well.
Unlike on x86, where libopus provides inline assembly, most of the
performance optimizations for ARM Neon are implemented in the NE10
library. We need to build it separately for optimal performance on
arm64 targets.
This adds a reshape-only mode for Dolby Vision videos without
a compatibility layer. In this mode, only Dolby Vision reshaping
will be performed, and the output will still be in SMPTE 2084
transfer.
The GPU-based filters already support this mode. This will be
useful in the future when we implement HDR transcoding.
Stop trying to invent some "framerate-based" timebase when there is no
reason to think the stream is CFR at all.
This is a backport of upstream commit 10185e2d4c1e9839bc58a1d6a63c861677b13fd0
The `par->extradata` buffer filled from some matroska files may be
null terminated, and use `ffio_write_lines` using the full buffer
length will copy this null character into the output files. This
results in a file in which there is a null terminator after the
header, but preceeding the actual content of the subtitle file.
Treat this buffer as a string and write line with the text length of this buffer to skip the null character.
Regression from 7bf1b9b
We are not using OpenGL and this compatability key could introduce
performance penalty on some Macs. Performance difference is neglible
on Apple Silicon but on old Intel Macs like the 2018 MacBook Air,
decoding performance could be reduced by 15% as the IOSurface might
make extra frame copies to reshape the pixel buffer.
Both AV_CHANNEL_LAYOUT_7POINT1_WIDE_BACK and AV_CHANNEL_LAYOUT_7POINT1
should be mapped to kAudioChannelLayoutTag_AAC_7_1_B, otherwise neither
the back surround channel nor the side surround channel has sound.
aac_at has stricter sample rate than other encoders which should be
spcified to enable auto resampling.
alac with sample rate higher than 38400 will not play on Apple devices,
although higher sample rates are accepted by the encoder
All encoders provided by AudioToolbox can handle 32bit integer samples,
enable that for all encoders.
No observable performance difference observed, and some compilers
even generate the same instructions for mla and fma intrinsics.
This is just a cleanup to always use fma for float32 for consistency.
Xcode 16 marks VTPixelRotationSessionRef as macOS 13+ only type as well.
As it is technically only a pointer, use type void* in the struct and cast to VTPixelRotationSessionRef
when using.
Apple disabled hardware decoding for some h264 files with certain
condition and now ffmpeg will error out for such inputs because the
software fallback is disabled. Allow software fallback for all codecs
instead of only for HEVC to workaround this as the error handling
outside ffmpeg would be harder. Allowing software fallback has no
measurable performance impact when the hardware decoder is not
overloaded.
gnattu