腾讯云对FLV协议部分做了扩展，用于支持FLV AV1的直播场景，并在FFmpeg 4.1.3/4.2中做了对应的实现，具体细节如下：
针对最新的FLV标准，VIDEODATA部分扩展如下：
参考文档：
https://www.adobe.com/content/dam/acom/en/devnet/flv/video_file_format_spec_v10_1.pdf
VIDEODATA
The VideoTagHeader and VideoTagBody contains video-specific metadata/the video frame payload.
VideoTagHeader & VideoTagBody
Field
Type
Comment
Frame Type
UB [4]
Type of video frame. The following values are defined:
1 = key frame (for AVC and HEVC, a seekable frame)
2 = inter frame (for AVC and HEVC, a non-seekable frame)
3 = disposable inter frame (H.263 only)
4 = generated key frame (reserved for server use only)
5 = video info/command frame
CodecID
UB [4]
The following values are defined:
2 = Sorenson H.263
3 = Screen video
4 = On2 VP6
5 = On2 VP6 with alpha channel
6 = Screen video version 2
7 = AVC
13 = AV1
AV1PacketType
IF CodecID == 13
UI8
The following values are defined:
0 = Sequence Header
1 = OBU Frame(Full frames are required)
CompositionTime
IF CodecID==7 OR CodecID == 13
SI24
IF AVCPacketType == 1 OR AV1PacketType == 1
Composition time offset
ELSE
See ISO 14496-12, 8.15.3 for an explanation of composition
times. The offset in an FLV file is always in milliseconds.
VideoTagBody
IF FrameType == 5
UI8
ELSE (
IF CodecID == 2
H263VIDEOPACKET
IF CodecID == 3
SCREENVIDEOPACKET
IF CodecID == 4
VP6FLVVIDEOPACKET
IF CodecID == 5
VP6FLVALPHAVIDEOPACKET
IF CodecID == 6
SCREENV2VIDEOPACKET
IF CodecID == 7
AVCVIDEOPACKET
IF CodecID == 13 AV1VIDEOPACKET )
Video frame payload or frame info
If FrameType == 5, instead of a video payload, the
Video Data Body contains a UI8 with the following
meaning:
0 = Start of client-side seeking video frame
sequence
1 = End of client-side seeking video frame
sequence
For all but AVCVIDEOPACKET or AV1VIDEOPACKET,
see the SWF File Format Specification for details
See AV1 Codec ISO Media File Format Binding v1.2.0, 12 December 2019 for the description of AV1CodecConfigurationBox. This contains the same information that would be stored in av1C box in an MP4/FLV file.
AV1CodecConfigurationBox defined in AV1 Codec ISO Media File Format Binding v1.2.0, 12 December 2019 as follow:
参考文档：https://aomediacodec.github.io/av1-isobmff/
class AV1CodecConfigurationBox extends Box('av1C'){
AV1CodecConfigurationRecord av1Config; } aligned (8) class AV1CodecConfigurationRecord { unsigned int (1) marker = 1; unsigned int (7) version = 1; unsigned int (3) seq_profile; unsigned int (5) seq_level_idx_0; unsigned int (1) seq_tier_0; unsigned int (1) high_bitdepth; unsigned int (1) twelve_bit; unsigned int (1) monochrome; unsigned int (1) chroma_subsampling_x; unsigned int (1) chroma_subsampling_y; unsigned int (2) chroma_sample_position; unsigned int (3) reserved = 0; unsigned int (1) initial_presentation_delay_present; if (initial_presentation_delay_present) { unsigned int (4) initial_presentation_delay_minus_one; } else { unsigned int (4) reserved = 0; } unsigned int (8)[] configOBUs; }
关于FLV/AV1扩展patch的使用说明
获取patch并应用
· 获取FFmpeg 代码:
git clone https://github.com/FFmpeg.git FFmpeg
· 按照如下应用patch:
cd FFmpeg
git reset --hard origin/release/4.2 git am 0001-The-FLV-extensions-for-AV1.patch
以上即应用腾讯云关于AV1 FLV的扩展patch到FFmpeg 4.2
patch说明
涉及改动以下文件：
- libavcodec/av1_parse.c
- libavcodec/av1_parse.h
- libavcodec/cbs_av1.c
- libavcodec/cbs_av1.h
- libavform/flv.h
- libavform/flvenc.c
- libavform/flvdec.c
在关于H.264的分支判断处，加入了AV1判断，并支持了av1C的解析。具体协议细节说明请见上面的文档部分。下图是一个FLV AV1扩展示例：
由于目前FFmpeg 并无原生的AV1 decoder（需要依赖dav1d 或者libaom），在解析extradata部分有所缺失，为了支持FLV AV1， IVF AV1 等格式的转换，我们也增加就AV1 extradata部分的支持。在此基础上，推拉流的基本流程与原FLV h264/h265并未特别差异，如下所示：
Note：AV1 bit stream目前支持两种格式
Low overhead bit stream format // 低开销 bit stream格式
Length delimited bit stream format //长度分割bit stream格式
目前默认支持low overhead bitstream 格式obu，length delimited bit stream格式会造成一定浪费，因此建议用low overhead 格式。
参考文档
FLV Spec：https://www.adobe.com/content/dam/acom/en/devnet/flv/video_file_format_spec_v10_1.pdf
ISOBMFF:
Information technology — Coding of audio-visual objects — Part 12: ISO Base Media File Format. December 2015. International Standard.
URL: http://standards.iso.org/ittf/PubliclyAvailableStandards/c068960_ISO_IEC_14496-12_2015.zip
AV1 ISOBMFF：
https://aomediacodec.github.io/av1-isobmff/
扫描上方二维码，关注腾讯云视频云，了解视频产品更多信息~
