技术背景
在我的blog里面，最近很少有提到iOS平台RTMP推送|轻量级RTSP服务和RTMP|RTSP直播播放模块，实际上，我们在2016年就发布了iOS平台直播推拉流、转发模块，只是因为传统行业，对iOS的需求比较少，所以一直没单独说明，本文主要介绍下，如何在iOS平台播放RTMP或RTSP流。
技术实现
先说播放实现，iOS端，RTMP|RTSP直播播放，我们实现的功能如下：
- [支持播放协议]高稳定、超低延迟（毫秒级）
- [多实例播放]支持多实例播放；
- [事件回调]支持网络状态、buffer状态等回调；
- [视频格式]支持RTMP扩展H.265，H.264；
- [音频格式]支持AAC/PCMA/PCMU/Speex；
- [H.264/H.265软解码]支持H.264/H.265软解；
- [H.264硬解码]Windows/Android/iOS支持特定机型H.264硬解；
- [H.265硬解]Windows/Android/iOS支持特定机型H.265硬解；
- [H.264/H.265硬解码]Android支持设置Surface模式硬解和普通模式硬解码；
- [缓冲时间设置]支持buffer time设置；
- [首屏秒开]支持首屏秒开模式；
- [低延迟模式]支持低延迟模式设置(公网200~400ms)；
- [复杂网络处理]支持断网重连等各种网络环境自动适配；
- [快速切换URL]支持播放过程中，快速切换其他URL，内容切换更快；
- [实时静音]支持播放过程中，实时静音/取消静音；
- [实时音量调节]支持播放过程中实时调节音量；
- [实时快照]支持播放过程中截取当前播放画面；
- [渲染角度]支持0°，90°，180°和270°四个视频画面渲染角度设置；
- [渲染镜像]支持水平反转、垂直反转模式设置；
- [等比例缩放]支持图像等比例缩放绘制(Android设置surface模式硬解模式不支持)；
- [实时下载速度更新]支持当前下载速度实时回调(支持设置回调时间间隔)；
- [解码前视频数据回调]支持H.264/H.265数据回调；
- [解码后视频数据回调]支持解码后YUV数据回调；
- [解码前音频数据回调]支持AAC/PCMA/PCMU/SPEEX数据回调；
- [音视频自适应]支持播放过程中，音视频信息改变后自适应；
- [扩展录像功能]完美支持和录像SDK组合使用。
下面，我们看看技术实现细节，先说开始播放逻辑：
代码语言：objectivec复制//
// ViewController.m
// SmartiOSPlayerV2
//
// Author: daniusdk.com
// Created by daniulive on 2016/01/03.
//
- (void)playBtn:(UIButton *)button {
NSLog(@"playBtn only ");
button.selected = !button.selected;
if (button.selected)
{
if(is_playing_)
return;
[self InitPlayer];
//如需处理回调的用户数据
__weak __typeof(self) weakSelf = self;
_smart_player_sdk.spUserDataCallBack = ^(int data_type, unsigned char *data, unsigned int size, unsigned long long timestamp, unsigned long long reserve1, long long reserve2, unsigned char *reserve3)
{
[weakSelf OnUserDataCallBack:data_type data:data size:size timestamp:timestamp reserve1:reserve1 reserve2:reserve2 reserve3:reserve3];
};
Boolean enableUserDataCallback = YES;
[_smart_player_sdk SmartPlayerSetUserDataCallback:enableUserDataCallback];
//如需处理回调的用户数据---------
if(![self StartPlayer])
{
NSLog(@"Call StartPlayer failed..");
}
[playbackButton setTitle:@"停止播放" forState:UIControlStateNormal];
is_playing_ = YES;
}
else
{
if ( !is_playing_ )
return;
[self StopPlayer];
if(!is_recording_)
{
[self UnInitPlayer];
}
[playbackButton setTitle:@"开始播放" forState:UIControlStateNormal];
is_mute_ = NO;
[muteButton setTitle:@"实时静音" forState:UIControlStateNormal];
is_playing_ = NO;
}
}
其中，InitPlayer实现如下：
代码语言：objectivec复制-(bool)InitPlayer
{
NSLog(@"InitPlayer ");
if(is_inited_player_)
{
NSLog(@"InitPlayer: has inited before..");
return true;
}
//NSString* in_cid = @"";
//NSString* in_key = @"";
//[SmartPlayerSDK SmartPlayerSetSDKClientKey:in_cid in_key:in_key reserve1:0 reserve2:nil];
_smart_player_sdk = [[SmartPlayerSDK alloc] init];
if (_smart_player_sdk ==nil ) {
NSLog(@"SmartPlayerSDK init failed..");
return false;
}
if (playback_url_.length == 0) {
NSLog(@"playback url is nil..");
return false;
}
if (_smart_player_sdk.delegate == nil)
{
_smart_player_sdk.delegate = self;
NSLog(@"SmartPlayerSDK _player.delegate:%@", _smart_player_sdk);
}
NSInteger initRet = [_smart_player_sdk SmartPlayerInitPlayer];
if ( initRet != DANIULIVE_RETURN_OK )
{
NSLog(@"SmartPlayerSDK call SmartPlayerInitPlayer failed, ret=%ld", (long)initRet);
return false;
}
[_smart_player_sdk SmartPlayerSetPlayURL:playback_url_];
//[self try_set_rtsp_url:playback_url_];
//超低延迟模式设置
[_smart_player_sdk SmartPlayerSetLowLatencyMode:(NSInteger)is_low_latency_mode_];
//buffer time设置
if(buffer_time_ >= 0)
{
[_smart_player_sdk SmartPlayerSetBuffer:buffer_time_];
}
//快速启动模式设置
[_smart_player_sdk SmartPlayerSetFastStartup:(NSInteger)is_fast_startup_];
NSLog(@"[SmartPlayerV2]is_fast_startup_:%d, buffer_time_:%ld", is_fast_startup_, (long)buffer_time_);
//RTSP TCP还是UDP模式
[_smart_player_sdk SmartPlayerSetRTSPTcpMode:is_rtsp_tcp_mode_];
//设置RTSP超时时间
NSInteger rtsp_timeout = 10;
[_smart_player_sdk SmartPlayerSetRTSPTimeout:rtsp_timeout];
//设置RTSP TCP/UDP自动切换
NSInteger is_tcp_udp_auto_switch = 1;
[_smart_player_sdk SmartPlayerSetRTSPAutoSwitchTcpUdp:is_tcp_udp_auto_switch];
//快照设置 如需快照 参数传1
[_smart_player_sdk SmartPlayerSaveImageFlag:save_image_flag_];
//如需查看实时流量信息，可打开以下接口
NSInteger is_report = 1;
NSInteger report_interval = 3;
[_smart_player_sdk SmartPlayerSetReportDownloadSpeed:is_report report_interval:report_interval];
//录像端音频，是否转AAC后保存
NSInteger is_transcode = 1;
[_smart_player_sdk SmartPlayerSetRecorderAudioTranscodeAAC:is_transcode];
//录制MP4文件 是否录制视频
NSInteger is_record_video = 1;
[_smart_player_sdk SmartPlayerSetRecorderVideo:is_record_video];
//录制MP4文件 是否录制音频
NSInteger is_record_audio = 1;
[_smart_player_sdk SmartPlayerSetRecorderAudio:is_record_audio];
is_inited_player_ = YES;
NSLog(@"InitPlayer--");
return true;
}停止播放StopPlayer实现如下：
代码语言：objectivec复制-(bool)StopPlayer
{
NSLog(@"StopPlayer ");
if (_smart_player_sdk != nil)
{
[_smart_player_sdk SmartPlayerStop];
}
if (!is_audio_only_) {
if (_glView != nil) {
[_glView removeFromSuperview];
[SmartPlayerSDK SmartPlayeReleasePlayView:(__bridge void *)(_glView)];
_glView = nil;
}
}
NSLog(@"StopPlayer--");
return true;
}UnInitPlayer实现如下：
代码语言：objectivec复制-(bool)UnInitPlayer
{
NSLog(@"UnInitPlayer ");
if (_smart_player_sdk != nil)
{
[_smart_player_sdk SmartPlayerUnInitPlayer];
if (_smart_player_sdk.delegate != nil)
{
_smart_player_sdk.delegate = nil;
}
_smart_player_sdk = nil;
}
is_inited_player_ = NO;
NSLog(@"UnInitPlayer--");
return true;
}实时录像：
代码语言：objectivec复制- (void)RecorderBtn:(UIButton *)button {
NSLog(@"record Stream only ");
button.selected = !button.selected;
if (button.selected)
{
if(is_recording_)
return;
[self InitPlayer];
//设置录像目录
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *recorderDir = [paths objectAtIndex:0];
if([_smart_player_sdk SmartPlayerSetRecorderDirectory:recorderDir] != DANIULIVE_RETURN_OK)
{
NSLog(@"Call SmartPlayerSetRecorderDirectory failed..");
}
//每个录像文件大小
NSInteger size = 200;
if([_smart_player_sdk SmartPlayerSetRecorderFileMaxSize:size] != DANIULIVE_RETURN_OK)
{
NSLog(@"Call SmartPlayerSetRecorderFileMaxSize failed..");
}
[_smart_player_sdk SmartPlayerStartRecorder];
[recButton setTitle:@"停止录像" forState:UIControlStateNormal];
is_recording_ = YES;
}
else
{
[_smart_player_sdk SmartPlayerStopRecorder];
[recButton setTitle:@"开始录像" forState:UIControlStateNormal];
if(!is_playing_)
{
[self UnInitPlayer];
}
is_recording_ = NO;
}
}实时快照：
代码语言：objectivec复制- (void)SaveImageBtn:(UIButton *)button {
if ( _smart_player_sdk != nil )
{
//设置快照目录
NSLog(@"[SaveImageBtn] path ");
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *saveImageDir = [paths objectAtIndex:0];
NSLog(@"[SaveImageBtn] path: %@", saveImageDir);
NSString* symbol = @"/";
NSString* png = @".png";
// 1.创建时间
NSDate *datenow = [NSDate date];
// 2.创建时间格式化
NSDateFormatter *formatter = [[NSDateFormatter alloc] init];
// 3.指定格式
formatter.dateFormat = @"yyyyMMdd_HHmmss";
// 4.格式化时间
NSString *timeSp = [formatter stringFromDate:datenow];
NSString* image_name = [saveImageDir stringByAppendingString:symbol];
image_name = [image_name stringByAppendingString:timeSp];
image_name = [image_name stringByAppendingString:png];
NSLog(@"[SaveImageBtn] image_name: %@", image_name);
[_smart_player_sdk SmartPlayerSaveCurImage:image_name];
}
}Event回调处理如下：
代码语言：objectivec复制- (NSInteger) handleSmartPlayerEvent:(NSInteger)nID param1:(unsigned long long)param1 param2:(unsigned long long)param2 param3:(NSString*)param3 param4:(NSString*)param4 pObj:(void *)pObj;
{
NSString* player_event = @"";
NSString* lable = @"";
if (nID == EVENT_DANIULIVE_ERC_PLAYER_STARTED) {
player_event = @"[event]开始播放..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTING)
{
player_event = @"[event]连接中..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED)
{
player_event = @"[event]连接失败..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTED)
{
player_event = @"[event]已连接..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED)
{
player_event = @"[event]断开连接..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP)
{
player_event = @"[event]停止播放..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO)
{
NSString *str_w = [NSString stringWithFormat:@"%ld", (long)param1];
NSString *str_h = [NSString stringWithFormat:@"%ld", (long)param2];
lable = @"[event]视频解码分辨率信息: ";
player_event = [lable stringByAppendingFormat:@"%@*%@", str_w, str_h];
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED)
{
player_event = @"[event]收不到RTMP数据..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL)
{
player_event = @"[event]快速切换url..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE)
{
if ((int)param1 == 0)
{
NSLog(@"[event]快照成功: %@", param3);
lable = @"[event]快照成功:";
player_event = [lable stringByAppendingFormat:@"%@", param3];
tmp_path_ = param3;
image_path_ = [ UIImage imageNamed:param3];
UIImageWriteToSavedPhotosAlbum(image_path_, self, @selector(image:didFinishSavingWithError:contextInfo:), NULL);
}
else
{
lable = @"[event]快照失败";
player_event = [lable stringByAppendingFormat:@"%@", param3];
}
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE)
{
lable = @"[event]录像写入新文件..文件名:";
player_event = [lable stringByAppendingFormat:@"%@", param3];
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED)
{
lable = @"一个录像文件完成..文件名:";
player_event = [lable stringByAppendingFormat:@"%@", param3];
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING)
{
//NSLog(@"[event]开始buffer..");
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_BUFFERING)
{
NSLog(@"[event]buffer百分比: %lld", param1);
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING)
{
//NSLog(@"[event]停止buffer..");
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED)
{
NSInteger speed_kbps = (NSInteger)param1*8/1000;
NSInteger speed_KBs = (NSInteger)param1/1024;
lable = @"[event]download speed :";
player_event = [lable stringByAppendingFormat:@"%ld kbps - %ld KB/s", (long)speed_kbps, (long)speed_KBs];
}
else if(nID == EVENT_DANIULIVE_ERC_PLAYER_RTSP_STATUS_CODE)
{
lable = @"[event]RTSP status code received:";
player_event = [lable stringByAppendingFormat:@"%ld", (long)param1];
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
dispatch_async(dispatch_get_main_queue(), ^{
UIAlertController *aleView=[UIAlertController alertControllerWithTitle:@"RTSP错误状态" message:player_event preferredStyle:UIAlertControllerStyleAlert];
UIAlertAction *action_ok=[UIAlertAction actionWithTitle:@"确定" style:UIAlertActionStyleCancel handler:nil];
[aleView addAction:action_ok];
[self presentViewController:aleView animated:YES completion:nil];
});
});
}
else if(nID == EVENT_DANIULIVE_ERC_PLAYER_NEED_KEY)
{
player_event = @"[event]RTMP加密流，请设置播放需要的Key..";
}
else if(nID == EVENT_DANIULIVE_ERC_PLAYER_KEY_ERROR)
{
player_event = @"[event]RTMP加密流，Key错误，请重新设置..";
}
else
NSLog(@"[event]nID:%lx", (long)nID);
NSString* player_event_tag = @"当前状态:";
NSString* event = [player_event_tag stringByAppendingFormat:@"%@", player_event];
if ( player_event.length != 0)
{
NSLog(@"%@", event);
}
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
dispatch_async(dispatch_get_main_queue(), ^{
self.textPlayerEventLabel.text = event;
});
});
return 0;
}总结
iOS平台播放，由于设备和系统比较单一，所以优先考虑硬解码，除了基础播放外，我们还实现了实时快照、实时录像、实时回调YUV数据、实时音量调节等，实际体验下来，iOS平台RTMP和RTSP，可以轻松毫秒级，感兴趣的开发者，可以和我单独交流。
